Communication — System architecture

Clear boundary: logic (Platform/automation) decides what to send; dispatch ensures records exist; the send pipeline handles actual delivery. Automation handlers never call providers directly.

Overview

In one sentence: events and schedules from the left flow into NotificationScheduler; the worker pipeline (Scheduler → Generator → Distributor) renders and sends; webhook updates state back to OutgoingMessage.

Choose send path by producer

Producer	Path
Manual / immediate entity message	NotificationJob → NotificationScheduler
Direct reminder	NotificationJob → NotificationScheduler → OutgoingMessage
UI-only reminder	NotificationJob → NotificationScheduler → UserMessage
Report subscription	AutomationScheduler → MessagingJob+Blob and/or NotificationJob → NotificationScheduler → OutgoingMessage
Messaging campaign	MessagingJob+Blob → NotificationScheduler → OutgoingMessage
Automation skipped/deactivated	No send; result is in AutomationScheduler.DispatchResultJson + support screens

Ensure contracts (IDeliveryDispatcher)

Every artifact is created through an Ensure* contract:

IDeliveryDispatcher
  EnsureAutomationJobAsync       EnsureNotificationJobAsync
  EnsureMessagingJobAsync        EnsureUserMessageAsync
  EnsureOutgoingMessageAsync     EnsureNotificationSchedulerAsync

Each method: find by deterministic key → check compatibility (InputHash/ContentHash + immutable fields) → create if missing → return stable coordinates. Compatible → idempotent success; immutable mismatch → conflict (stop sending, report support).

Key policy (short version)

Keys decide "where history opens" and "how duplicates are prevented".

NotificationJob

Manual / transactional:     PartitionKey = PrimaryEntity.Guid
                            RowKey       = {CreatedReverseTicks}_{NotificationJobGuid}
Direct automation reminder: PartitionKey = PrimaryEntity.Guid
                            RowKey       = AutomationRowKey
Report subscription:        PartitionKey = ReportSubscription.Guid
                            RowKey       = AutomationRowKey
Messaging campaign:         PartitionKey = {BusinessUnit.Guid}_Campaign
                            RowKey       = Campaign.Guid

PartitionKey is the entity where the user expects to open history — not the internal id of automation. RowKey = AutomationRowKey is the idempotency bridge from automation to notification. Event-derived notifications use NotificationEventKey (see Platform).

UserMessage

Customer: PartitionKey = user:{User.Id}
Staff:    PartitionKey = employee:{Employee.Guid}
RowKey    = {SourceReverseTicks}_{NotificationJob.RowKey}

Prefixes user: / employee: keep identity domains separate. SourceReverseTicks must come from a stable source timestamp (not write-time) so Generator retry does not create duplicate rows.

OutgoingMessage + send claim

Notification-generated: PartitionKey = NotificationJob.RowKey
Report/campaign:        PartitionKey = MessagingJob.Guid
RowKey = {RecipientKeyHash}_{DeliveryMethod}   (+ _{AttemptNo} only when intentionally a new attempt)

RecipientKeyHash is derived from stable recipient identity + normalized destination, not display text. One row / recipient / channel; Generator retry reuses the row.

Claim before provider call:

Created/RetryReady -> Sending  WHERE PartitionKey, RowKey, Status in (Created, RetryReady)

Rows already Sending | SentToProvider | Delivered | Opened | FailedFinal → duplicate queue message exits without calling provider. Provider success → SentToProvider + provider message id; webhook advances state. Only audited support-retry may reset to RetryReady.

NotificationScheduler dispatch guard

UNIQUE(ArtifactTypeId, ArtifactPartitionKey, ArtifactRowKey, NotificationEventTypeId)

EnsureNotificationSchedulerAsync returns the existing row when the guard matches. NotificationEventTypeId is the domain notification discriminator — mapped explicitly at the notification boundary, not assumed equal to the EntityEvent EventTypeId.

Partial failure handling

Found during retry	Behavior
NotificationJob exists, scheduler missing	Recreate scheduler through dispatch guard
UserMessage missing recipient row	Insert missing row with deterministic key
OutgoingMessage already exists	Reuse; provider call is decided by send claim
MessagingJob exists, blob incompatible	Stop, report conflict
Any key hash mismatch	Stop, report conflict

SMS usage ledger (billing)

• Each SMS accepted by a provider writes exactly one SmsUsage in SQL, append-only — aggregatable by BusinessUnit × time range (monthly billing) without scanning OutgoingMessage.
• The record carries BusinessUnitId, NotificationEventTypeId (assignable to Messaging Campaign / Incident / Onsite App SMS), SentOn (UTC at provider accept), MessageCount (=1), SmsCount.
• SmsCount = the number of segments the provider bills: provider-reported count → record that exact value; none → derive by encoding rules (GSM-7: 160/153; UCS-2: 70/67); always ≥ 1.
• Only recorded at provider accept: message not dispatched, dispatch rejected/thrown, short-circuit for missing mobile, simulated send not reaching the provider → not recorded, not billed.
• Idempotent per outgoing message: each record carries the identity of the outgoing message it bills; lease recovery / redelivery / duplicate queue item does not record a second time.
• Ledger write is a side effect, not a precondition: a failed persist does not change the delivery outcome, only logs an error with the outgoing message identity for reconciliation.

Real-world example: Twilio accepts a 200-character SMS for BU ABC (event IncidentUpdate). → The ledger records 1 record MessageCount=1, SmsCount=2 (UCS-2). → Monthly billing sums SmsCount by BusinessUnitId + SentOn; lease recovery of the same message does not add a second record.

Campaign: cancel, liveness, payload

• Cancel depends on scheduler dispatch state and reports the outcome honestly:
  • Before the scheduler row is claimed → deactivate schedule, no artifacts created.
  • During generation (row claimed) → stop creating remaining artifacts at the checkpoint; artifacts already on the provider delivery path stay; report partial cancel.
  • After generation completes → do not reset authoring state to draft; report conflict.
  • HTTP 200 = full cancel; HTTP 202 = cancel in progress; X-Campaign-Cancel-Outcome = machine-readable outcome code (old numeric response body kept).
  • A claimed row that is generating and then edited/rescheduled → reject, the running input stays.
• Liveness: generation refreshes the dispatch row's liveness signal at each checkpoint — long runs are not treated as stuck; a worker dying mid-way → the row becomes eligible for recovery after timeout.
• Payload API authenticates + authorizes by current business unit: resolve the campaign within the BU before reading payload storage; cannot resolve → not-found, no payload read. Topi uses this endpoint instead of fetching MessageJobBlob directly.
• Serial generation (P1): host config keeps concurrency = 1; a single Generator instance is a prerequisite for operation; raising above 1 breaks the invariant → dispatch-claim required first.

Real-world example: An operator cancels while generation is running, 300 emails already in the send path. → Cancel. → Generation stops at the next checkpoint; the 300 emails stay; response 202 — the operator knows what was sent, not told "stopped" while part already went out.

Next: concrete files & workers in Codebase flow.