Communication — Core concepts

Artifacts and "grain"

"Grain" = the level of detail represented by one record (what one record stands for).

Artifact	Store	Grain	Role
NotificationJob	Azure Table	1 notification event	Input: "there is work to send".
NotificationScheduler	SQL	1 pending job	Queue row for Scheduler to pick up.
MessagingJob + Blob	SQL + Blob	1 large payload	Coordinates campaign/report (large content lives in Blob).
UserMessage	Azure Table	1 recipient	In-app message; inbox in the app.
OutgoingMessage	Azure Table	1 recipient × 1 channel	Message sent through provider; carries delivery state.
SmsUsage	Azure SQL	1 accepted SMS provider	SMS billing ledger by BusinessUnit (billing).

UserMessage: metadata in Table, body in Blob

UserMessage is an Inbox record for one recipient owner; it is not a copy of OutgoingMessage and does not use Redis as its primary data source.

Data group	What is stored	Purpose
Identity & scope	UserMessageId, UserId, owner ids (AccountId/EmployeeId/EnterpriseUserId), BusinessUnitId, SiteId	Identifies the recipient, portal scope and location filter.
Source & classification	MessageCategoryId, MessageSourceTypeId, EventTypeId, CampaignGuid, NotificationJobPartitionKey/RowKey, entity coordinates	Traces the source, classifies Notification/MessagingCampaign/Reminder, and supports deep links.
Display	Title, MessagePreview, provider/attendee/site display names, ActionUrl	Renders list/detail without loading the body.
Private content	MessageBodyBlobName	Points to HTML/text in Blob container message-content; the body is streamed only after authorization.
Recipient-owned state	StatusId, ReadOn, ArchivedOn, SavedOn, Reply*, ETag	Read/archive/save/reply are owned by the recipient; producer retry must not reset them.

CompatibilityHash is a SHA-256 of immutable fields. A retry for the same message is idempotent through ensure; different content/action hash fails closed with VERSION_CONFLICT.

Real-world example: The Generator retries the same Campaign for an Account. → The hash and row identity remain the same. → The system preserves ReadOn/SavedOn and creates no extra Inbox row.

Send pipeline: three steps

Step	What it does
Scheduler	Scans Ready work; evaluates Notification Settings (is this type enabled, who receives it).
Generator	Renders templates + resolves recipients; creates UserMessage (in-app) and/or OutgoingMessage (external channel).
Distributor	Sends for real through SendGrid/Twilio; "claim before send".
Webhook & Tracker	Updates state (delivered/bounce/opened) back to OutgoingMessage.

Four core principles

Every Communication decision follows from 4 principles (shared with automation):

1. Identity keys instead of random keys — each occurrence has exactly one deterministic key; every downstream key (OutgoingMessage, etc.) is derived from it. Duplicate events/retries produce the same key → no duplicates.

2. Ensure* instead of Create* — every artifact creation operation means "ensure it exists": already exists and compatible → always success (idempotent); content differs (InputHash differs) → conflict, stop automatic sending, report support. Never Upsert Replace — avoid accidentally sending again.

3. Claim before send — Distributor must move OutgoingMessage to Sending with a conditional update before calling the provider. Duplicate messages that see the row already Sending/Sent exit — no duplicate emails.

4. Bounded reconciliation instead of outbox — no complex outbox; each provider declares a bounded scan-repair path (watermark, date window, batch) at low frequency to repair rare issues.

⚠️ Exception to no-replace: EntityHistory (audit) may upsert InsertOrReplace by deterministic key — its RowKey is not externally referenced, overwriting on retry is harmless. NotificationJob keeps query-before-append.

Template

Message content is a template with {{...}} placeholders (e.g. {{INCIDENT_DETAILS_LINK}}); Generator fills real data at render time. A template has key + version ("immutable routing fields" — not changed mid-send).

Immutable vs mutable fields

When Ensure* matches an existing artifact, it distinguishes:

• Immutable (message identity): source coordinates, event type, recipient identity, delivery method, template key/version, run key, automation coordinates. Mismatch → conflict, stop send.
• Mutable (safe to change): retry counters, scheduler coordinates, provider status, webhook timestamps, bounded diagnostics.

Campaign delivery: idempotency per recipient × channel

• Each campaign OutgoingMessage has a deterministic key from (Campaign, stable recipient, normalized destination, delivery method) — retry resolves to exactly one artifact; new keys do not overwrite old random-key rows (still readable).
• A recipient's email and SMS are two independent artifacts, completeness evaluated per channel — a channel that exists does not let a missing channel be skipped; a successful channel is not recreated. Retry only creates the missing part.
• Ensure does not overwrite provider-progress (claim, provider id, timestamps, terminal state); retry hitting an immutable conflict → fail closed (stop automatic sending) + bounded diagnostic.
• A row already persisted but not yet queued → generation retry repairs it (publish/restore exactly one work item) within the campaign partition scope; a row already delivered to the provider is not published again.

In-app is an independent campaign channel

• Campaign supports an optional In-app channel, creating UserMessage independent of OutgoingMessage (Email/SMS); legacy payload without the field → In-app off; disabling In-app does not disable the provider channel.
• UserMessage is created only when the target resolves to exactly one valid owner (Account / Employee / EnterpriseUser); no portal identity or ambiguous target → SkippedNotEligible (skipped, not a system failure), no multiple inbox rows.
• Campaign does not create/modify Account/Employee/login to "make" a target valid; no legacy Customer_{UserId} is built for new deliveries.

Real-world example: Campaign Holiday Notice: generation failed after creating the Email, the SMS was not created. → Retry. → Only the SMS is created additionally; Email keeps its provider-progress, parents do not receive a second email.

Campaign detail drawer: per-part diagnostics

• The sent-campaign drawer shows diagnostics independently per part: recipients, email content, attachments — one part failing does not hide the state of other parts.
• Each diagnostic includes status, reason (human-readable), error code (optional), failure timestamp (when known); missing metadata → omit/null, no fabricated values.
• The portal prefers structured diagnostics; it only falls back to a generic error (Failed to load recipient list) when there is no metadata — allowing backend/frontend to deploy out of sync.

Real-world example: Campaign Holiday Notice: recipients failed with Permission denied (403, with timestamp), attachments failed with no code, email content loaded. → Admin opens the drawer. → Recipients show reason/code/time; attachments show failed + reason; email content renders normally.

Campaign: manual recipient selection across pages

• Within one search result, manual recipient selection persists across pagination; the checked state reflects the global selection (the union of selected pages).
• Actions only change the current page's selection: Select All / Deselect All in the header applies only to the current page; deselecting one row does not affect other pages.
• One row identity rule for render/match/save of selection; each selected key has exactly one recipient record (no undefined); missing attendee id → fall back to contact/account/employee id.
• On Add: SelectedRowKeys + SelectedRecipients hold the multi-page union; count = size of that union; reopening the draft keeps off-page selection.
• A new search resets selection + back to page 1; changing pagination within the current result does not reset.

Real-world example: A search for parents returns 3 pages; select a few on pages 1–2, then Select All on page 3. → Add. → Only page 3 is added via Select All; selection on pages 1–2 is kept; count = the union of all 3 pages.

Notification traceability

• SourceKey gives a stable source coordinate after resolution, such as invoice:{guid} or account:{guid}. Booking carts use the account coordinate and retain OrderGuids for in-memory filtering.
• The source key is stored at OutgoingMessage grain (one message per source/recipient/channel), not mechanically on NotificationJob, because one job can contain multiple invoices/orders.
• Distributor claim leases and provider progress remain authoritative; retry does not rewrite an old row's identity. A generic read API and retention policy are outside the current write capability.

Real-world example: One invoice job contains several invoices. → Support opens each invoice history separately. → The generator keeps each invoice:{guid} partition during retry.

Recipient copy scope

Site-contact copy is valid only for site-level notifications. Business Unit contact copy is deduplicated with Site contact; list toggles are BU-wide, while mixed site state is shown explicitly.

Realtime Inbox: invalidation signals

Realtime only reports that the Inbox may have changed; UserMessage and REST remain the authoritative source for data, permissions and read state.

• The persistence path emits one inbox.changed only after a new UserMessage is created successfully; compatible retry/ensure, state update, archive or delivery failure emits no creation event.
• The SSE payload contains only minimal metadata (eventId, type, reason, category, occurredAt); it does not send rowKey, body, Blob URL, recipient metadata or authorization scope.
• Tuke identifies the connection from JWT sub; query context only serves REST scope authorization and does not choose the event recipient.
• On reconnect, the client sends Last-Event-ID; retained events are replayed, while expired or invalid events return inbox.sync-required so the client calls REST again.
• Redis or SSE failure does not roll back UserMessage; the UI continues with REST, focus/visibility refresh and polling fallback.
• SSE is server → client only; Inbox does not add compose, reply or a client-to-server message method.

Real-world example: The Generator creates a new UserMessage for a parent. → SSE reports inbox.changed without private content. → Tuke calls UnreadCount and the Inbox page through REST under the parent's authorized scope.

Got it? See how they assemble into the system in Architecture.