Automation & Reminders

This section handles automatic scheduled/threshold reminders: expiring documents, invoices due soon, classes nearly full, etc. Unlike "immediate" notifications (reacting right away to an event), reminders are scheduled ahead of time and sent when due.

📌 This is the core of the automation pipeline. Reminder is the first business use case; Report Subscription, UserMessage, and AutomationJob are thin paths that reuse the same pipeline shape.

Picture: three stages

Stage	Who does it	What it does
① Create work	Event Handler + trigger providers	Event → "work schedule" in AutomationScheduler. Does not send anything.
② Due	Worker Automation Scheduler	Scans SQL, claims each row (lease + Version), hands off delivery work to NotificationScheduler.
③ Delivery	Notification pipeline	Render + send (see Communication architecture).

⚠️ "Automation completed" = delivery work has been queued, not that it reached the recipient.

AutomationScheduler — source of truth for "what is due"

SQL table coordinating due-work. Worker only polls SQL; AutomationJob (Azure Table) is a thin trace mirror and is never used to scan work.

Typical columns (shortened):

AutomationTypeId   -- Reminder | ReportSubscription
TriggerTypeId      -- DateBased | ThresholdCondition | Recurring | ScheduledRun | ManualRun
SchedulerKey       -- khóa nghiệp vụ tất định (UNIQUE)
ScheduledOn        -- UTC; (ScheduledLocalDate + TimeZoneId giữ lịch local của BU)
StatusId           -- Pending | Processing | Completed | Failed | Deactivated | Cancelled
ProcessingLeaseExpiresOn  -- hạn lease chống crash, KHÔNG phải danh tính worker
RetryCount / NextRetryOn / FailureReasonCode / IsOnSupportHold
InputHash          -- sha256 input chuẩn hóa + settings ảnh hưởng khóa
DispatchResultJson -- tọa độ downstream + tóm tắt kết quả (nhỏ)
Version            -- concurrency stamp (chống hai worker giẫm chân)

The row coordinator must stay small: do not put large inputs, recipient snapshots, rendered output, or deep diagnostics in SQL — use TraceBlobName (blob) when support needs that.

SchedulerKey — idempotency key

Same "identity key" philosophy as the whole system:

reminder:{settingGuid}:{sourceType}:{sourceGuid}:{triggerKey}:{targetDate}:{offsetDays}
reminder:{settingGuid}:{sourceType}:{sourceGuid}:{triggerKey}:threshold:{thresholdKey}
report-subscription:{guid}:scheduled:{periodStartUtc-or-scheduledOn}
report-subscription:{guid}:manual:{manualRunGuid}

Event, retry, reconciliation all produce the same key. It must not contain random GUIDs (except pre-generated manualRunGuid), display text, recipient list, or template.

Ensure semantics

Situation	Behavior
Row missing	Insert.
Compatible row in Pending/Failed/Deactivated/Cancelled	Update/reactivate, preserve diagnostics + attempts + old dispatch coordinates.
Row Completed	Return coordinates, do not resend (to really send again needs a new trigger key or support-retry key).
Same key, different InputHash	Support hold / conflict, never overwrite.

Processing — Lease + Version (prevent workers stepping on each other)

1. Query Pending, ScheduledOn <= now, NextRetryOn null/<=now, IsOnSupportHold=false
2. Claim: Status=Processing, lease = now + N  WHERE Status=Pending AND Version=read.Version
3. Chỉ xử lý nếu đúng 1 row đổi; giữ Version mới làm token
4. Complete/Fail/Deactivate WHERE Version=token

• Keys prevent duplicate artifacts; Version prevents an old worker from overwriting newer state (after lease expires and another claim happens).
• Stale cleanup: Processing + expired lease → Pending, RetryCount += 1, backoff.
• Backoff: 5 minutes → 15 minutes → 1 hour → IsOnSupportHold (only approved long-running handlers retry daily).
• Source/settings invalid at due time → Deactivated/Skipped, not Failed.

Reminders — settings & trigger providers

Event handler ensures/deactivates/recreates instances when source data changes. They do not send, do not render, and do not scan broad domain tables.

Main flow

parse event → load state nguồn → resolve BU timezone → load ReminderSetting + NotificationSettings
→ DEACTIVATE row cũ trước → ensure AutomationScheduler rows → (ensure immediate notification nếu cần) → metrics

Provider contracts (worker routes to provider, not one giant switch)

Interface	Role
IReminderTriggerProvider	Build scheduler candidate (only schedules work, does not resolve recipient).
IEventInvalidationProvider	Deactivate rows that are no longer valid.
IImmediateNotificationProvider	Create immediate notification when needed.
IReminderRecipientProvider / IReminderDeliveryPolicy	Resolve recipients / channel at execution time (so pending rows stay stable when config changes).

IEventHandler is registered on the seam post-processor (an IEntityEventPostProcessor with its own registry — WKR-009), not the Handler switch. See Platform.

Two trigger types

Type	How	Example
DateBased	scheduledLocalDate = targetDate − offsetDays; convert to UTC by BU timezone (DST by product rule).	Police vetting/first-aid expiry, work anniversary, invoice reminder.
ThresholdCondition	Aggregate update event → calculate metric → compare threshold → when met ensure row, when no longer met deactivate.	Booking capacity nearly full.

thresholdKey = {metricKey}:{thresholdValue}:{firingPolicy}:{scopeKey} — firing policy (OncePerSource | OncePerThreshold | OncePerDay | FalseToTrue) is part of the key.

When settings change

Change	Behavior
Offset/date/threshold	Deactivate + recreate (occurrence identity changes).
Send-time	Update in place (same SchedulerKey, new ScheduledOn).
Template/delivery	Pending row stays unchanged (execution reloads config).
Setting deactivated	Deactivate pending rows. Completed rows are always kept for history.

Execution-time revalidation

When due, handler reloads setting/source/recipients/template and compares with InputHash. Source/setting no longer valid → Deactivated (ReasonCode SourceNoLongerValid/ReminderSettingInactive) — this is an outcome, not a failure.

Bounded reconciliation (no outbox)

There is no AutomationEventOutbox. Source event is primary; each trigger provider declares a bounded scan-repair path: source filter (active only), "changed-since" watermark, due window (today..today+N), allowlist BU/org, batch size, cursor; runs at low frequency; only advances watermark when the batch succeeds; uses the same key helper as event path.

Forbidden: frequent full-domain scan, unbounded historical scan, or reconciliation producing different keys.

Related

• Report Subscription — thin automation path for recurring reports.
• Communication architecture — send pipeline & key policy.
• Platform — seam post-processor & EntityEvent.