Inbox Agent — Synchronization, Event Processing & Reconciliation Specification

Status: Implementation Baseline
Audience: Cursor, Charter, engineering, QA
Purpose: Define reliable ingestion and synchronization of mail provider state into the canonical Inbox Agent model.

1. Goals

Synchronization must produce a complete, convergent local representation of connected mailboxes while tolerating duplicated events, missed notifications, out-of-order delivery, provider throttling, restarts, and partial failures.

The design is provider-adapter based. Provider-specific identifiers and cursors are retained, while downstream application behavior targets the canonical model.

2. Canonical Synchronization Rule

MailMessage represents the logical message/content identity.
MessageInstance represents a provider/account/mailbox-specific occurrence and mutable state of that message.

Synchronization therefore:

A message copied, surfaced, or represented in multiple provider contexts may have multiple MessageInstance rows associated with one MailMessage.

3. Provider Adapter Contract

Each provider adapter must expose equivalent capabilities where the provider supports them:

Provider-specific behavior must not leak into domain services except through explicitly modeled capability flags.

4. Initial Synchronization

Initial sync is checkpointed and restartable.

Recommended sequence:

  1. Register connected account and capabilities.
  2. Enumerate mailbox/folder/label metadata.
  3. Establish provider synchronization cursor where possible.
  4. Backfill messages in bounded pages.
  5. Canonicalize message identity.
  6. Upsert MessageInstance state idempotently.
  7. Queue content/attachment/AI work independently.
  8. Persist progress checkpoints.
  9. Run reconciliation before declaring initial sync healthy.

Large mailboxes must not require one transaction or one uninterrupted worker execution.

5. Incremental Synchronization

Incremental sync uses the provider's strongest available change mechanism:

  1. Provider webhook/push notification when available.
  2. Delta/history cursor retrieval.
  3. Scheduled polling as fallback.

Notifications are wake-up hints, not trusted as a complete event log. A notification causes the adapter to read authoritative changes from the provider.

Cursor advancement occurs only after the corresponding changes have been durably processed or safely checkpointed.

6. Idempotency

All synchronization operations must be replay-safe.

Use stable provider keys and unique constraints for:

Upsert logic must make duplicate webhook delivery and repeated page retrieval harmless.

7. Event Processing

Internal events are produced from committed canonical changes, not directly from raw provider callbacks.

Event envelope should include:

Use transactional outbox semantics, or an equivalent guarantee, so database state and event publication cannot silently diverge.

Consumers must be idempotent.

8. Ordering and Concurrency

Global ordering is not required. Correctness is required at the smallest relevant aggregate boundary.

Concurrency controls must prevent stale processing from overwriting newer state. Appropriate techniques include:

Out-of-order events must converge after reconciliation.

9. Deletions and Missing Items

Provider deletion semantics vary. The local model must distinguish:

Do not delete the canonical MailMessage merely because one MessageInstance disappears. Canonical content may still be referenced by another instance, thread, audit record, or retained metadata.

Physical purging is a separate retention process.

10. Folder and Label Semantics

Folder/label membership is instance/provider state. Adapters normalize provider semantics into canonical mailbox-state structures without pretending all providers behave identically.

The canonical layer must be able to represent:

11. Write-Back and Echo Suppression

A local mutation follows:

  1. validate user/policy authorization;
  2. create a durable mutation request with idempotency key;
  3. send provider mutation;
  4. capture provider response;
  5. observe the change through normal sync;
  6. reconcile desired vs observed state.

Do not assume API success means final synchronized state.

Locally initiated provider events may be recognized through correlation/idempotency metadata, but they should still be processed. Echo suppression prevents duplicate workflows, not canonical state updates.

12. Retry and Dead-Letter Behavior

Retry transient failures with bounded exponential backoff and jitter. Respect provider retry-after and rate-limit information.

After the configured attempt/window limit:

Poison events must not block the account's entire synchronization stream unless continuing would violate correctness.

13. Reconciliation

Reconciliation is mandatory because notifications and incremental APIs are not assumed perfect.

Reconciliation levels:

Reconciliation detects:

Repairs must be idempotent and auditable.

14. Cursor Recovery

If a provider cursor/delta token expires or becomes invalid:

Cursor loss must not require destructive database reset.

15. Observability

Per connected account track at minimum:

Logs must carry account, correlation, and work-item identifiers without exposing message bodies by default.

16. Security

Provider tokens are encrypted and least-privilege scoped. Raw webhook requests are authenticated/validated according to provider requirements.

Message content and attachments are not written to operational logs. Error diagnostics use identifiers and sanitized metadata.

17. Acceptance Criteria

Synchronization is implementation-ready when: