ADR 0017: Three-level reconciliation and non-destructive cursor recovery

Status

Accepted

Context

Duckbill™ cannot treat provider notifications or incremental tokens as complete. Mailboxes drift when a user acts in Outlook, a webhook is missed, or a delta token expires. Sprint 03 already persisted reconciliation_runs and left initial sync at awaiting_reconciliation. Sprint 04 Stream B has to make those rows do real work without wiping canonical MailMessage / MessageInstance data.

Conversation attention already lives on conversation_states (ADR 0009). Equating provider READ with COMPLETE would clear RESPOND the moment Outlook marked a copy read. Stream A stores the incremental cursor under scope incremental-delta (ADR 0016).

Decision

1. Run three reconciliation levels against the provider port: lightweight (recent instances and cursor health), periodic (bounded window), and deep (full provider vs local scan).
2. Persist diagnostics on reconciliation_runs (compared_count, drift_count, repaired_count, diagnostics). Repairs are idempotent upserts or tombstones and emit audit_events.
3. Detect missing instances, orphan locals, stale read/folder state, and duplicate canonical mappings. Flag duplicates; do not auto-merge.
4. Keep READ ≠ COMPLETE. Sync MessageInstance.is_read from the provider. Never clear RESPOND from a read-state change.
5. When attention already exists as RESPOND, an externally sent reply associated to the thread may move attention to WAITING.
6. Expired incremental cursors mark the incremental-delta checkpoint degraded, upsert a bounded resync, run reconciliation, then store a new cursor. No TRUNCATE or delete-all path.

Alternatives

• Destructive mailbox reset on cursor loss — rejects the sync specification and loses canonical history.
• Derive attention from is_read — rejected by ADR 0009 and the lifecycle rule READ ≠ COMPLETE.
• Auto-merge duplicate MailMessage rows — false merges are worse than leftover duplicates (Stream C owns identity).

Consequences

runReconciliation can complete awaiting_reconciliation after initial sync. Cursor recovery is degraded sync, not data loss. Webhook ingress stays on Stream A. MailMessage identity stays on Stream C.