Inbox Agent Processing Pipeline and Event Orchestration Specification

1. Purpose

This document defines the end-to-end processing lifecycle for Inbox Agent events.

It describes how the platform responds when:

The goal is to ensure processing is:

DETERMINISTIC WHERE POSSIBLE
SECURITY-FIRST
EVENT-DRIVEN
IDEMPOTENT
AUDITABLE
RECOVERABLE
PROVIDER-INDEPENDENT
EVENTUALLY CONSISTENT

2. Core Processing Principle

The canonical lifecycle is:

PROVIDER CHANGE
      ↓
DISCOVER
      ↓
NORMALIZE
      ↓
SECURITY INSPECTION
      ↓
DETERMINISTIC RULES
      ↓
AI IF NECESSARY
      ↓
CONVERSATION / ATTENTION UPDATE
      ↓
POLICY EVALUATION
      ↓
COMMANDS
      ↓
PROVIDER MUTATIONS
      ↓
RECONCILIATION
      ↓
AUDIT + UI UPDATE

No stage should silently assume responsibility belonging to another stage.

3. Event Sources

The system may receive work from:

Microsoft Graph notifications
Microsoft delta synchronization
Gmail Pub/Sub notifications
Gmail history synchronization
iCloud polling / IMAP synchronization
User API requests
Agent requests
Rule engine
Scheduled jobs
Migration engine
Reconciliation jobs
Security services

All should normalize into a common internal event model where practical.

4. Internal Event Envelope

Every durable internal event should contain:

event_id
event_type
event_version
occurred_at
received_at
user_id
mail_account_id_optional
entity_type_optional
entity_id_optional
provider_reference_optional
correlation_id
causation_id_optional
payload

5. Event ID

event_id must uniquely identify one logical internal event.

It should support deduplication.

6. Correlation ID

The correlation ID links a complete workflow.

Example:

Provider notification
    ↓
Message synchronization
    ↓
Security scan
    ↓
Classification
    ↓
Rule execution
    ↓
Archive command

All events may share:

correlation_id = ABC123

7. Causation ID

A causation ID identifies the immediate upstream event.

Example:

MAIL_DISCOVERED
causes
SECURITY_SCAN_REQUESTED

The second event references the first event ID.

This allows full event-chain reconstruction.

8. Event Versioning

Durable event payloads should carry a version.

Example:

event_type = MESSAGE_DISCOVERED
event_version = 2

Queued work may outlive an application deployment.

Consumers must therefore handle known event versions safely.

9. Event Categories

Primary event categories:

PROVIDER
MAIL
SECURITY
CLASSIFICATION
CONVERSATION
RULE
COMMAND
APPROVAL
LEARNING
MIGRATION
SYNC
SYSTEM

10. Provider Events

Representative provider events:

PROVIDER_CHANGE_RECEIVED
PROVIDER_AUTH_CHANGED
PROVIDER_SUBSCRIPTION_EXPIRING
PROVIDER_SUBSCRIPTION_FAILED
PROVIDER_RATE_LIMITED

Provider events should not expose raw provider semantics beyond adapter boundaries unless necessary.

11. Mail Events

Representative mail events:

MESSAGE_DISCOVERED
MESSAGE_UPDATED
MESSAGE_MOVED
MESSAGE_DELETED
MESSAGE_READ_STATE_CHANGED
MESSAGE_SENT
DRAFT_DISCOVERED
THREAD_CHANGED

12. Security Events

Representative security events:

SECURITY_INSPECTION_REQUESTED
SECURITY_INSPECTION_COMPLETED
MESSAGE_QUARANTINED
MESSAGE_RELEASED
ATTACHMENT_HELD
ATTACHMENT_RELEASED
SECURITY_FINDING_CREATED

13. Classification Events

CLASSIFICATION_REQUESTED
CLASSIFICATION_COMPLETED
CLASSIFICATION_FAILED
ATTENTION_CHANGED
PRIORITY_CHANGED
RETENTION_CHANGED
PROJECT_ASSIGNED

14. Rule Events

RULE_EVALUATION_REQUESTED
RULE_MATCHED
RULE_ACTION_REQUESTED
RULE_EXECUTED
RULE_FAILED
RULE_CONFLICT_DETECTED

15. Command Events

COMMAND_REQUESTED
COMMAND_AUTHORIZED
COMMAND_APPROVAL_REQUIRED
COMMAND_QUEUED
COMMAND_STARTED
COMMAND_SUCCEEDED
COMMAND_FAILED
COMMAND_PARTIAL
COMMAND_UNDO_REQUESTED

16. Learning Events

USER_CORRECTION_RECORDED
PATTERN_OBSERVED
RULE_SUGGESTION_CREATED
RULE_SUGGESTION_APPROVED
RULE_HEALTH_CHANGED

17. Migration Events

MIGRATION_STARTED
MIGRATION_ITEM_SELECTED
MIGRATION_ITEM_PROCESSED
MIGRATION_CONFLICT
MIGRATION_RECONCILIATION_STARTED
MIGRATION_COMPLETED

18. Event Processing Architecture

Recommended logical model:

Event Producer
     ↓
Durable Event / Job Queue
     ↓
Worker
     ↓
Application Service
     ↓
Domain Update / Command
     ↓
New Events

Long-running work should not depend on request-response execution.

19. Durability

The following work must survive:

Application restart
Deployment
Worker crash
Temporary provider outage
Temporary AI outage
Temporary scanner outage

Examples:

Historical synchronization
Migration
Classification backlog
Subscription renewal
Bulk archive
Reconciliation

20. Inbox Notification Handling

Provider webhooks should do minimal synchronous work.

Preferred lifecycle:

Webhook received
      ↓
Validate webhook
      ↓
Persist notification
      ↓
Queue synchronization event
      ↓
Return provider acknowledgment

Do not execute classification and AI reasoning inside the webhook request.

21. Microsoft Notification Flow

Conceptual flow:

Microsoft Graph Notification
        ↓
Validate Notification
        ↓
Identify MailAccount
        ↓
Queue Graph Delta Sync
        ↓
Retrieve Changed Resources
        ↓
Normalize Messages
        ↓
Generate Internal Mail Events

Graph notifications should be treated as:

Something changed

not necessarily a complete canonical description of the change.

22. Gmail Notification Flow

Conceptual flow:

Google Pub/Sub Notification
        ↓
Validate
        ↓
Identify MailAccount
        ↓
Read Gmail historyId
        ↓
Retrieve history changes
        ↓
Normalize
        ↓
Generate Mail Events

23. iCloud Synchronization Flow

Initial iCloud flow may be polling/incremental:

Scheduled Sync
      ↓
Read Folder Checkpoint
      ↓
Fetch Changed UIDs
      ↓
Normalize
      ↓
Generate Mail Events
      ↓
Update Checkpoint

iCloud is initially lower-frequency and more read-oriented than Microsoft/Gmail.

24. Synchronization Must Be Incremental

Normal synchronization should avoid scanning the entire mailbox repeatedly.

Provider checkpoint examples:

Graph delta link
Gmail historyId
IMAP UID / UIDVALIDITY

25. Full Synchronization

Full synchronization should occur only when required.

Examples:

Initial account connection
Lost checkpoint
Provider invalidates token
Manual repair operation
Reconciliation detects major drift
Migration inventory

Full sync should be a durable background job.

26. Message Discovery Pipeline

When a message is first discovered:

MESSAGE DISCOVERED
      ↓
Canonical Message Upsert
      ↓
Recipient / Identity Resolution
      ↓
Conversation Resolution
      ↓
Security Inspection
      ↓
Rule / AI Processing

27. Canonical Upsert

Provider messages should be upserted using provider-safe identifiers.

The operation must tolerate duplicate delivery of the same provider event.

Example:

Notification 1 → insert
Notification 2 → same message detected → update/no-op

28. Message Identity Resolution

The normalization stage should determine:

Physical mailbox
Delivered identity
Known alias
Original recipient where available
Sender profile
Provider folder / label state

Identity should be resolved before ordinary classification.

29. Conversation Resolution

The system should attempt to attach the message to a canonical conversation using available signals:

Provider thread ID
Internet Message-ID references
In-Reply-To
References headers
Normalized subject
Participants
Temporal proximity

Provider thread IDs are useful but should not be the only possible basis.

30. Conversation Re-evaluation

Whenever a new message enters an existing conversation, the conversation state should be re-evaluated.

Example:

Current state:
WAITING

Incoming reply arrives

New state:
UNPROCESSED
or
ACTION
or
RESPOND
or
RESOLVED

The system should not leave the conversation blindly in WAITING.

31. Security Inspection Position

Security runs before normal content-based processing when necessary.

Pipeline:

Normalize
   ↓
Security
   ↓
Normal Rules
   ↓
AI

A quarantined item should not proceed as though it were ordinary trusted mail.

32. Security Processing Modes

Security inspection may have separate phases:

METADATA SECURITY CHECK
BODY SANITIZATION
URL INSPECTION
ATTACHMENT INSPECTION
OPTIONAL DEEP SCAN

Not every phase must block all downstream processing.

33. Security Fast Path

A common clean message may follow:

Basic checks
      ↓
No concern
      ↓
CLEAN
      ↓
Continue processing

34. Security Hold Path

Suspicious message:

Security finding
      ↓
QUARANTINED
      ↓
Store finding
      ↓
Generate security alert/review item
      ↓
Stop ordinary automation

35. Quarantine Processing Boundary

While quarantined, ordinary actions such as:

Auto archive
Auto move
Create identity recommendation
Generate external-content action

should generally be suspended unless specifically security-safe.

36. Safe Quarantine Classification

The system may still perform limited classification using:

Headers
Sender
Safe metadata
Sanitized text

if security policy allows.

This can help categorize the quarantine queue.

37. Security Release Event

When the user releases a message:

MESSAGE_RELEASED
      ↓
Update security state
      ↓
Re-enter normal processing
      ↓
Rule evaluation
      ↓
AI if needed
      ↓
Attention update

Release should therefore be an explicit processing transition.

38. Rule Engine Position

After security clearance:

Security Approved
       ↓
Explicit User Rules
       ↓
Approved Learned Rules
       ↓
Known Deterministic Patterns

AI should only handle unresolved semantic ambiguity.

39. Rule Evaluation Input

The rule engine may inspect canonical fields such as:

mail_account
mailbox_type
identity
sender
sender_profile
subject
headers
classification already known
security state
attachment metadata
system event metadata
project signals

40. Rule Evaluation Output

Rule evaluation should produce structured results:

matched_rules
classification_changes
attention_changes
priority_changes
retention_changes
provider_actions_requested
terminal_processing_indicator

41. Terminal Rules

A rule may declare processing sufficiently resolved.

Example:

Successful Vercel deployment
→ SYSTEM
→ BACKGROUND
→ ARCHIVE

No AI classification may be required.

However security policy always remains higher priority.

42. Non-Terminal Rules

Some rules add context without finishing processing.

Example:

Sender = accountant
→ tag FINANCIAL

The message may still need AI to determine:

ACTION?
RESPOND?
REFERENCE?

43. AI Invocation Decision

AI should run only when it adds value.

Possible trigger:

No deterministic classification
Semantic interpretation required
Conversation intent unclear
Action/response detection uncertain
Project inference needed
Summary requested

44. AI Avoidance

Do not invoke AI merely because a message exists.

Examples requiring no AI:

Exact approved rule matches
Known system success event
Provider state synchronization
Mark read command
Folder inventory
Duplicate detection by Message-ID

45. AI Classification Request

A classification job should include carefully selected context:

Message metadata
Sanitized content
Relevant conversation context
Sender profile
Applicable learned preferences
Project signals

It should not include unrestricted mailbox history.

46. AI Classification Output

Structured output:

primary_classification
secondary_tags
attention_state
priority
retention
requires_response
requires_action
waiting_candidate
project_candidate
due_date_candidate
confidence
reason_summary

47. Schema Validation

AI output must be validated before application.

Failure examples:

Invalid enum
Malformed date
Missing required field
Unrecognized project
Impossible action

Invalid output should not mutate canonical state.

48. AI Retry

For transient AI errors:

Timeout
Provider unavailable
Rate limit

retry according to policy.

For schema errors:

Retry with constrained correction prompt

up to a defined maximum.

49. AI Failure Fallback

If AI remains unavailable:

Message stays safely visible
attention = UNPROCESSED
classification = UNKNOWN

The deterministic pipeline continues functioning.

50. Confidence Policy

After AI classification:

HIGH
→ normal permitted low-risk handling

MEDIUM
→ classification may apply
→ consequential automation restricted

LOW
→ Review Queue

Confidence must not override security or explicit user rules.

51. Attention-State Resolution

Attention state should be determined using:

Rules
AI result
Conversation history
Sent mail
Waiting state
User corrections
Due-date signals

52. Respond Detection

Potential flow:

Incoming direct human message
      ↓
Question/request detected
      ↓
No user response yet
      ↓
RESPOND

53. Waiting Creation

When the user sends a message:

MESSAGE_SENT
      ↓
Conversation analysis
      ↓
Did user ask/submit/request something?
      ↓
Yes
      ↓
WAITING candidate

At high confidence or explicit user action:

Create WaitingState

54. Waiting Resolution

Incoming message from the relevant participant:

WAITING
   ↓
Reply received
   ↓
Re-evaluate conversation

Possible outcomes:

RESPOND
ACTION
REFERENCE
RESOLVED
WAITING continues

55. Stale Waiting Evaluation

Scheduled job:

Find active WaitingState
      ↓
Compare current time to:
expected_response_at
follow_up_after
learned typical response time
      ↓
Mark stale/overdue if applicable
      ↓
Generate follow-up suggestion

Do not auto-send initially.

56. Due-Date Extraction

Due dates may be:

Explicitly stated
User-set
Detected from message
Detected from attachment
Derived from known event
Suggested by AI

Derived dates should preserve their source.

57. Rule Actions and Commands

A rule should not directly mutate the provider.

Example:

Rule says:
ARCHIVE

becomes:

RULE_ACTION_REQUESTED
      ↓
Policy evaluation
      ↓
MailCommand
      ↓
Provider Adapter

58. Policy Evaluation

Every consequential action should pass through policy.

Inputs:

Actor
Action
Target
Security state
Rule authorization
User autonomy settings
Risk level
Provider capability
Approval state

59. Policy Outcomes

Possible outcomes:

ALLOW
APPROVAL_REQUIRED
DENY

60. Example Allowed Action

Actor:
Approved deterministic rule

Action:
Archive successful build notification

Risk:
LOW

Policy:
ALLOW

61. Example Approval Action

Actor:
AI Agent

Action:
Send email

Policy:
APPROVAL_REQUIRED

62. Example Denied Action

Actor:
Email content / prompt injection

Action:
Change forwarding

Policy:
DENY

63. Command Creation

Allowed operations create durable MailCommand.

Command should exist before provider mutation.

This allows:

Audit
Retry
Status tracking
Idempotency
Undo

64. Command Execution Pipeline

COMMAND_QUEUED
      ↓
Worker Claims Command
      ↓
Validate Current State
      ↓
Check Provider Capability
      ↓
Execute Adapter Operation
      ↓
Record Result
      ↓
Update Canonical State
      ↓
Audit
      ↓
Publish Completion Event

65. Pre-Execution State Check

A queued command must validate that assumptions are still valid.

Example:

Archive command queued

Before execution:
Message already deleted manually in Outlook

The command should reconcile instead of failing unpredictably.

66. Command Idempotency

Commands require an idempotency key.

If the same command is retried:

Execute once
Return same logical outcome

where provider semantics permit.

67. Provider Mutation Confirmation

The system should not mark a command fully successful merely because an API request was sent.

Prefer provider-confirmed state where practical.

68. Canonical State Update

After successful provider operation:

Update provider-state representation
Update message/conversation derived state
Write audit event
Publish relevant internal event

69. Partial Bulk Failure

Bulk commands may result in:

SUCCEEDED = 94
FAILED = 4
SKIPPED = 2

The overall state should become:

PARTIALLY_SUCCEEDED

not simply FAILED or SUCCEEDED.

70. Retry Classification

Errors should be categorized as:

TRANSIENT
AUTHORIZATION
VALIDATION
CONFLICT
PERMANENT
POLICY

71. Transient Retry

Examples:

Network timeout
Provider 5xx
Temporary rate limit
Scanner unavailable
AI temporary failure

Use exponential backoff with jitter.

72. Authorization Errors

Examples:

Expired OAuth
Revoked consent
Missing provider scope

Do not endlessly retry.

Mark provider connection degraded and request intervention.

73. Validation Errors

Example:

Message no longer exists
Invalid provider state
Unsupported operation

These require reconciliation or terminal handling rather than repeated retries.

74. Dead-Letter Handling

After maximum retry attempts, unresolved processing should enter:

DEAD_LETTER

or equivalent operational state.

The item should remain inspectable and retryable manually.

75. Dead-Letter Metadata

Store:

event/job ID
event type
mail account
attempt count
last error
first failure
last failure
correlation ID

76. Processing Jobs

Long-running work should use ProcessingJob.

Examples:

Historical mailbox sync
Migration
Rule simulation
Security backlog
AI classification backlog
Large natural-language search
Bulk archive
Reconciliation

77. Job Lifecycle

QUEUED
RUNNING
SUCCEEDED
FAILED
RETRYING
PARTIAL
CANCEL_REQUESTED
CANCELLED

78. Job Progress

Jobs should expose meaningful progress where measurable.

Example:

Processed:
4,218

Total:
10,550

Avoid false precision when total is unknown.

79. Cancellation

Cancellation should be cooperative.

Example:

User cancels migration

Current item completes safely
No new items claimed
Job enters CANCELLED

Do not abruptly leave provider mutations in undefined state.

80. Event Ordering

Provider events may arrive:

Late
Duplicated
Out of order

The architecture must not rely on perfect delivery order.

81. Version / Timestamp Comparison

Where provider semantics permit, compare:

Provider version
change key
updated timestamp
history ID

before overwriting newer state.

82. Ordering by Entity

Some event processing may need serialized behavior for a single entity.

Example:

Conversation A

should not simultaneously process contradictory attention-state transitions without concurrency control.

Global serialization is unnecessary.

83. Optimistic Concurrency

Canonical records may use:

version
updated_at

Updates can require an expected version where conflicts matter.

84. Duplicate Event Handling

Processing an already-completed event should:

Recognize duplicate
Return successful no-op

rather than repeat provider mutations.

85. Reconciliation

Events and webhooks are not sufficient by themselves.

Periodic reconciliation should compare:

Provider state
vs
Canonical application state

86. Why Reconciliation Is Required

State can drift because:

User acted in Outlook
User acted in Gmail
Notification missed
Webhook expired
Worker failed
Provider changed message ID
External client changed folder

87. Reconciliation Scope

Periodic reconciliation may inspect:

Recent message changes
Folder state
Read state
Sent mail
Drafts
Subscriptions
Provider checkpoints

88. Sent-Mail Reconciliation

Sent mail is particularly important because the user may reply outside Inbox Agent.

Example:

Conversation = RESPOND

User replies from Outlook

Reconciliation discovers sent message

Conversation:
RESPOND → WAITING

This is central to client independence.

89. Read-State Reconciliation

Read/unread should synchronize with provider state where supported.

However:

READ ≠ COMPLETE

A provider read-state change should not clear ACTION/RESPOND.

90. Folder Reconciliation

Manual provider folder changes should update canonical provider state.

They should not automatically redefine semantic classifications.

91. Subscription Renewal Pipeline

Scheduled job:

Find subscriptions nearing expiration
      ↓
Renew
      ↓
Validate
      ↓
Store new expiration

Failure should create provider-health alert.

92. Gmail Watch Renewal

Gmail watches require periodic renewal.

The system should renew before expiration and preserve the latest valid history checkpoint.

93. Graph Subscription Renewal

Graph subscriptions should similarly renew before expiration.

Subscription loss should not silently create stale mail.

94. Subscription Failure Fallback

If push fails:

Mark push degraded
      ↓
Enable incremental polling fallback
      ↓
Surface health issue
      ↓
Attempt repair

95. Provider Health State

Suggested canonical states:

HEALTHY
DEGRADED
AUTH_REQUIRED
SYNC_DELAYED
RATE_LIMITED
OFFLINE

96. Stale Data Propagation

If an account is stale, views should know.

Example:

Needs Me

Microsoft:
Current

System Gmail:
Last synced 3 hours ago

The platform must not equate missing recent data with no work.

97. User Action Pipeline

Example user presses:

Archive

Lifecycle:

UI
 ↓
Application API
 ↓
Authorization
 ↓
Create MailCommand
 ↓
Optimistic UI optional
 ↓
Worker
 ↓
Provider Adapter
 ↓
Canonical update
 ↓
Audit
 ↓
UI synchronization

98. User Correction Pipeline

Example:

User changes:
REFERENCE → ACTION

Lifecycle:

Update canonical state
      ↓
Record UserCorrection
      ↓
Audit
      ↓
Emit USER_CORRECTION_RECORDED
      ↓
Learning system observes
      ↓
Potential pattern update

99. Rule Suggestion Pipeline

Repeated user behavior
      ↓
PatternObservation updated
      ↓
Threshold reached
      ↓
Generate candidate
      ↓
Simulate rule
      ↓
RuleSuggestion created
      ↓
User Review Queue

No active behavior changes yet.

100. Rule Approval Pipeline

User approves suggestion
      ↓
Create Rule
      ↓
Create RuleVersion
      ↓
Activate according to execution mode
      ↓
Audit approval

Future messages may then match deterministically.

101. Rule Health Pipeline

After executions:

RuleExecution
      ↓
Monitor undo/correction/exception
      ↓
Calculate health
      ↓
HEALTHY / WATCH / REVIEW / PAUSED

102. Automatic Rule Pause

For sufficiently severe failures, policy may pause a rule.

Example:

High-risk rule produces repeated incorrect actions

System:

Pause rule
Surface alert
Require review

103. Security Rule Pipeline

Security rules should evaluate separately from ordinary organizational rules.

Security metadata
      ↓
Security policy
      ↓
Security rules
      ↓
Security state

Ordinary rules cannot override a security hold.

104. Agent Command Pipeline

Example:

User:
Archive all successful deployment messages this week.

Lifecycle:

Agent interprets intent
      ↓
Search resolves targets
      ↓
Structured command proposal
      ↓
Policy / blast-radius evaluation
      ↓
Preview
      ↓
User approval if required
      ↓
Command batch
      ↓
Provider execution

105. Agent Search Pipeline

Natural-language request
      ↓
Intent classification
      ↓
Structured query plan
      ↓
Application metadata search
      +
Provider content search as required
      ↓
Merge / dedupe
      ↓
Return results

106. Search Should Not Mutate

No search operation should alter:

Read state
Folder
Attention
Priority
Provider metadata

merely because the message was inspected.

107. Daily Brief Pipeline

Scheduled or on-demand:

Read canonical workload
      ↓
Gather:
ACTION
RESPOND
WAITING
SYSTEM_ALERT
Important New
Security
      ↓
Summarize
      ↓
Store optional brief
      ↓
Present

The Daily Brief should use canonical state rather than rescanning every mailbox through AI.

108. Migration Pipeline

Migration must remain separate from active-mail processing.

Inventory
  ↓
Mapping
  ↓
Dry Run
  ↓
Duplicate Analysis
  ↓
Pilot
  ↓
Validation
  ↓
Bulk Migration
  ↓
Reconciliation
  ↓
Cleanup

109. Historical Migration Attention Policy

Migrated historical messages should normally enter as:

attention = ARCHIVE
is_historical = true

They should not flood current workload.

110. Historical Security Policy

Before historical messages are actively rendered:

Sanitize content
Inspect suspicious attachments on demand

Historical age does not establish trust.

111. Duplicate Processing

Duplicate detection should run before target import where migration requires it.

Primary signal:

Internet Message-ID

Fallback:

Sender
Recipients
Subject
Timestamp
Body hash
Attachment hash

112. Migration Conflict

Uncertain duplicates should become:

CONFLICT

rather than being discarded automatically.

113. Migration Reconciliation

For each run:

Selected
=
Succeeded
+ Skipped
+ Duplicate
+ Conflict
+ Failed

The numbers should reconcile explicitly.

114. Security Scanner Pipeline

Attachment scan:

Attachment discovered
      ↓
Metadata inspection
      ↓
Policy determines deeper scan?
      ↓
Fetch to isolated scanner
      ↓
Scan result
      ↓
Security finding
      ↓
AVAILABLE / HELD / BLOCKED

115. Scanner Failure

If scan fails:

Do not mark CLEAN

Instead:

scan_status = UNKNOWN
attachment_state = HELD

where policy requires.

116. URL Inspection Pipeline

URL extracted
      ↓
Normalize safely
      ↓
Check scheme/host
      ↓
Reputation check optional
      ↓
Redirect analysis optional
      ↓
Create findings
      ↓
Determine:
ALLOW / MEDIATE / DISABLE

117. Sanitization Pipeline

Raw HTML
      ↓
Parse
      ↓
Remove unsafe elements
      ↓
Rewrite/disable remote resources
      ↓
Rewrite mediated links
      ↓
Create isolated safe representation

Raw provider HTML should never be rendered directly.

118. Sanitizer Failure

A sanitizer error should fall back to:

Safe plain-text representation

not raw HTML.

119. Observability Pipeline

Every major processing stage should emit operational metrics.

Examples:

events processed
event lag
sync lag
classification latency
security scan latency
command latency
provider error rate
retry count
dead-letter count
queue depth
migration throughput

120. Processing Latency

Different workflows have different urgency.

Suggested conceptual classes:

REALTIME
INTERACTIVE
BACKGROUND
BULK

Examples:

Incoming important message:
REALTIME

User archive:
INTERACTIVE

Rule simulation:
BACKGROUND

Historical migration:
BULK

121. Priority Queues

The architecture may use separate job priorities.

Example:

P0 SECURITY / USER INTERACTION
P1 ACTIVE MAIL
P2 NORMAL PROCESSING
P3 BACKGROUND
P4 MIGRATION / HISTORICAL

A massive historical migration must not delay an interactive archive request.

122. Backpressure

When downstream services are slow:

Queue work
Throttle producers where possible
Preserve checkpoints
Avoid memory-only backlog

123. AI Backpressure

If the model provider slows down:

Rules continue
Security continues
Mail sync continues
AI-needed items accumulate in classification queue

124. Security Backpressure

If deep attachment scanning slows:

Message metadata still processes
Attachments remain held
Security-safe parts remain accessible

125. Provider Rate Limiting

When provider rate limits apply:

Central rate-limit manager
      ↓
Throttle affected account/provider
      ↓
Retry later

Do not allow independent workers to overwhelm the provider.

126. Per-Provider Isolation

A Gmail outage should not halt:

Microsoft processing
iCloud search
Local rules
Existing canonical search

127. Per-Mailbox Isolation

A broken authentication state for one Microsoft mailbox should not invalidate another independent Microsoft mailbox.

128. Transaction Boundaries

Local database operations requiring atomicity should use transactions.

Example:

Create MailCommand
+
Create AuditEvent
+
Update approval state

where these must remain consistent.

Provider operations cannot participate in the same relational transaction.

129. Outbox Pattern

For reliable event publication, use a transactional outbox pattern or equivalent.

Example:

Database transaction:
Update message
Write OutboxEvent

Commit

Publisher:
Reads OutboxEvent
Publishes durable event

This prevents:

DB updated
but event lost

130. Inbox Pattern

Event consumers should record processed event IDs or equivalent idempotency state.

This protects against duplicate event delivery.

131. Saga-Style Workflows

Long multi-step workflows such as migration may use saga-style orchestration.

Example:

Copy
→ Validate
→ Reconcile
→ Mark complete

Each step is durable and recoverable.

132. Compensation

When full rollback is impossible, workflows should define compensating actions.

Example:

Move message A succeeded
Move message B failed

The system may:

Continue and report partial
or
Move A back

depending on command policy.

133. No Hidden Partial Mutations

If partial completion occurs, it must be visible.

Do not report:

Archive complete

when only 80% succeeded.

134. UI State Updates

Clients should receive updated application state through:

Request refresh
Polling
Server-sent events
WebSockets
Push notification

The exact transport may vary by client.

135. UI Should Consume Canonical State

The UI should not reconstruct workflow state from provider-specific responses.

Example:

Conversation state:
WAITING

comes from the Inbox Agent backend.

136. Optimistic UI Boundary

Suitable optimistic operations:

Archive
Mark read
Set priority

provided command reconciliation exists.

Less suitable:

Migration completion
Security release with failed scan
Bulk delete

137. Activity Feed Update

Successful user-visible operations should generate activity records.

Examples:

Archived 23 routine system notifications
Created draft response to Jane
Moved conversation to Waiting
Quarantined suspicious message

138. Audit vs Event Store

The internal event system and audit system serve different purposes.

EVENT
Drives processing

AUDIT
Explains durable meaningful actions

Not every low-level event must become a user-facing audit record.

139. Event Retention

Event retention should depend on operational requirements.

Short-lived transport events may expire sooner than:

Audit
Migration history
Security findings
Rule execution history

140. Replay

Selected event streams may support replay for:

Rebuilding projections
Recovering failed processing
Testing new classifiers

Replay must not accidentally repeat external provider mutations.

141. Replay Safety

Replayed events should default to:

ANALYSIS / REBUILD MODE

unless explicit command re-execution is intended.

142. Classifier Reprocessing

A new classifier version may re-evaluate historical message metadata without altering provider state.

Example:

Reclassify 10,000 messages

This should run as a background job.

143. Rule Simulation Processing

Rule simulation should:

Read historical canonical data
Evaluate candidate
Calculate matches
Detect exceptions/conflicts
Produce report

and never emit provider mutation commands.

144. Learning Processing Isolation

The learning engine should not use its own automated outputs as positive evidence.

Example prohibited feedback loop:

Agent auto-archives messages
→ learns user always archives them
→ strengthens own rule

Only valid human or externally authoritative evidence should count.

145. Correction Reprocessing

A user correction may cause:

Current message update
Conversation re-evaluation
Pattern update
Evaluation dataset update

It should not automatically retroactively change every historical message.

146. Reprocessing Command

If desired, the user may explicitly request:

Apply this new rule to existing messages.

That becomes a separate previewable bulk command.

147. Failure Containment

A failure in one stage should have a defined boundary.

Example:

AI classification fails

must not:

Rollback successful security inspection
Lose synchronized provider message
Break unrelated conversations

148. Poison Message Handling

A malformed or pathological message that repeatedly crashes processing should be isolated.

Possible flow:

Processing failure
→ retry
→ repeated deterministic failure
→ isolate message
→ mark processing error
→ safe quarantine
→ continue queue

One bad email must not block an entire mailbox.

149. Poison Attachment Handling

Same principle for attachments:

Attachment parser failure
→ hold attachment
→ record finding
→ continue message processing safely

150. Processing State Visibility

Administrative diagnostics should allow inspection of:

Current queue
Failed jobs
Dead-letter items
Sync checkpoints
Provider lag
Subscription status
Security backlog
Classification backlog
Migration progress

151. Manual Retry

Authorized admin UI should support:

Retry Event
Retry Job
Retry Command
Retry Provider Sync

where safe.

152. Manual Reconciliation

The user/admin should also be able to request:

Reconcile this mailbox

without performing a full migration or reconnect.

153. Processing History

For a message, diagnostics should be able to reconstruct:

10:02 discovered
10:02 normalized
10:02 security clean
10:02 rule no match
10:02 AI classified PROFESSIONAL
10:02 attention RESPOND
10:03 user opened
10:06 user sent reply in Outlook
10:07 sent-mail sync detected
10:07 attention WAITING

This is invaluable for troubleshooting.

154. Security Processing History

For quarantined mail:

09:12 discovered
09:12 SPF failed
09:12 URL mismatch detected
09:12 message quarantined
09:15 user reviewed
09:16 released once

155. Service Ownership

Recommended ownership:

Provider Adapter
Provider communication

Sync Service
Provider change discovery

Normalization Service
Canonical message representation

Security Service
Threat evaluation / quarantine

Rule Service
Deterministic processing

AI Service
Semantic ambiguity

Conversation Service
Attention / waiting / action state

Policy Service
Authorization

Command Service
Mutations

Learning Service
Corrections / patterns

Migration Service
Historical movement

Audit Service
Durable explanation

156. No Cross-Layer Shortcuts

Examples that should not occur:

UI directly calls Gmail

AI directly calls Graph

Rule writes provider folder itself

Migration changes attention state without migration policy

Security scanner edits message classification directly

Services communicate through defined contracts/events.

157. Initial Deployment vs Logical Services

The logical services do not require separate microservices.

Initial deployment may be:

One application codebase
One PostgreSQL database
Durable queue/job system
Multiple worker processes/functions

while preserving logical boundaries.

158. Avoid Premature Microservices

Do not create separate deployable services merely because logical service names exist.

Service boundaries are architectural boundaries first.

Physical separation should occur only when justified by:

Scaling
Security isolation
Failure isolation
Independent deployment
Technology requirements

159. Processing Environment Separation

Each environment should have separate:

Database
Queues
Provider OAuth configuration
Webhook endpoints
Secrets
Migration runs
Audit

Environments:

LOCAL
PREVIEW / DEVELOPMENT
PRODUCTION

160. Production Mail Protection

Preview/development environments should not accidentally mutate production mailboxes.

Default development mode should favor:

Mock providers
Synthetic messages
Read-only provider access
Explicit development mailbox

161. Synthetic Event Fixtures

Development/test fixtures should include:

Human question
Professional request
Receipt
Travel confirmation
Vercel success
Vercel failure
GitHub security alert
Marketing newsletter
Phishing email
Macro attachment
Prompt injection email
Duplicate message
Provider timeout

162. Pipeline Contract Tests

Tests should verify:

Security executes before ordinary automation
Quarantined mail does not auto-archive
Explicit rules beat AI
AI failure leaves message visible
User reply moves RESPOND appropriately
Outlook sent reply can create WAITING
Duplicate webhook does not duplicate command
Bulk command reports partial failures

163. Event Replay Tests

Tests should intentionally deliver:

Duplicate events
Out-of-order events
Late events
Repeated provider notifications

and verify stable outcomes.

164. Provider Failure Tests

Test:

Microsoft unavailable
Gmail rate limited
iCloud auth failure
Graph subscription expired
Gmail watch expired

Other providers should continue functioning.

165. Security Failure Tests

Test:

Sanitizer error
Scanner unavailable
Malformed HTML
Recursive zip bomb
Suspicious redirect

Expected behavior should fail safely.

166. AI Failure Tests

Test:

Timeout
Invalid JSON
Invalid enum
Hallucinated action
Prompt-injected email

No unauthorized mutation may result.

167. Reconciliation Tests

Test manual changes made outside Inbox Agent:

Reply in Outlook
Archive in Gmail
Mark read in Microsoft
Delete provider draft

Canonical state should eventually reconcile.

168. Operational SLO Considerations

Exact service-level objectives can be established later.

Useful measures include:

Time from provider notification to message visibility
Time to security decision
Time to attention classification
Interactive command completion latency
Provider synchronization lag

169. Processing Pipeline Acceptance Criteria

The orchestration architecture is acceptable when:

170. Canonical End-to-End Flow

For ordinary incoming mail:

PROVIDER
   ↓
NOTIFICATION / SYNC
   ↓
NORMALIZE
   ↓
IDENTITY RESOLUTION
   ↓
CONVERSATION RESOLUTION
   ↓
SECURITY
   ↓
QUARANTINE IF NEEDED
   ↓
DETERMINISTIC RULES
   ↓
AI IF NEEDED
   ↓
ATTENTION / PRIORITY / RETENTION
   ↓
POLICY
   ↓
LOW-RISK APPROVED COMMANDS
   ↓
PROVIDER
   ↓
RECONCILIATION
   ↓
UI / DAILY BRIEF / AUDIT

171. Core Orchestration Principle

The system should preserve this separation:

EVENT
Something happened

PROCESSOR
Determines what that event means

SECURITY
Determines whether content is safe to process

RULE
Defines deterministic behavior

AI
Handles semantic ambiguity

STATE
Represents what Inbox Agent currently believes

POLICY
Determines what may happen

COMMAND
Requests an external consequence

PROVIDER ADAPTER
Performs that consequence

RECONCILIATION
Determines what actually exists

AUDIT
Explains the complete history

No single processing component should be allowed to bypass this chain merely for implementation convenience.