Design brief
Design asynchronous order confirmations and delivery updates for five million events daily, averaging two channel deliveries each. Peak demand approaches 1,160 attempts per second before retries, while one provider allows 500 requests per second. Aim for 99% of ordinary confirmations to be attempted within one minute. Define acceptance, attempted delivery, and provider confirmation separately, including preference and unsubscribe behavior.
Draw and explain
In Excalidraw, label the business transaction boundary, durable notification ownership, channel scheduling, provider boundaries, and callback ingress. Trace an order commit to an eventual delivery attempt. Show event identity, delivery identity, attempt ownership, status transitions, and where current preferences are checked. Estimate backlog growth during the peak and explain how draining it affects the delay objective.
Stress the design
The provider accepts a message, but the worker times out before recording the outcome. Show what a replacement worker knows and the duplicate risk with and without provider idempotency. Separately, throttle a provider during a sustained burst. Explain how retries, new confirmations, and permanent address failures compete for capacity.
Review your design
Check the business-event handoff, duplicate intake, stale worker results, and authenticated callbacks. Compare retry budgets and provider failover against duplicate delivery and spend. Explain isolation between urgent and bulk work, privacy of contact data, and operational recovery for exhausted jobs. Choose signals for oldest pending age, unknown outcomes, throttling, and suppression.
Capture assumptions, sizing, and the tradeoffs behind your diagram.
Review is your own assessment of the design.