Design brief
Design metrics ingestion, querying, and alerting for 20,000 instances exposing 500 series every 15 seconds. Ten million active series produce about 667,000 samples per second and roughly 922 GB daily at 16 raw bytes per sample, before overhead. Detect outages within a minute, support recent interactive queries, and retain cheaper historical aggregates. Define missing-data behavior and tenant capacity budgets.
Draw and explain
In Excalidraw, label collection, durable ingestion, time-series ownership, query execution, alert evaluation, and retention tiers. Trace a latency measurement into a service-wide percentile and an alert window. Show series identity, histogram compatibility, replay identity, and late-data handling. Estimate the effect of retention, replication, and label growth; identify where query resource limits protect ingestion during an incident.
Stress the design
An endpoint starts attaching request IDs as labels. Show how cardinality is bounded and how rejected measurements remain observable. Separately, a collector disappears while an error-rate alert is evaluating. Explain how absence differs from zero errors, how delayed samples affect the decision, and what detects the monitoring gap itself.
Review your design
Compare push and pull collection for failure visibility and operational cost. Explain histogram precision against sample volume, and what downsampling can preserve. Check counter resets, duplicate replica reads, ingestion backpressure, and tenant isolation. Choose signals for sample age, dropped data, active-series rejection, buffer exhaustion, and query saturation.
Capture assumptions, sizing, and the tradeoffs behind your diagram.
Review is your own assessment of the design.