Back-of-the-Envelope Calculations

A back-of-the-envelope calculation turns product behavior into demand: requests/s, database work/s, bytes/s, occupied resources, and retained bytes. Its value is exposing the assumption that changes a design. A request count alone cannot tell you whether the first constraint will be CPU, database connections, network traffic, or storage.

Keep units attached to each quantity, and distinguish an input you chose from a limit you measured. This lesson uses one board API throughout. Every number describing its workload is a planning assumption, not a published service capacity.

The examples require Python 3.11+ and its standard library. Download the complete calculation fixture, extract it, and run python3 capacity.py and python3 -m unittest -v test_capacity.py from that directory. The fixture includes the model, sensitivity report, and boundary checks; it makes no network calls.

Turn behavior into rates

Assume 240,000 daily active users. Each makes 36 board-read requests and nine board-write requests per day. These count API requests, so a screen that calls three endpoints contributes three requests. We choose an eight-times daily-average rate for a one-hour busy window, keeping the same read/write mix.

users = 240_000                         # users/day
reads_day = users * 36                  # 8,640,000 read requests/day
writes_day = users * 9                  # 2,160,000 write requests/day
seconds_day = 86_400                    # seconds/day

average_reads = reads_day / seconds_day # 100 read requests/s
average_writes = writes_day / seconds_day  # 25 write requests/s
peak_reads = average_reads * 8          # 800 read requests/s
peak_writes = average_writes * 8        # 200 write requests/s

daily_requests = reads_day + writes_day
busy_requests = (peak_reads + peak_writes) * 3600
remaining_rps = (daily_requests - busy_requests) / (seconds_day - 3600)
assert round(remaining_rps, 2) == 86.96

The dimensional cancellation is users/day × requests/user = requests/day, then requests/day ÷ seconds/day = requests/s. The result is 125 requests/s averaged over the day and 1,000 requests/s averaged over the chosen busy window. “Peak” needs a window: a one-second burst can exceed a one-hour mean.

The one-hour window consumes 3.6 million of the day's 10.8 million requests. The remaining 23 hours average about 87 requests/s. Extending that same 1,000 requests/s plateau beyond three hours would consume more than the entire daily budget; the fixture rejects that inconsistent combination. Multiplying the busy rate by 86,400 would invent a different daily workload.

Daily active users also leave out bots, background refreshes, retries, and batch jobs. Add them as explicit request streams if they exist. If writes spike independently of reads, give each a separate busy-window factor rather than preserve this convenient mix.

Count the work behind each request

Our board service does one authorization SQL read on every request. For a board read, it then looks in a cache: a hit avoids board-data queries; a miss issues two SQL reads and fills the cache. A write performs three SQL modifications: append an event, update current board state, and insert an outbox record.

Assume a 90% cache hit ratio among read requests. The SQL calculation follows the request path:

read_rps, write_rps = 800, 200           # requests/s in busy window
request_rps = read_rps + write_rps
cache_misses = read_rps * (1 - 0.90)     # 80 misses/s

sql_reads = request_rps + cache_misses * 2  # 1,160 SELECT statements/s
sql_writes = write_rps * 3              # 600 modifications/s
sql_total = sql_reads + sql_writes      # 1,760 SQL statements/s
assert round(sql_total) == 1760

# Application payload sizes, before compression or protocol overhead.
ingress = read_rps * 512 + write_rps * 2048   # 819,200 bytes/s
egress = read_rps * 8192 + write_rps * 1024   # 6,758,400 bytes/s
assert egress / 1024**2 == 6.4453125         # MiB/s
assert egress * 8 / 1_000_000 == 54.0672     # Mbit/s

There are also 800 cache lookups/s and 80 fills/s, before any write-side cache maintenance. At the daily-average traffic rate, the model produces 145 SQL reads/s and 75 modifications/s. Here, “QPS” counts individual SQL statements, including statements inside a transaction; state that boundary when comparing a dashboard that counts transactions instead.

This is logical read/write amplification: one API write becomes three SQL modifications. It is not a three-times disk-write estimate. Index maintenance, transaction logs, page caching, batching, and storage layout determine physical I/O. Likewise, one SQL read might touch several pages or avoid disk entirely.

A cache hit does not remove authorization in this design. Even a perfect board cache leaves 1,000 SQL reads/s plus 600 modifications/s at peak. With no cache hits, board misses add 1,600 reads/s, bringing the total to 3,200 statements/s. This identifies which queries the cache can actually save; the Caching lesson examines its freshness and failure behavior.

The payload calculation assumes 512-byte read requests, 2 KiB write requests, 8 KiB read responses, and 1 KiB write responses. It budgets 0.78125 MiB/s inbound and 6.4453125 MiB/s outbound at peak. MiB uses powers of 1,024; Mbit/s uses powers of 1,000, with eight bits/byte. TLS, HTTP headers, retransmissions, internal service traffic, and replica traffic require additional budgets.

For a transfer bill, use daily volumes: 8,640,000 × 8,192 + 2,160,000 × 1,024 = 72,990,720,000 bytes/day, about 67.98 GiB/day. Peak bandwidth and total billable transfer answer different questions.

Translate rates into occupied resources

Little's law relates the average number of items inside a boundary to the average arrival rate and average time spent there: L = λW. Use the same observation window and boundary for all three quantities. In a stable busy window with a 120 ms mean request duration:

1,000 requests/s × 0.120 seconds = 120 requests in flight on average.

That includes requests waiting for dependencies. It does not mean 120 CPU cores or 120 database connections. Assume each request holds at most one database connection, and the mean total connection-hold time across all request types is 32 ms, including transactions. Then average occupied connections are 1,000 × 0.032 = 32. Count time from checkout to return, including waits inside a transaction; a SQL execution-time metric alone misses that occupancy.

The fixture integrates a small deterministic set of request lifetimes. The following code uses its occupancy() function, which counts active requests over time and requires every request to start and finish inside the observation window:

from capacity import occupancy

spread = [(0, 2), (1, 3), (2, 4), (3, 5)]  # arrival/departure seconds
burst = [(0, 2)] * 4
for intervals in (spread, burst):
    mean, rate, duration, peak = occupancy(intervals, 5)
    assert mean == rate * duration == 1.6  # 0.8 arrivals/s × 2 seconds
    print(peak)                           # 2, then 4 simultaneous requests

Equal averages can hide different instantaneous peaks. Neither 120 in-flight requests nor 32 occupied connections is therefore a safe hard limit. Test burstiness, tail latency, pool waits, and dependency degradation. Do not multiply a mean rate by p99 latency and call the result Little's-law mean occupancy.

If mean request time grows to 600 ms and mean connection hold grows to 200 ms while arrivals stay at 1,000 requests/s, the model needs 600 requests in flight and 200 occupied connections. A bounded pool can turn that demand into waits or rejections. If requests accumulate without bound, the stable-window assumption fails; calculate backlog growth separately rather than treat the concurrency estimate as a sustainable state.

CPU requires its own input. At an assumed 6 ms of CPU time/request, this workload demands six CPU-seconds each second. Dividing by a chosen 60% utilization budget gives ten equivalent continuously available execution units. Actual throughput per vCPU depends on the runtime, processor, contention, and workload. This arithmetic motivates a load test; the Numbers to Know lesson covers concrete hardware and service specifications.

Retain data and test the assumptions

Each of the 2.16 million daily writes appends one immutable 768-byte event. Budget 30 days of events, 25% extra row overhead, indexes equal to 50% of row bytes, three total copies, and 30% free space. These overhead ratios are assumptions to replace with measurements from representative data.

writes_day = 2_160_000                  # events/day
payload = writes_day * 30 * 768         # retained payload bytes
rows = payload * 1.25                  # row metadata/layout allowance
indexed = rows * 1.50                   # rows plus all indexes, one copy
live_copies = indexed * 3               # leader + two replicas, total
allocated = live_copies / (1 - 0.30)    # live data occupies 70% of allocation

assert payload == 49_766_400_000
assert live_copies == 279_936_000_000
assert round(allocated / 1024**3, 2) == 372.44  # GiB across all copies

Adding 30% to live bytes would leave only about 23% of the resulting allocation free; divide by the used fraction to reserve 30%. Count three copies once. This is an event-storage component budget: current board snapshots, outbox retention, logs, backups, and temporary growth during maintenance need separate line items. Expiring records may not immediately shrink allocated database files.

The fixture varies inputs independently to expose the assumptions worth investigating:

The cache row deliberately holds latency constant to isolate query demand; a real miss increase may also slow requests. The slow-dependency row holds SQL count constant to isolate occupancy. Combine adverse assumptions when testing the system, including failure of a replica or a cold cache.

Longer retention changes storage without changing foreground request rate. More indexes may reduce read time while increasing modification work and retained bytes. Replicas add copies and traffic without necessarily increasing write capacity. These are distinct cost drivers: stored byte-time, transferred bytes, database work, and CPU time. Apply a dated price sheet only after choosing the relevant billing units and deployment shape.

The next useful measurement is the input with the largest decision impact: endpoint fan-out, cache misses during a cold start, connection-hold time, or actual bytes per retained event. Replace assumptions one by one and keep the arithmetic executable so the design changes with the evidence.