Capacity Estimation for System Design Interviews¶

Capacity estimation turns a product description into a workload model. It helps you reason about how much traffic a system must handle, which resources will become constrained, how much data it will retain, and what capacity must remain available when something fails.

In a system design interview, the goal is not to guess the exact number of servers. It is to make assumptions explicit, use consistent units, show the arithmetic, identify the dominant bottleneck, and explain how measurements would replace estimates before production.

A useful estimate is a model with visible uncertainty—not a precise-looking number built from invented assumptions.

1. Clarify the workload before calculating¶

Start by asking what the system must do and what "working" means. A service with 100,000 daily users can have radically different resource needs depending on whether each user makes two reads or uploads several large files.

Clarify these dimensions:

Dimension	Questions to answer
Users and usage	Daily active users, actions per user, sessions, automation and bot traffic
Request mix	Reads, writes, searches, uploads, background jobs, expensive operations
Time profile	Average day, busy hour, short burst, seasonal peak, launch traffic
Payloads	Request and response sizes, file sizes, compression, fan-out
Data lifecycle	Records created, bytes per record, retention, deletion and growth
Performance	Latency objective, throughput, concurrency and acceptable degradation
Availability	Failure domains to survive, recovery objectives and regional constraints
Dependencies	Database limits, external API quotas, connection pools and managed-service limits

Do not collapse these into one "requests per second" number. A search request that scans a large dataset may cost much more than a redirect that performs a small key lookup. A write may trigger validation, indexing and downstream work that a read does not.

Also distinguish a product requirement from an assumption. "The service must support 10,000 requests per second" is a requirement only if someone has established it. "We expect 10,000 requests per second" is a forecast that needs evidence.

2. Convert user activity into request rates¶

Use a time unit that makes the workload easy to compare. For daily usage:

Daily requests = daily active users × actions per user per day × requests per action.
Average requests per second = daily requests ÷ 86,400.
Peak requests per second = average requests per second × peak-to-average factor.

The peak factor must come from observed traffic, a comparable product, a launch plan or an explicit interview assumption. There is no universal factor that works for every system. Calculate it separately for different endpoints when their usage patterns differ.

Worked traffic example¶

Assume a URL redirection service has the following illustrative workload:

20 million redirects per day.
200,000 new links per day.
A simplified 6× peak-to-average factor for both request classes.

The resulting rates are:

Workload	Daily volume	Average rate	Estimated peak
Redirects	20,000,000	231.5 req/s	1,389 req/s
Link creation	200,000	2.3 req/s	13.9 req/s

The service is read-heavy: roughly 100 redirects for every new link. That ratio matters when you choose database indexes, cache capacity, connection pools and read-replica strategy.

The 6× factor is only an assumption for this worked example. A real system should derive separate peak curves for redirects and writes from time-series data. A marketing campaign may create a sharp redirect spike without a matching increase in link creation.

Model request cost, not just request count¶

Break the workload into request classes:

Cheap reads and redirects.
Database-heavy reads and searches.
Writes that update indexes or enforce invariants.
Requests that call external services.
Uploads, reports or batch jobs with large resource footprints.

For each class, estimate CPU time, database operations, bytes transferred and downstream calls where possible. If request cost varies substantially, use a weighted workload or resource consumption rather than treating every request as equivalent.

This distinction is important during overload. Google's SRE guidance describes why queries per second can be a misleading capacity signal when requests have different costs, and why measuring resource consumption can be more useful than counting requests alone.

3. Estimate in-flight concurrency with Little's Law¶

Request rate tells you how quickly work arrives. It does not tell you how much work is simultaneously in progress.

Little's Law relates the long-run average number of items in a stable system to the effective arrival rate and the average time each item spends in that system:

Average in-flight work = arrival rate × average time in the system

In notation, this is commonly written as L = λW.

For the redirect example, assume the service receives 1,389 requests per second during a sustained busy period and the measured mean end-to-end response time is 80 milliseconds:

1,389 requests/second × 0.080 seconds ≈ 111 requests in flight on average.

This is useful when reasoning about connection pools, asynchronous tasks, memory held by requests and the amount of work waiting for dependencies.

Use the mean time in the system for this calculation, not p95 or p99 latency. Little's Law relates averages over a stable observation period. A short, rapidly changing burst is not automatically a steady-state queueing system; inspect time-varying queue length and validate burst behavior with load tests.

Concurrency is also not the same as thread count. An asynchronous server can have many in-flight I/O operations with far fewer operating-system threads. Conversely, a thread-per-request server can exhaust its thread or memory budget before CPU is fully used.

If you are sizing a database pool, estimate database operations separately. One API request may issue zero, one or several queries, and the mean query duration can be very different from end-to-end request latency.

4. Estimate storage from data creation and retention¶

Storage estimates should start with the data lifecycle, not the database product.

A basic logical-storage estimate is:

Daily records × logical bytes per record × retention days

For the example, assume:

200,000 new links per day.
1 KiB of logical row data per link.
365 days of retained records.

That is approximately 195.3 MiB of logical data per day, or 69.6 GiB over a year.

This is not the physical disk requirement. Add or measure the costs that the simple row estimate omits:

Primary and secondary indexes.
Tuple headers, page fill, alignment and database bloat.
Write-ahead logs and replication retention.
Replicas and standby copies.
Backups, snapshots and point-in-time recovery retention.
Temporary files, migrations and operational headroom.

Do not apply one unexplained multiplier and call the result a database size. Index overhead depends on schema and access patterns; WAL depends on write volume and retention; backups depend on policy and change rate. Estimate these components independently, then validate them against a representative dataset.

Also state whether the system retains all records, expires them, archives them or supports user deletion. A five-year retention policy can dominate a system whose request rate is modest.

For media or document systems, separate metadata from object storage. Estimate object count, average object size, upload rate, download rate, lifecycle transitions and CDN delivery independently. The database row may be tiny while the stored objects dominate capacity and cost.

5. Estimate network bandwidth from payloads¶

For a first-order bandwidth estimate:

Bytes per second = requests per second × average payload bytes

Convert bytes per second to bits per second by multiplying by eight. Calculate ingress and egress separately because response sizes are often much larger than request sizes.

If each peak redirect returns an average 1.5 KiB response:

1,389 redirects/second × 1.5 KiB ≈ 2.03 MiB/second.
That is approximately 17.1 Mbit/second of response payload.

This estimate excludes HTTP headers, TLS framing, TCP/IP overhead, retransmissions and other traffic. It also assumes the stated response size is representative. Measure the payload distribution rather than relying on the smallest response.

For uploads, large downloads and media delivery, network capacity may be the bottleneck even when application CPU is low. A CDN can move delivery work away from the origin, but it does not eliminate origin traffic for cache misses, writes or uncached personalized responses.

6. Translate workload into compute capacity¶

The strongest first estimate for service replicas comes from a representative load test:

Required serving replicas = ceiling(peak workload ÷ safe workload per replica)

"Safe workload per replica" is not the highest throughput a machine can briefly achieve. It is the throughput the instance can sustain while meeting the agreed latency and error objectives, using the real request mix and realistic dependencies.

Suppose a load test shows that one service replica can sustain 230 redirects per second while meeting the target latency and error rate. The estimated peak of 1,389 redirects per second requires seven serving replicas:

1,389 ÷ 230 = 6.04.
Round up to seven replicas.
Seven replicas provide 1,610 requests per second of measured serving capacity.

That is a model for the chosen workload and test environment—not a promise that every instance type, code version or region will achieve the same rate.

Use CPU as a cross-check¶

If profiling shows an average of 2 milliseconds of CPU time per redirect, the peak CPU demand is approximately:

1,389 requests/second × 0.002 CPU-seconds/request = 2.78 CPU cores at full utilization.

If the design target is 65% CPU utilization, a rough CPU requirement is:

2.78 ÷ 0.65 ≈ 4.27 vCPUs.

This is a sanity check, not a replacement for load testing. The service may be constrained by database connections, memory, garbage collection, locks, network throughput or a downstream quota before CPU reaches the target. Use measured throughput to size replicas and resource profiles to explain the bottleneck.

A useful capacity model is bounded by the first resource that reaches its safe limit:

Application CPU and memory.
Database query throughput, IOPS, locks and connection capacity.
Network ingress and egress.
External-service quotas and latency.
Queue consumers and background-worker throughput.
Per-tenant or per-request-class limits.

Count shared connections across replicas¶

A frequent sizing mistake is to configure a large connection pool per instance and forget that every replica creates its own pool.

For example, nine replicas with a pool limit of 30 can open up to 270 application database connections before accounting for migrations, monitoring, administrative sessions or other services. Compare the aggregate against the database's usable connection budget, not just the per-process setting.

A connection pool is a concurrency control, not a guarantee that the database can execute that many expensive queries efficiently. Consider a pooler, query concurrency limits and workload isolation when the database is the bottleneck.

7. Add headroom for the failure model you actually need¶

Capacity must be evaluated after the failure the system is expected to survive.

Ask whether the design must continue through:

One instance or host failure.
An availability-zone outage.
A regional outage.
A deployment that temporarily removes instances.
A dependency slowdown that reduces effective throughput.
A traffic surge that exceeds the forecast.

If the example must continue serving peak traffic after losing two replicas, it needs seven replicas' worth of safe serving capacity after those failures. With the measured 230 requests per second per replica, provision nine replicas: seven can remain available if two are lost.

This is an N+2-style calculation. It is not a universal rule or a substitute for placement. If all nine replicas sit in the same availability zone, the design does not survive losing that zone. Distribute capacity across the failure domains that matter and recalculate the surviving capacity for each scenario.

Headroom also has a cost. Too little creates saturation and cascading failures; too much can waste money and hide poor efficiency. Choose a target using the service's SLO, traffic variability, scale-up delay, recovery time and failure requirements.

Autoscaling is useful, but it is not instantaneous. New instances need to start, pass readiness checks, warm caches, establish connections and obtain quota. For predictable launches or scheduled peaks, forecast and provision ahead of time rather than assuming reactive scaling will arrive before the system saturates.

8. Validate the model with load tests and production data¶

A spreadsheet gives you a hypothesis. A representative test tells you whether the system behaves as expected.

A useful validation plan includes:

Representative data. Use realistic row counts, key distributions, payload sizes and hot-key patterns. Empty databases and uniform test data can hide production bottlenecks.
Real request mix. Reproduce the proportions of reads, writes, searches, expensive operations and background work.
Steady load. Run long enough to observe connection pools, memory growth, garbage collection, storage behavior and queue accumulation.
Ramps and bursts. Test gradual growth and sudden spikes separately. Autoscaling and caches respond differently to each.
Failure tests. Remove an instance or zone, slow a dependency, exhaust a quota and verify the expected degraded behavior.
Tail latency and errors. Record p50, p95 and p99 latency, timeouts, rejected requests and successful throughput.
Resource signals. Track CPU, memory, network, disk, database connections, query latency, queue depth and per-request resource cost.
Forecast review. Compare actual traffic and resource use with the assumptions. Update the model when request mix or software behavior changes.

Stop increasing load when the service violates its latency or error objective, begins accumulating an unstable queue, or exhausts a dependency. The useful capacity number is the sustainable operating point before the system becomes unstable—not the largest number printed by a benchmark.

For overload behavior, decide which work can be rejected, deferred or degraded safely. Admission control and load shedding protect the critical path when demand exceeds supply. The handbook's distributed rate-limiting guide covers shared quotas and admission decisions; its caching guide covers cache load and consistency trade-offs.

9. Common estimation mistakes¶

Sizing from daily averages only. Averages hide peak hours, short bursts and uneven regional traffic.
Treating all requests as equal. QPS alone hides expensive endpoints, fan-out and different database costs.
Using p95 in Little's Law. The law uses average arrival rate and average time over a stable system.
Confusing latency with CPU time. An 80 ms request may spend most of that time waiting on a database or network.
Ignoring retries. A dependency slowdown can increase attempts and amplify offered load. Estimate original requests and retry traffic separately.
Forgetting connection multiplication. Pool limits apply per process; aggregate them across every replica and service.
Counting only logical data. Indexes, WAL, replicas, backups and retention can materially increase physical storage.
Assuming autoscaling is immediate. Scale-up delay, warm-up and provider quotas matter during bursts.
Using N+2 without a failure-domain model. Extra instances in one zone do not protect against losing that zone.
Treating an estimate as a guarantee. Validate it with load tests and update it when real traffic diverges.

10. A small, reproducible capacity worksheet¶

The following Python example performs the arithmetic used in the worked scenario. Every input is an explicit assumption; replace it with measured traffic, payloads, latency and per-replica test results before using the result for a real deployment.

from math import ceil

SECONDS_PER_DAY = 86_400
BYTES_PER_GIB = 1024**3

daily_redirects = 20_000_000
daily_links = 200_000
peak_factor = 6.0
mean_latency_ms = 80
response_kib = 1.5
logical_row_kib = 1.0
safe_rps_per_replica = 230.0
replica_failure_reserve = 2
cpu_seconds_per_request = 0.002
target_cpu_utilization = 0.65

average_redirect_rps = daily_redirects / SECONDS_PER_DAY
peak_redirect_rps = average_redirect_rps * peak_factor
average_write_rps = daily_links / SECONDS_PER_DAY
peak_write_rps = average_write_rps * peak_factor

# Little's Law: average in-flight work during a sustained peak window.
in_flight = peak_redirect_rps * (mean_latency_ms / 1000)

# Logical row bytes only; indexes, WAL, replicas and backups are excluded.
annual_logical_gib = (
    daily_links * logical_row_kib * 1024 * 365 / BYTES_PER_GIB
)

# Response payload bandwidth; protocol overhead is excluded.
egress_mbps = peak_redirect_rps * response_kib * 1024 * 8 / 1_000_000

active_replicas = ceil(peak_redirect_rps / safe_rps_per_replica)
total_replicas = active_replicas + replica_failure_reserve

cpu_cores_at_target = (
    peak_redirect_rps * cpu_seconds_per_request / target_cpu_utilization
)

print(f"Average redirects: {average_redirect_rps:.1f} req/s")
print(f"Peak redirects: {peak_redirect_rps:.0f} req/s")
print(f"Peak link writes: {peak_write_rps:.1f} req/s")
print(f"Average in-flight requests at peak: {in_flight:.0f}")
print(f"Annual logical row data: {annual_logical_gib:.1f} GiB")
print(f"Peak response payload: {egress_mbps:.1f} Mbit/s")
print(f"Serving replicas required: {active_replicas}")
print(f"Replicas with two failures reserved: {total_replicas}")
print(f"CPU cores at target utilization: {cpu_cores_at_target:.2f}")

Expected rounded results:

Estimate	Result
Average redirects	231.5 req/s
Peak redirects	1,389 req/s
Peak link writes	13.9 req/s
Average in-flight requests during peak	111
Annual logical row data	69.6 GiB
Peak response payload	17.1 Mbit/s
Serving replicas at 230 safe req/s each	7
Replicas with two failures reserved	9
CPU requirement at 65% target	4.27 vCPUs

The worksheet intentionally keeps several quantities separate. Logical storage is not disk allocation; payload bandwidth is not total network traffic; CPU-derived cores are not automatically the replica count; and the N+2 reserve is valid only for the stated instance-failure model.

11. Interview questions with reasoned answers¶

Q1. Why estimate peak traffic instead of dividing daily users by 86,400?¶

Daily averages are useful for a first pass, but capacity is usually constrained by busy periods. Estimate average traffic first, then derive a peak profile from measurements or an explicit assumption. Also account for bursts that are shorter than the autoscaling response time.

Q2. Is a 3× or 5× peak factor a standard assumption?¶

No. A factor is a workload assumption, not a law. A consumer product, an internal API and a scheduled batch system have different traffic shapes. Ask for historical metrics or model low, expected and high scenarios.

Q3. How do you estimate concurrent requests?¶

Use Little's Law with the average arrival rate and average time spent in the system over a stable period. It estimates average in-flight work, not the number of threads, maximum burst concurrency or a safe queue limit. Validate short bursts separately.

Q4. Why is QPS not enough to size a service?¶

Requests have different resource costs. A report query, a cache hit and a lightweight redirect can each count as one request while consuming very different CPU, memory, database and network resources. Measure representative request classes and their resource use.

Q5. How do you choose the number of replicas?¶

Load-test a representative instance against the latency and error objectives. Divide peak workload by the safe sustained throughput per instance and round up. Then add capacity for the explicit failure and deployment scenarios, placing replicas across the required failure domains.

Q6. Does 65% CPU utilization guarantee enough headroom?¶

No. It is only one signal. The database pool, memory, garbage collector, network, locks or an external quota may saturate first. A target utilization should be justified by observed behavior and the service's SLO.

Q7. Does autoscaling remove the need for capacity planning?¶

No. Autoscaling needs time to react and may be limited by quotas, startup time, warm-up or unavailable capacity. A service must survive the load it receives while scaling catches up. Predictable peaks should be forecast and prepared for.

Q8. How should you communicate uncertainty in an interview?¶

State the assumption, show the calculation and explain what evidence would replace it. Give a low/base/high range when the input is uncertain, then identify which variable changes the result most. A transparent range is more credible than false precision.

12. Interview checklist¶

Before presenting a capacity estimate, confirm that you can:

Define the workload, request mix, latency objective and failure model.
Convert user activity into average and peak rates with consistent units.
Distinguish cheap requests from expensive resource-consuming operations.
Estimate concurrency with average latency and explain Little's Law's limits.
Calculate logical data growth and separately account for physical storage overhead.
Estimate ingress and egress from realistic payload sizes.
Use load-tested safe throughput to size replicas.
Cross-check CPU, memory, database connections, network and downstream quotas.
Explain headroom, failure domains, autoscaling delay and overload behavior.
Identify the assumptions that need production measurements.

The best capacity estimate is not the one with the most arithmetic. It is the one that makes the workload explicit, exposes the likely bottleneck, survives the stated failure scenario, and gives the team a clear plan for validating the model.