Skip to content

Application Caching: Consistency, Invalidation, and Stampede Protection

A cache can make a read-heavy service faster and reduce pressure on its primary database. It can also serve an old value after a write, multiply database traffic when a popular key expires, or expose one tenant's data to another if keys are designed carelessly.

The important design question is not simply which cache should we use? It is: what staleness can this operation tolerate, how will cached data be refreshed or invalidated, and what happens when many requests need the same missing value at once?

This guide focuses on application data caches such as an in-process map or Redis in front of a relational database. It does not cover PostgreSQL shared buffers, operating-system page caches, HTTP proxy internals, or semantic caching for LLM responses. Those are related but have different ownership and correctness boundaries.

1. Treat the cache as a derived copy

For most backend services, the database or another durable system remains the source of truth. The cache is a derived copy that can be evicted, restarted, or rebuilt. A cache hit is useful only when the cached representation is safe to return for the current request.

A useful first decision is whether the data is a good cache candidate:

  • Many requests read the same value or representation.
  • Recomputing or retrieving it from the source is meaningfully expensive.
  • The product can define an acceptable freshness window.
  • The key can include every dimension that changes the result.
  • A cache miss or cache outage has a safe fallback.

Caching is usually a poor fit when most requests are unique, the origin read is already cheap, or stale data would violate a security or business invariant. Avoid using a cache as the authority for balances, authorization decisions, inventory reservations, or other state where a stale answer can authorize an irreversible action. A cache may still hold a display projection of such data, but the write or decision must be checked against the authoritative store.

A cache should not be introduced only because Redis is already available. Every extra cache adds a network dependency, memory pressure, operational alerts, key lifecycle rules, and another place where stale data can exist.

2. Understand the common caching patterns

Cache-aside (lazy loading)

The application owns the read-through behavior:

  1. Read the key from the cache.
  2. On a hit, return the cached value.
  3. On a miss, read the authoritative store.
  4. Store the result with an expiry policy.
  5. Return the result.

On a write, update and commit the source of truth, then invalidate the corresponding cache entry. The next read repopulates it.

READ
Client -> Service -> Cache
                    | hit  -> return value
                    | miss
                    v
                 Database -> populate cache -> return value

WRITE
Client -> Service -> Database transaction commits
                    -> invalidate cache key

Cache-aside is a useful default because it loads only data that is requested and a newly empty cache can warm itself as traffic arrives. Its trade-off is that the application must maintain invalidation, and a cache miss adds a cache lookup, an origin read, and a cache write. It does not guarantee consistency between the two stores.

Compare the alternatives

Pattern Read behavior Write behavior Main trade-off
Cache-aside Application checks cache; loads origin on miss Update origin, then invalidate cache Simple and demand-driven; stale windows and invalidation are application concerns
Read-through Cache layer loads from origin on miss Usually separate write policy Centralizes loading, but the cache layer becomes part of the request path
Write-through Reads use cache Write path updates origin and cache Can improve read-after-write behavior, but adds write latency and still needs failure handling across two systems
Write-behind / write-back Reads use cache Cache acknowledges before asynchronously persisting to origin Lower write latency, but data loss, replay, ordering, and durability become critical design problems
Refresh-ahead Reads use cache Refresh before expected expiry Can avoid a miss for predictable hot keys, but may refresh data nobody needs

Do not describe a cache and database update as atomic merely because both calls happen in one application function. If they are separate systems, a process crash or network timeout can leave one updated and the other unchanged. Write-through still needs an explicit failure and reconciliation strategy. Write-behind is especially risky when the cache is not a durable log.

3. Design cache keys as part of the data contract

A cache key must identify the exact representation being cached. A key that is too broad can return incorrect or unauthorized data; a key that is too specific can destroy the hit rate and create unbounded cardinality.

For a tenant-scoped product view, a key might look like:

catalog:v3:tenant:acme:locale:en-IN:product:42

Consider the dimensions that can change the response:

  • Tenant or organization: include the trusted tenant identity when data is tenant-scoped.
  • Resource identifier: use a stable identifier, not a display name that can change.
  • Representation version: include a version when the serialized shape or interpretation changes.
  • Locale, currency, or region: include them when they affect the result.
  • Filters and pagination: normalize and include query parameters when caching list responses.
  • Authorization context: avoid caching permission-filtered responses unless the key and invalidation policy account for the principal and policy version. Re-check authorization independently; a cache key is not an authorization control.

Never build a tenant key from an untrusted body field when the authenticated principal already establishes the tenant. Never assume that a random or hard-to-guess key prevents object-level authorization failures. Resolve identity and permission from trusted server-side state before returning the cached representation.

Avoid putting secrets, access tokens, or unnecessary personal data in keys. Keys may appear in cache diagnostics, metrics, command logs, and operational tooling. Normalize input before constructing keys, cap key length, and avoid allowing callers to create arbitrary high-cardinality namespaces. Also decide whether cached values are immutable: returning the same mutable object to multiple callers can let one request change what another request observes. Cache immutable representations or copy values at a well-defined boundary.

4. Choose an explicit freshness policy

A time-to-live (TTL) is a bound on how long an entry may remain eligible for use under that cache's policy. It is not proof that the value is correct until the last millisecond, and it does not synchronize the cache with the database.

Choose TTL from a business staleness budget, not from a copied example:

  • Product descriptions may tolerate minutes of staleness.
  • A dashboard aggregate may tolerate a short delay if it is clearly approximate.
  • A permission change or account disablement may require immediate authoritative enforcement rather than a long-lived cache entry.
  • A stock display may be cached, while the actual reservation must use an atomic database or inventory-service operation.

A shorter TTL reduces the maximum age of an untouched entry but increases origin traffic and refresh work. A longer TTL improves reuse but increases the time stale data can survive if invalidation is missed. Use TTL as a safety net alongside deliberate invalidation, not as the only correctness mechanism for data that changes frequently.

Avoid synchronized expiry

If thousands of keys are populated with the same fixed TTL during a deployment or bulk refresh, they can expire together. Add bounded random jitter to the expiry time so keys do not all become cold at once:

expiry_seconds = base_ttl_seconds + random.uniform(0, jitter_seconds)

Jitter reduces synchronized expiry across many keys. It does not stop a stampede on one extremely popular key; that needs request coalescing, controlled refresh, or another per-key strategy.

Negative caching

A short-lived negative entry can protect the origin from repeated lookups for a resource that does not exist. Keep the negative TTL short, scope it to the same tenant and lookup dimensions as a positive entry, and invalidate it when the resource is created. Otherwise a newly created record may remain invisible until the negative entry expires.

Do not cache transient origin failures as if they were "not found". Distinguish a confirmed absence from a timeout, permission error, or database outage.

5. Invalidate after the authoritative write—and understand the race

For a basic cache-aside write, the safe ordering is:

  1. Validate and authorize the command.
  2. Commit the change to the authoritative store.
  3. Invalidate the affected cache key after commit.
  4. Let a later read repopulate from the source of truth.

Invalidating before the database commit is dangerous: a reader can miss the cache, read the old committed row, and repopulate the old value before the writer commits.

Even update-then-delete has a subtler race:

Reader A                         Writer B
--------                         --------
cache miss
read old database value
                                 update database and commit
                                 delete cache key
write old value into cache

The delete happened, but an in-flight reader repopulated stale data afterwards. TTL eventually limits the damage, but it does not make the race disappear.

Choose a mitigation based on the required consistency:

  • Versioned keys: associate a cache entry with a source version or generation. A reader may publish a value only if the version it read is still current. A write advances the version, so an older in-flight read cannot become the current entry.
  • Compare-and-set / atomic scripts: store a version with the value and atomically reject a fill when its generation no longer matches. The comparison and write must be one atomic operation at the cache boundary.
  • Serialize per-key work: coordinate readers and writers for a key when the stronger ordering is worth the latency and operational cost. A distributed lock is not automatically a correctness proof; lease expiry and paused processes must be considered.
  • Write-through or read-your-writes bypass: update the cache after the authoritative commit, or let the caller that just wrote read from the authoritative path for a defined period. These approaches still need recovery for partial failure between systems.
  • Accept bounded staleness: for non-critical display data, document the stale window and rely on invalidation plus TTL. Do not claim linearizability when the design only provides eventual convergence.

If invalidation must reach multiple services, publish a durable change event after the database commit. If losing the event would leave stale copies for too long, record the event in the same database transaction as the business change and relay it asynchronously. Consumers should be idempotent and tolerate duplicate or delayed invalidations. The transactional outbox pattern solves the database/event dual-write problem; it does not make every downstream cache instantly consistent. For the event-publication mechanics, see the separate AKSTechies transactional outbox guide rather than duplicating that article here.

A local in-process cache is private to one application instance. Invalidating one process does not invalidate another. Use a shared cache, a reliable invalidation channel, bounded local TTLs, or a combination. If a client-side cache depends on server tracking or pub/sub invalidations, define what happens when the connection drops; a lost invalidation stream may require flushing the local copy.

6. Prevent cache stampedes

A cache stampede occurs when many callers observe the same popular key as missing or expired and all query the origin at once. The cache has moved the bottleneck rather than removed it: the database receives a burst precisely when it is already under pressure.

The mitigation should match the topology and the cost of the origin read.

Per-process single-flight

Within one process, keep one in-flight load per key. The first caller performs the origin read; other callers wait and reuse the result. Re-check the cache after acquiring the per-key lock because another caller may have filled it while the current caller waited.

This is useful for a single process and is the approach in the runnable Python example. It does not coordinate across multiple application instances, and cancellation of the loader must release the lock so a waiter can retry.

Distributed single-flight

Across instances, a shared cache can coordinate a short refresh lease. A common Redis-style acquisition is:

SET lock:product:42 <unique-owner-token> NX PX 5000

Only the caller that successfully creates the key loads the origin. Other callers wait briefly, re-check the cache, or serve an explicitly permitted stale value. Release the lock only if the stored token still matches the caller's token; a plain DEL can delete a lock that expired and was acquired by another process.

A lease is not a guarantee that the holder will stop when the lease expires. A process can pause, lose network connectivity, or continue after another caller has acquired a replacement lease. If stale writers could corrupt correctness, use a monotonically increasing fencing token or source version that the protected resource validates. For cache refresh, use atomic version checks so an old loader cannot overwrite a newer value.

Do not introduce a distributed lock for every cache miss by default. It adds another network round trip and failure mode. Use it for genuinely hot keys where duplicate origin reads are expensive, and test lock expiry, owner loss, retries, and cache-node failover. For the broader correctness model of lease ownership, fencing generations, and leader election beyond cache refresh, see Distributed leases and fencing tokens.

Stale-while-revalidate

For data where bounded staleness is acceptable, keep a fresh-until time and a separate stale-until time. During the stale window, return the old value while one controlled refresher loads a replacement. After the stale window ends, callers must wait for a successful refresh or use the normal failure policy.

This can keep a popular key available while it refreshes, but it is inappropriate for security decisions, account status, or other data where stale output can cause harm. The HTTP stale-while-revalidate directive is defined for HTTP caches; an application cache can implement a similar policy, but it must define its own freshness and observability rules.

Other controls

  • TTL jitter: spreads expiry across many keys; it is not per-key single-flight.
  • Probabilistic early refresh: refreshes some hot keys before expiry so not every request reaches the hard-expiry boundary. Tune it from observed load and refresh cost.
  • Refresh-ahead: proactively refresh a known hot set, while limiting the number of concurrent refreshes.
  • Origin concurrency limits: cap simultaneous cache-miss loads even if the cache is unavailable.
  • Backoff and load shedding: avoid retrying the origin on every request during an outage. Fail fast or serve bounded stale data only when the product contract permits it.

7. Plan for cache failure and overload

A cache is an additional dependency. Decide what the service does when it is slow, unreachable, full, or returning errors.

Failure Risk Safer response
Cache timeout Every request waits on a slow dependency Use a short cache timeout; then apply a bounded origin fallback policy
Cache outage All traffic can fall through to the database Limit concurrent fallback reads, use backpressure, and shed non-essential work before exhausting the origin
Origin outage on miss No fresh value is available Return an error unless a bounded stale value is explicitly safe for this data
Eviction pressure Hit rate falls and origin load rises Inspect memory, key cardinality, object size, and eviction policy; do not just increase TTL
Hot-key expiry Many callers query the same origin row Use single-flight, stale-while-revalidate, or early refresh
Invalidation delivery failure Different instances disagree Keep a TTL safety bound, monitor lag, retry idempotently, and reconcile where needed
Cache poisoning or key collision Wrong or unsafe representation is returned Canonicalize inputs, scope keys correctly, validate origin data, and keep authorization outside the cache

Fail-open versus fail-closed is a product decision. For a public catalogue description, a bounded origin fallback may be acceptable. For access control, payment authorization, or inventory reservation, a stale cache should not grant permission or finalize a transaction. A fallback that overwhelms the database is not resilient; it simply moves the outage downstream.

8. Measure whether caching helps

Track the cache and the origin together. Useful signals include:

  • Hit and miss rate by cache namespace or operation, not only one global ratio.
  • Origin requests and database connection-pool saturation during misses.
  • Cache lookup latency and origin load latency at p50, p95, and p99.
  • Evictions, memory usage, key count, item size, and rejected writes.
  • Refresh duration, refresh failures, stale responses served, and invalidation lag.
  • Single-flight waiters, lock contention, lock expiry, and duplicate origin loads.
  • Cache availability and the number of requests using fallback behavior.

A high hit ratio is not automatically success. It can hide a stale-data bug, an overly broad key, or a cache that is serving data to the wrong tenant. A low hit ratio may be acceptable for a workload with cheap origin reads and low repetition. Measure end-to-end latency, origin protection, correctness, and cost together.

9. Runnable Python example

The companion runnable Python cache example implements a deliberately small, process-local cache-aside helper. It uses:

  • time.monotonic() for elapsed-time expiry;
  • a bounded LRU entry map;
  • per-key asyncio.Lock instances to coalesce concurrent misses in one process;
  • a generation check so an invalidation that occurs during an awaited load prevents that older load from repopulating the cache;
  • cleanup of per-key lock bookkeeping after callers finish.

Run it with:

python -m unittest discover -s interview-handbook/examples/python/cache_patterns -p 'test_*.py' -v

The tests cover concurrent misses, expiry, invalidation during a load, loader failure, key/TTL validation, and LRU eviction.

A service might use the helper like this:

async def get_product(tenant_id: str, product_id: str) -> Product:
    key = f"catalog:v2:{tenant_id}:{product_id}"
    return await product_cache.get_or_load(
        key,
        lambda: repository.get_product(tenant_id, product_id),
    )

async def update_product(command: UpdateProduct) -> Product:
    product = await repository.update_and_commit(command)
    product_cache.invalidate(
        f"catalog:v2:{command.tenant_id}:{command.product_id}"
    )
    return product

This example is not a Redis client and is not safe as a cross-process invalidation mechanism. The generation check protects only the in-memory helper in one event loop. A distributed implementation needs atomic version checks or another explicit coordination design, plus bounded memory and operational limits. Keep the source of truth authoritative.

10. Interview questions

Q1. What is cache-aside, and why is it common?

Answer: The application checks the cache, reads the source on a miss, stores the result with an expiry, and returns it. On writes it updates the source and invalidates the cache. It is demand-driven and easy to introduce without making the cache the system of record. Its costs are miss latency, application-owned invalidation, and an eventual-consistency window.

Q2. Does a TTL guarantee consistency?

Answer: No. A TTL limits how long an entry remains eligible to be served under the configured policy. It does not synchronize writes, prevent an old in-flight read from repopulating a key, or guarantee read-after-write behavior. Use invalidation or version checks for correctness and TTL as a fallback bound.

Q3. Why invalidate after the database commit?

Answer: Invalidating before commit allows a concurrent reader to miss the cache, read the old committed row, and repopulate it before the write commits. Invalidation after commit avoids that ordering, but an older read already in flight can still refill stale data afterwards. Versioned or atomic conditional fills address that race more directly.

Q4. What is a cache stampede?

Answer: Many requests miss the same hot key at once and all query the origin. It can exhaust database connections and increase tail latency. Use per-process single-flight, distributed coordination for hot keys, stale-while-revalidate where safe, early refresh, TTL jitter, and a global origin concurrency limit.

Q5. Does TTL jitter solve a hot-key stampede?

Answer: Not by itself. Jitter spreads expiry times across different keys and helps avoid a cache avalanche. Requests for one popular key still share the same expiry. That key needs request coalescing, controlled refresh, or an explicitly allowed stale window.

Q6. Why is a distributed lock not automatically safe?

Answer: A lease can expire while the original holder is paused or partitioned. A second holder can then start work while the first resumes. Use unique owner tokens for safe release, short bounded work, and fencing tokens or source-version checks when a stale holder must not overwrite newer state.

Q7. When would you use write-through instead of cache-aside?

Answer: When reads need a value to be updated in the cache as part of the write workflow and the added write latency is acceptable. If the cache and database are separate systems, the two writes still need a failure/reconciliation policy; write-through alone does not create a distributed transaction.

Q8. What is the difference between cache stampede, avalanche, and penetration?

Answer: A stampede is many concurrent misses for the same hot key. An avalanche is many keys expiring or failing around the same time. Penetration is repeated requests for keys that do not exist or are never cacheable. Use single-flight for hot keys, TTL jitter for synchronized expirations, and short negative caching plus request validation/rate limits for repeated nonexistent lookups.

Q9. What should be in a cache key for a multi-tenant API?

Answer: At minimum, the resource identity and trusted tenant scope, plus every representation dimension such as locale, filters, or schema version. If the response is permission-filtered, include the relevant authorization context or avoid caching that representation. Authorization must still be checked independently; key design is not access control.

Q10. What happens when Redis is unavailable?

Answer: The application needs a bounded fallback. Blindly sending every request to the database can turn a cache outage into a database outage. Use short cache timeouts, concurrency limits, backpressure, circuit breaking, and load shedding. Serve stale data only when the domain explicitly allows it; otherwise return an error or use the authoritative path.

11. Design checklist

Before shipping a cache, be able to answer:

  • What is the authoritative source, and which values are safe to cache?
  • What is the maximum acceptable staleness for each data class?
  • Does the key include tenant and representation boundaries?
  • What invalidates the entry after a write, and what happens if invalidation is delayed or lost?
  • Can an in-flight old read refill the key after invalidation?
  • What prevents many callers from loading the same missing hot key?
  • What happens if the cache is slow or unavailable?
  • Are memory, key cardinality, origin concurrency, and retry budgets bounded?
  • Which metrics prove that the cache improves the whole system without hiding correctness failures?

References