Caching in System Design
Understand cache hits, misses, cache-aside, write strategies, TTL, invalidation, stampedes, penetration, eviction and Redis failure handling.
Concept Overview
Caching stores frequently accessed data in a faster storage layer so the application can avoid repeating expensive work or database reads. It can lower latency, increase throughput and reduce database load. A cache is normally a performance layer rather than the source of truth. The central design challenge is deciding what to cache, how long to keep it, and how to prevent stale or missing entries from creating correctness or availability problems.
1. Cache hit and cache miss
A cache hit occurs when the requested key exists and can be returned from the cache. A cache miss occurs when the key is absent or expired; the application must retrieve the data from the authoritative store and may then populate the cache. The benefit depends on hit rate, lookup overhead, serialization cost and the cost of the operation being avoided. Adding a cache to a rarely accessed or inexpensive query can make the system more complex without meaningful improvement.
Application -> Redis
|
HIT ----+----> Return cached value
MISS -> PostgreSQL -> Redis.set(key, value) -> Return2. Cache-aside is an application-managed strategy
In cache-aside, the application checks the cache first. If the value exists, it returns it. If not, it queries the database, stores the result in the cache and returns the result. The cache does not automatically know which database records changed. The application owns the read-through sequence and must also decide how writes affect cached entries.
data = redis.get(key)
if data exists:
return data
data = database.query()
redis.set(key, data, ttl)
return data3. Write-through and write-behind trade-offs
In a write-through approach, an application writes through the cache layer to the database so cached values are kept relatively fresh, at the cost of additional coupling and write latency. In write-behind, the cache accepts updates and a background process persists them later. That can make writes appear fast, but data may be lost if the cache fails before persistence. The supplied learning notes caution that write-behind is usually inappropriate for critical task data unless carefully designed.
4. TTL and cache invalidation
TTL means time to live: an entry expires after a configured duration. TTL limits how long an old value remains available, but does not guarantee that all data is fresh throughout that duration. Invalidation removes or refreshes an entry when the underlying record changes. For example, if a task's status changes from TODO to COMPLETED, a cached task or task-list entry may continue returning TODO unless it is updated or invalidated. Cache invalidation is therefore part of the data consistency design, not merely a Redis setting.
5. Choose cache candidates and keys deliberately
The supplied Task Management API notes identify user profiles, project metadata, project membership and frequently accessed task lists as possible cache candidates. Be careful with highly volatile data and security-sensitive authorisation decisions. Cache keys should be predictable and unambiguous, for example user:123, project:456 or project:456:tasks:page:1. Include parameters that change the result, such as page, filter or tenant, so unrelated query results cannot collide.
6. Cache stampede and cache penetration
A cache stampede occurs when a popular entry expires and many requests simultaneously miss the cache and hit the database. Possible mitigations include request coalescing, locking, jittered TTLs, background refresh and cache warming. Cache penetration occurs when clients repeatedly request data that does not exist, causing repeated misses and database lookups. Negative caching, input validation and rate limiting can reduce the impact.
7. Eviction and Redis failure
Redis has finite memory. When memory is constrained, configured eviction policies determine which keys may be removed; examples include least recently used and least frequently used policies. Redis should not automatically become the source of truth for task records. In the supplied architecture, PostgreSQL is the source of truth and Redis is a performance layer. If Redis fails, the application may fall back to PostgreSQL where the operation permits it, with higher latency and additional database load. That fallback itself should be tested and protected against overload.
Key Takeaways
- ✓A cache improves performance only when the access pattern and avoided work justify it.
- ✓Cache-aside makes the application responsible for reads and invalidation.
- ✓TTL helps bound staleness but does not replace a consistency strategy.
- ✓Design for stampedes, penetration, eviction and Redis failure.