07-caching-strategies.md

Caching Strategies in System Design

Interview Definition:

"Caching is like keeping your most-used items on your desk instead of packed away in a cabinet. It stores frequently requested data in fast memory (RAM) so the system can serve it instantly without having to do a slow database lookup."

1. Why Caching?

  1. Reduce Latency: Reading from memory (Redis/Memcached) takes microseconds, whereas reading from a disk-based database takes milliseconds.
  2. Reduce Database Load: By absorbing heavy read traffic, the primary database is protected from being overwhelmed, allowing it to focus on writes.

2. Caching Strategies (Reading & Writing)

A. Cache-Aside (Lazy Loading)

The most common caching strategy. The application is responsible for reading and writing to the cache.

Flow (Read):

  1. App asks Cache for data.
  2. If Cache Hit: Return data immediately.
  3. If Cache Miss: App fetches data from Database -> App writes data into Cache -> App returns data.

Pros: Cache only contains data that is actually requested (efficient memory use). Cons: The first request for any data results in a cache miss, causing a delay (latency spike). Data can become stale if updated in the DB but not invalidated in the cache.

B. Write-Through

The application uses the cache as the main data store. Every time data is written to the cache, it is synchronously written to the database.

Flow (Write):

  1. App writes data to Cache.
  2. Cache (or App) synchronously writes data to Database.
  3. Return success to user.

Pros: Data in the cache is never stale. Cons: Every write operation suffers the latency of writing to both the cache and the database.

C. Write-Behind (Write-Back)

Similar to Write-Through, but the write to the database is done asynchronously.

Flow (Write):

  1. App writes data to Cache.
  2. Return success to user immediately!
  3. Later, a background process flushes the data from the Cache to the Database.

Pros: Incredible write performance. Cons: If the cache server crashes before flushing to the database, data is permanently lost. Used in systems where speed is prioritized over absolute durability (e.g., YouTube view counters).

3. Cache Eviction Policies

Memory is expensive and limited. When the cache is full, the system must decide which items to evict to make room for new ones.

  • LRU (Least Recently Used): Discards the items that haven't been accessed for the longest time. (Most common).
  • LFU (Least Frequently Used): Discards items with the lowest access count over time.
  • FIFO (First In First Out): Discards the oldest inserted items regardless of how often they are accessed.
  • TTL (Time To Live): Items are automatically deleted after a specified time period (e.g., expire an OTP after 5 minutes).

4. Cache Invalidation

"There are only two hard things in Computer Science: cache invalidation and naming things." - Phil Karlton

Cache invalidation is the process of removing or updating stale data in the cache when the underlying data in the database changes. If this is done incorrectly, users will see outdated information.

5. Caching Technologies

  • Redis: An in-memory data structure store. It supports strings, hashes, lists, sets, and sorted sets. It also supports persistence (saving to disk) and advanced features like Pub/Sub and Geospatial indexes.
  • Memcached: A simpler, purely in-memory key-value store. Often used for caching small chunks of arbitrary data (like strings, objects) from database calls or API calls.