04-system-scaling-and-design.md

04 - System Scaling & High-Level Design

The Juspay final interview involves System Design discussions. Because Juspay is a payment gateway, the architectural themes center around reliability, consistency, and massive scale.

1. Idempotency (The Holy Grail of Payments)

What is it? A mathematical and computer science concept where applying an operation multiple times yields the exact same result as applying it once. f(f(x)) = f(x).

Why does Juspay care? A user is on Flipkart. They enter their UPI pin and click "Pay". The network drops for 3 seconds. The user panics and clicks "Pay" again. Two identical requests reach Juspay's servers. If not handled, the user loses ₹5,000 twice.

How to Implement Idempotency:

  1. Idempotency Key: The client (Flipkart frontend) generates a unique UUID (e.g., txn_8f3a91...) when the user reaches the checkout page.
  2. Both "Pay" clicks send the exact same payload containing this Idempotency-Key in the HTTP Header.
  3. The Server checks its Database / Redis:
    • Is this key present?
    • If NO: Insert key with status IN_PROGRESS. Process payment. Update to SUCCESS.
    • If YES and status is IN_PROGRESS: Drop the request, return 429 Too Many Requests or just hold the connection.
    • If YES and status is SUCCESS: Return the cached success response. Do NOT charge again.

2. Distributed Session Management

Interview Question: How does a server differentiate between two laptops on the same Wi-Fi?

  • Both laptops share the same Public IP address (due to Network Address Translation - NAT).
  • The NAT router assigns a unique Source Port to each laptop's outgoing TCP connection.
  • The server identifies them via the combination of (Source IP, Source Port, Dest IP, Dest Port, Protocol).
  • At the Application Layer (HTTP), the server differentiates them using unique Session Cookies or JWT Tokens sent in the headers.

Scaling Sessions across Multiple Nodes

If you have 10 servers behind a Load Balancer, how do you handle sessions?

  1. Sticky Sessions: Load Balancer routes user A to Server 1 always. Flaw: If Server 1 dies, the session is lost. Uneven load distribution.
  2. Centralized Session Store: Store sessions in an in-memory database like Redis or Memcached. All 10 servers query Redis. Fast and fault-tolerant.
  3. Stateless (JWT): Server doesn't store anything. The JWT contains the user ID and an encrypted signature. The server verifies the signature mathematically. Highly scalable, but hard to forcefully invalidate/logout a user before expiry.

3. Database Scaling Fundamentals

When one database server isn't enough, how do we scale?

ACID Properties

Crucial for financial transactions.

  • Atomicity: All or nothing. (Money deducted from A AND added to B, or neither).
  • Consistency: Database moves from one valid state to another.
  • Isolation: Concurrent transactions don't interfere. (See Isolation Levels).
  • Durability: Once committed, data is safe even if power is lost.

Transaction Isolation Levels

Higher isolation = lower concurrency (slower).

  1. Read Uncommitted: Transaction can read uncommitted data from other transactions (Dirty Read). Very fast, very dangerous for payments.
  2. Read Committed: Only reads committed data. Fixes dirty reads.
  3. Repeatable Read: If you read a row twice in a transaction, it guarantees the same data.
  4. Serializable: Strictly sequential execution. Safest, but kills performance.

Vertical vs. Horizontal Scaling

  • Vertical (Scale Up): Buy a bigger machine (More RAM/CPU). Easy, no code changes, but has a hardware ceiling and single point of failure.
  • Horizontal (Scale Out): Add more machines. Infinite scale, but requires complex distributed architecture.

Horizontal Scaling Techniques

  1. Read Replicas: One Master DB (handles Writes). Multiple Slave DBs (handle Reads). Replication is asynchronous (slight delay). Excellent for read-heavy systems.
  2. Sharding: Splitting data across multiple databases. (e.g., Users A-M on DB 1, Users N-Z on DB 2). Complex queries across shards are very slow.

4. Caching & Rate Limiting

Caching Strategies

  • Cache-Aside (Lazy Loading): Application checks Cache. If miss, query DB, put in Cache, return.
  • Write-Through: Application writes data to Cache and DB simultaneously. Slower writes, but cache is always perfectly up to date.

Rate Limiting

Protecting APIs from DDoS and brute force attacks.

  • Token Bucket: Bucket holds tokens. Each request costs a token. Bucket refills at a constant rate. Allows short bursts of traffic.
  • Leaky Bucket: Requests enter a queue (bucket) and are processed at a strictly constant rate. Smooths out traffic spikes completely.
  • Sliding Window: Uses timestamps (e.g., in Redis Sorted Sets) to accurately limit requests in a rolling time window.