System Design Interview

Vertical scaling (scale up) adds more power (CPU/RAM) to a single machine. Horizontal scaling (scale out) adds more machines and distributes load across them. Horizontal scaling is more fault-tolerant and elastic.

A component that distributes incoming requests across multiple backend servers to improve throughput, reduce latency, and provide redundancy. It can operate at L4 (transport) or L7 (application) layer.

Round robin, weighted round robin, least connections, least response time, IP hash (sticky by client IP), and random. The choice affects fairness and session affinity.

Storing frequently accessed data in a faster/closer layer (memory, CDN) to reduce latency and load on the origin/database. Trades freshness and memory for speed.

The fraction of requests served from cache rather than the origin: hits / (hits + misses). A higher ratio means more load is absorbed by the cache and lower average latency.

LRU (Least Recently Used), LFU (Least Frequently Used), FIFO, and TTL-based expiry. LRU is the most common default, evicting the entry untouched the longest.