Load
Load is the volume of demand placed on a system at a given moment — the concurrent requests, active sessions, or units of work that callers are asking the system to handle. It is an input to the system, not a property of it. Load is what the world does to the system; capacity is what the system can do in response.
Load is often confused with its close neighbours. Throughput is the rate at which the system actually completes work — an output rather than an input. A system under heavy load may have low throughput if it is saturated or shedding work. Latency is the time each individual request spends in the system, and it is what load most directly degrades. As load approaches capacity, queues lengthen and latency rises, often sharply. Capacity is the ceiling on load the system can sustain while still meeting its other quality targets, and the relationship between load and latency is not linear. This is why capacity is always stated alongside a target latency or service level, rather than as a bare number.
Load is not a single number. It has a magnitude — how much demand — and a shape, determined by the access patterns of the workload: read-heavy or write-heavy, bursty or steady, fan-out or concentrated, uniform or skewed. Two services with identical request rates can place very different loads on their back-ends, depending on how expensive each request is and where the work falls. A workload that is uniform across partitions scales cheaply; one that is skewed concentrates load on a small number of hot nodes and saturates them long before the rest of the fleet.
Managing load is the central problem of scalability and performance engineering. Load balancing spreads load across multiple resources so no single one saturates. Auto-scaling adds capacity in response to rising load. Rate limiting and circuit breakers shed or reject load to protect a system that cannot keep up. The ceiling a system can sustain is established empirically through load testing, and the system’s behaviour beyond that ceiling is probed by stress testing.