Symptom

Your service handles 2,000 requests per second with a median latency of 50 milliseconds. Someone asks how many threads the pool needs. You guess 500, because it sounds safe.

Later the pool is exhausted under normal load, and separately, you notice the box is mostly idle. Both at once. You add threads; latency gets worse. You remove threads; throughput drops. There is clearly a relationship between arrival rate, latency, and the number of requests in flight, and you are tuning three knobs independently as if there were not.

There is a relationship. It is one line of algebra, it requires almost no assumptions, and it turns two of those three numbers into the third.

Statement

Little’s law. For any stable system over a long enough interval,

$$L = \lambda W$$

where $L$ is the average number of items in the system, $\lambda$ is the average arrival rate, and $W$ is the average time an item spends in the system.

The assumptions are the remarkable part, because there are almost none.

No distributional assumptions. Arrivals need not be Poisson, service times need not be exponential or independent or identically distributed.

No scheduling assumptions. FIFO, LIFO, priority, round-robin, random — all give the same $L = \lambda W$.

No structural assumptions. Any number of servers, any network of queues, any topology.

The only requirements are stability — nothing accumulates without bound, so arrival rate equals departure rate in the long run — and that the averages exist.

That generality is what makes it usable on a real system, where you know nothing about the distributions and cannot get the scheduler to behave.

Argument

Why it is true, geometrically. Draw a timeline. For each item, draw a horizontal segment from its arrival to its departure. Over a long window $T$:

The total area of all segments, measured in item-seconds, can be computed two ways. Summing over items gives (number of items) × (average time each), which is $\lambda T \cdot W$. Summing over time gives $\int_0^T L(t),dt = L \cdot T$. The two expressions describe the same area, so $L T = \lambda T W$, hence $L = \lambda W$.

That is the whole proof. It is a statement about the area of a region computed by rows versus by columns, which is why no probability enters and why no scheduling policy can affect it — rearranging when segments occur does not change their total length.

Applying it to the symptom. At $\lambda = 2000$ per second and $W = 50\text{ ms} = 0.05\text{ s}$:

$$L = 2000 \times 0.05 = 100$$

One hundred requests are in flight on average, not 500. If your pool has 500 threads, 400 are idle at any moment and the pool is not your constraint — but under a latency spike to 250 ms, $L$ becomes 500 and the pool saturates exactly then. The pool size is not a capacity decision, it is a decision about how much latency degradation you will absorb before shedding load. That reframing is the practical payoff.

Run it the other way. Concurrency is capped at 200 and average latency is 50 ms, so the maximum sustainable throughput is

$$\lambda = \frac{L}{W} = \frac{200}{0.05} = 4000 \text{ per second}$$

No amount of tuning exceeds that without reducing $W$ or raising $L$. Every capacity claim is one of these three numbers derived from the other two, and a plan that names all three independently is over-determined and probably wrong.

Choosing the boundary is where the leverage is. The law holds for any consistently-defined system boundary, and shifting the boundary answers different questions about the same service.

  • The thread pool. $L$ = busy threads, $W$ = time holding a thread.
  • The whole service including the queue. $L$ = requests accepted, $W$ = end-to-end latency including waiting.
  • The database connection pool. $L$ = active connections, $W$ = query time. At 500 queries per second and 20 ms per query, $L = 10$ connections. The common practice of configuring 100 connections against a database with 16 cores is refuted by this line, and oversized pools actively harm throughput by increasing contention inside the database.
  • The users. $L$ = concurrent users, $W$ = session length. 10,000 users with 30-minute sessions means $\lambda = 10000/1800 \approx 5.6$ new sessions per second.

A subtlety that catches people: which $W$. If some requests are rejected, the $\lambda$ in the law is the accepted rate, and rejected requests are not in the system. Load shedding therefore reduces $L$ by reducing $\lambda$, which is precisely the mechanism by which shedding protects latency. A system that queues everything instead of shedding keeps $\lambda$ high, so $L$ grows, so $W$ grows — the law shows why unbounded queues convert an overload into a latency collapse rather than a throughput ceiling.

Beyond averages, the law is silent, and that is important. $L = \lambda W$ constrains means only. Two systems with identical $L$, $\lambda$, and $W$ can have wildly different tail latencies. This is why Little’s law tells you how many threads you need on average and tells you nothing about p99, and why the next post is needed: the variance behaviour comes from utilization, not from this identity.

The distribution-free version is stronger than it looks. Because the proof is an area argument, it also applies to weighted versions. Setting the “items” to be item-seconds of memory gives the average memory occupancy from allocation rate times object lifetime, which is exactly how generational garbage collection sizing is reasoned about: allocation rate × object lifetime = live heap, and raising either raises the heap.

Using it as a measurement cross-check, which is its most underrated use. The three quantities are usually gathered by three different mechanisms: $\lambda$ from a request counter, $W$ from a latency histogram, $L$ from a pool gauge or a thread dump. They are collected independently, so the law is a consistency check on your telemetry. If measured $L$ is far from $\lambda W$, something is wrong with the instrumentation rather than with the system.

The discrepancies are diagnostic. Measured $L$ larger than $\lambda W$ usually means requests are in the system that your latency histogram never records — typically ones that timed out, were abandoned by the client, or are stuck and have not yet emitted a completion event, which is exactly the population you most want to find. Measured $L$ smaller than $\lambda W$ usually means $\lambda$ is being counted at a different boundary than $W$ is measured, such as counting arrivals at the load balancer while timing latency inside the application.

Forbids

Choosing pool sizes, queue depths, or connection counts independently of latency and throughput. Any two determine the third. A configuration naming all three inconsistently is describing a system that cannot exist.

Increasing throughput by adding concurrency without reducing latency. If $W$ is fixed by the work itself, raising $L$ raises $\lambda$ only until a real resource saturates, after which $W$ rises and the gain evaporates.

Unbounded queues protecting a service. With $\lambda$ above capacity, $L$ grows without bound and so does $W$. The queue converts overload into unbounded latency, and the system is not stable, so the law’s stability precondition fails in the only direction that matters.

Claiming capacity without measuring one of the three. You need two of $L$, $\lambda$, $W$ measured. Guessing all three is not planning.

Does not forbid

It does not require Poisson arrivals, and this is the most common reason people wrongly believe it does not apply to them. The law is distribution-free. It holds for bursty traffic, correlated arrivals, and adversarial patterns. M/M/1’s $1/(1-\rho)$ formula needs distributions; Little’s law does not, and conflating the two is what makes people dismiss it.

It does not require a single queue or a single server. It applies to networks of queues, to microservice meshes, and to each component individually. Applying it per-service across a call graph is standard practice, and the sub-systems compose because the law holds at every boundary.

It does not require steady state at every instant. It is a long-run average statement, and it tolerates arbitrary short-term fluctuation as long as the system does not diverge.

It does not say anything about tail latency. $L$, $\lambda$, and $W$ are means. A p99 SLO is not addressable with this law alone, and treating average capacity as though it governs tails is how systems get sized for a load they meet only on average.

It does not require items to be identical. Mixed workloads are fine; $W$ is the average over whatever mix arrives. This is why it applies to a service with both 1 ms cache hits and 500 ms cold reads without any adjustment.

Boundary

  • Averages only. No variance, no percentiles, no tail.
  • Stability required. In an overloaded system $L$ and $W$ both diverge and the long-run averages do not exist.
  • The boundary must be consistent. $L$, $\lambda$, and $W$ must all refer to the same system, and mixing “requests accepted” with “end-to-end latency including rejected requests” produces nonsense.
  • It is descriptive, not causal. The identity holds; it does not tell you which variable moves when you change the system. Adding threads changes $L$, but whether $\lambda$ or $W$ absorbs it depends on the bottleneck.
  • Long enough intervals. Over a window shorter than a few $W$, the boundary effects of items in flight at the start and end dominate.

The identity to carry: three numbers, two free. It is the cheapest true statement in capacity planning, and most sizing arguments that feel like intuition are really this law applied badly.