L2 on CS Theorems

Mutual Exclusion Without Hardware Support

Fri, 24 Dec 2027 12:00:00 +0100

Symptom

Two threads increment a shared counter a million times each. The final value is 1,374,522.

You know why: the increment is a read, an add, and a write, and the interleavings lose updates. The fix is a lock. But a lock is provided by your runtime, which gets it from the OS, which gets it from a hardware instruction — lock xchg, cmpxchg, ldrex/strex.

Work Stealing and Fair Scheduling

Fri, 17 Dec 2027 12:00:00 +0100

Symptom

You parallelize a recursive algorithm. Quicksort, or a tree traversal, or a divide-and-conquer numeric kernel. You have 16 cores and a central task queue, and you measure a speedup of 3.

Resolution and CDCL

Fri, 10 Dec 2027 12:00:00 +0100

Symptom

SAT is NP-complete (T027). It is the NP-complete problem, the one everything else reduces to. The textbook conclusion is that you should avoid it.

Meanwhile: a hardware verification team routinely discharges instances with two million variables and ten million clauses in under a minute. Package managers solve dependency resolution with SAT. Every bounded model checker (T049) is a SAT solver with a front end. The planning community abandoned custom search for SAT encodings and got faster.

Register Allocation Is Graph Coloring

Fri, 03 Dec 2027 12:00:00 +0100

Symptom

You write a function with forty local variables. The x86-64 machine it runs on has sixteen general-purpose registers, two of which are effectively spoken for.

Something must decide which values live in registers and which live in memory. Get it right and the function runs at register speed. Get it wrong and every operation costs a load and a store, and on a modern machine an L1 hit is around 4 cycles against roughly 1 for a register, with an L2 miss costing 12 to 20 and a trip to DRAM 200 or more.

Hoare Logic and Weakest Preconditions

Fri, 19 Nov 2027 12:00:00 +0100

Symptom

You changed a loop. It looks right. The tests pass. You are not confident.

The reason you are not confident is that the tests check a finite set of inputs and the loop runs on all of them. You know this. So you stare at the loop and try to convince yourself by simulating it in your head, which works for three iterations and then stops working.

Codd's Theorem

Fri, 05 Nov 2027 12:00:00 +0100

Symptom

You write:

SELECT name FROM users WHERE age > 30 AND city = 'Berlin';

You did not say whether to scan the table or use an index. You did not say which of the two predicates to apply first. You did not say anything about join order, because there is no join, but if there were you would not have said that either.

Parametricity: Theorems for Free

Fri, 15 Oct 2027 12:00:00 +0100

Symptom

Someone hands you a function and no source:

f :: forall a. [a] -> [a]

How much can you say about it? The instinct is nothing: any implementation is possible, that is what polymorphism means.

Progress and Preservation

Fri, 01 Oct 2027 12:00:00 +0100

Symptom

“Well-typed programs don’t go wrong.” You have heard it. You have probably said it. And then:

String s = null;
s.length(); // NullPointerException

That program is well typed and it went wrong. So either the slogan is false or it means something narrower than it sounds. It means something narrower, and the narrowness is precisely specifiable.

Hindley–Milner and Principal Types

Fri, 24 Sep 2027 12:00:00 +0100

Symptom

You write this in OCaml or Haskell and annotate nothing:

let rec map f = function
 | [] -> []
 | x :: xs -> f x :: map f xs

The compiler reports ('a -> 'b) -> 'a list -> 'b list. It found the most general possible type, on its own, with no hints, and it will reject any call that does not fit.

The Curry–Howard Correspondence

Fri, 17 Sep 2027 12:00:00 +0100

Symptom

You write a function signature and it looks like a sentence:

f :: (a -> b) -> (b -> c) -> (a -> c)

Read it aloud: if $a$ implies $b$, and $b$ implies $c$, then $a$ implies $c$. That is transitivity of implication, a rule of logic you have known since school. And the implementation, f g h = h . g, is the proof of it: given evidence for $a$, apply the first, then apply the second.

The Lambda Calculus and the Y Combinator

Fri, 10 Sep 2027 12:00:00 +0100

Symptom

You are writing a language, or a configuration evaluator, or a template engine, and you need recursion. So you add a letrec form, and now the evaluator needs to bind a name before the value it names exists. You reach for a mutable cell, a placeholder that gets patched after construction, and something about it feels like cheating.

The CAP Theorem and PACELC

Fri, 27 Aug 2027 12:00:00 +0100

Symptom

Someone in a design review says “we’re AP, not CP,” and the room nods.

Nobody asks which consistency, which availability, or what happens when there is no partition, which is 99.9% of the time. The phrase has done the work of a decision without containing one.

Paxos, Raft, and the Shape of Consensus

Fri, 20 Aug 2027 12:00:00 +0100

Symptom

Your database has two nodes that both believe they are the primary.

Each has been accepting writes. Each has a coherent log. Neither has crashed. The old primary was declared dead by a health check that timed out during a GC pause, a new one was elected, and then the old one woke up still holding what it thinks is a valid lease. You now have two divergent histories of the same data and no principled way to merge them, because both contain acknowledged writes that customers were told had succeeded.

The Myhill–Nerode Theorem

Fri, 06 Aug 2027 12:00:00 +0100

Symptom

You need to prove a language is not regular. Everyone points you at the pumping lemma, and you spend an afternoon losing to it.

The statement is a nest of quantifiers: for every regular language there exists a pumping length $p$ such that for every string $w$ with $|w| \ge p$ there exists a decomposition $w = xyz$ with $|xy| \le p$ and $|y| > 0$ such that for all $i \ge 0$, $xy^i z$ is in the language. To use it you negate all of that and play a game against an adversary who picks $p$ and the decomposition while you pick $w$ and $i$.

Kleene's Theorem: Regexes Are Finite Automata

Fri, 30 Jul 2027 12:00:00 +0100

Symptom

Your service goes down. CPU pinned at 100% on one core, no memory growth, no error logs, requests timing out.

The cause is a validation regex — something reasonable-looking like ^(a+)+$, or in real life a URL or email validator with nested quantifiers — matched against a 30-character string that happens not to match. The engine is exploring an exponential number of ways to split the input among the nested groups, and it will get there eventually, some time after the heat death of everything.

The Exponential Time Hypothesis and Fine-Grained Complexity

Fri, 16 Jul 2027 12:00:00 +0100

Symptom

You have a string algorithm. Edit distance between two sequences, the classic dynamic program, $O(n^2)$ time. It has been in production for years.

Now the inputs are genome-scale. At $n = 10^5$ characters, $n^2 = 10^{10}$ operations, about 10 seconds at a billion ops per second. At $n = 10^6$ it is $10^{12}$ operations, about 1000 seconds — seventeen minutes for one pair of strings. You need to do a million pairs.

The No-Cloning Theorem

Fri, 02 Jul 2027 12:00:00 +0100

Symptom

You have spent this whole part learning that information wants to be copied. Entropy bounds compression (T018), Huffman hits the bound (T020), redundancy buys error correction (T021). Every result so far treats “make another copy” as the free operation — the thing you do without thinking, the reason cp has no interesting failure modes.

The Nyquist–Shannon Sampling Theorem

Fri, 25 Jun 2027 12:00:00 +0100

Symptom

Your dashboard shows CPU utilization averaging 40%, comfortably under budget. Users are reporting timeouts. You add more logging and the average stays at 40%.

Eventually someone captures a one-second trace and finds the truth: the service spikes to 100% for eight seconds out of every ninety, and your monitoring samples every sixty seconds. The spikes are real, periodic, and invisible.

The Byzantine Generals Problem

Fri, 18 Jun 2027 12:00:00 +0100

Symptom

A node in your cluster is not down. It is worse than down.

Its disk is returning corrupted blocks that pass the checksum because the checksum is corrupted too. Or a NIC with a firmware bug is duplicating and reordering frames. Or a bad memory module flipped a bit in a config value and the node now believes it owns a shard it does not. Or someone has root on one machine.

Vector Clocks and Causal Consistency

Fri, 04 Jun 2027 12:00:00 +0100

Symptom

You took the lesson from T058 and stopped trusting wall clocks. Every event now carries a Lamport timestamp, and your writes are ordered by it. The vanishing updates stopped.

Parameterized Complexity (FPT and W[1])

Fri, 21 May 2027 12:00:00 +0100

Symptom

Your problem is NP-hard, and yet it keeps being easy.

You are computing a minimum vertex cover to select monitoring points in a network. Twelve thousand nodes. NP-hard, per T028, so you brace for the worst — and the answer comes back in under a second, every time. The cover is small, around thirty nodes, because the network is sparse and the interesting nodes are few.

Approximation Algorithms and Ratios

Fri, 14 May 2027 12:00:00 +0100

Symptom

You have accepted that the problem is NP-hard. Now what?

The literature offers a wall of results with numbers attached: 2-approximation, $\ln n$-approximation, PTAS, FPTAS, 0.878. Nobody explains what these numbers buy you, how they are proved, or how they connect to the heuristic you already wrote. Meanwhile your greedy heuristic is running in production and you have no idea whether it is within 5% or a factor of 50 of optimal, because you cannot compute the optimum to compare against — that was the whole problem.

Streaming Lower Bounds and Sketching

Fri, 30 Apr 2027 12:00:00 +0100

Symptom

Product wants the daily unique-visitor count. You have a firehose of events.

The obvious implementation is a set. Add each visitor ID, report the size. At a billion distinct IDs, eight bytes each, that is 8 GB before any hash table overhead, and in practice a HashSet will cost you two to three times that. Per day. Per dimension you want to slice by. Multiply by country, by platform, by campaign, and the memory bill is absurd for a number nobody looks at past two significant figures.

Universal Hashing

Fri, 23 Apr 2027 12:00:00 +0100

Symptom

Your service went down under a hash collision attack.

Somebody noticed your web framework put POST parameters into a hash table, found thousands of distinct keys colliding under its hash function, and posted a form with 20,000 of them. Every insert walked a chain. Quadratic behaviour, one CPU pinned per request, service dead. This actually happened, across PHP, Python, Ruby, Java and .NET in 2011, and again against Rust’s default HashMap before it switched to SipHash.

Minimum Description Length and Occam's Razor

Fri, 16 Apr 2027 12:00:00 +0100

Symptom

You fit a model. It scores 94% on training data and 71% on held-out data.

So you simplify: fewer parameters, more regularization. Training drops to 88%, held-out rises to 84%. You simplify further and both drop. Somewhere in there was an optimum, and you found it by trial and error, with a validation set and patience.

Kolmogorov Complexity

Fri, 09 Apr 2027 12:00:00 +0100

Symptom

You have two files, each exactly one megabyte.

The first is a megabyte of the byte 0x00. The second is a megabyte from /dev/urandom. Gzip the first and you get a few hundred bytes. Gzip the second and you get slightly more than a megabyte, because the header costs something and there is nothing to exploit.

Karp's 21 Problems

Fri, 26 Mar 2027 12:00:00 +0100

Symptom

Three weeks into a project, you are still trying to write an exact algorithm.

The problem is yours and it looks specific: assign delivery vans to routes such that every stop is covered, no van exceeds its capacity, and the total distance is minimized. Nothing in the literature matches it exactly. So you keep going — better data structures, smarter pruning, a cleverer greedy pass with a repair step — and each version works on your test set and falls over on production data.

The Cook–Levin Theorem

Fri, 19 Mar 2027 12:00:00 +0100

Symptom

You know reductions (T017): to prove your problem hard, reduce a known-hard problem to it. Fine. But that begs the obvious question, and it is the question a sharp colleague asks the first time you use the technique.

The Hamming Bound and Error-Correcting Codes

Fri, 05 Mar 2027 12:00:00 +0100

Symptom

You are specifying ECC memory for a fleet. The vendor quotes SECDED: single error correct, double error detect, at 8 check bits per 64 data bits. That is 12.5% overhead, and someone in the room asks the obvious question — why not correct two errors? Or three? What does it cost?

Huffman Coding Is Optimal

Fri, 19 Feb 2027 12:00:00 +0100

Symptom

You are compressing a log file. The symbols are wildly skewed: INFO is 90% of the lines, WARN is 9%, ERROR and FATAL split the rest. Fixed-width two-bit codes give you exactly 2 bits per symbol, and entropy (T018) says the floor is about 0.53 bits. There is a factor of four sitting there.

Balls Into Bins and the Power of Two Choices

Fri, 12 Feb 2027 12:00:00 +0100

Symptom

You have 100 backends and a load balancer hashing request IDs to pick one. Expected load per backend is exactly 1%, and you have checked the hash is good.

Reductions

Fri, 05 Feb 2027 12:00:00 +0100

Symptom

You have a new problem. Your build system needs to decide whether two configuration files can ever produce conflicting outputs. Or your linter needs to decide whether a regex with backreferences can match a string of a given length. Or your scheduler needs to decide whether a set of periodic tasks is feasible.

Godel's Incompleteness Theorems

Fri, 22 Jan 2027 12:00:00 +0100

Symptom

The proof assistant is failing you. You are trying to verify a piece of concurrent code in Coq or Lean, and the tool will not accept your termination argument. A colleague says, without much conviction, “well, Gödel — you can’t prove everything anyway.”

Kleene's Recursion Theorem

Fri, 15 Jan 2027 12:00:00 +0100

Symptom

Write a program that prints its own source code. No file I/O, no reading __file__, no cheating.

The first attempt fails instantly. To print the source you must contain the source, and then the containing text is also part of the source, so you must contain that too. The regress is obviously infinite, and most people conclude after ten minutes that the task is impossible.

Concentration Bounds (Chernoff/Hoeffding)

Fri, 18 Dec 2026 12:00:00 +0100

Symptom

Your service handles four billion requests a month and you want the error rate. Computing it exactly means a job over four billion log lines. Someone suggests sampling ten thousand of them.

Amortized Analysis and the Potential Method

Fri, 11 Dec 2026 12:00:00 +0100

Symptom

You have a dynamic array. push writes one element and bumps a counter, which is clearly $O(1)$ — except when the array is full, in which case it allocates a new buffer of twice the size, copies every element across, and frees the old one. That is $O(n)$.

Adversary Arguments

Fri, 04 Dec 2026 12:00:00 +0100

Symptom

You need the largest and second-largest element of an array. The obvious way is two passes: $n-1$ comparisons for the max, then $n-2$ for the max of the rest. That is $2n - 3$.

The Omega(n log n) Comparison-Sort Bound

Fri, 20 Nov 2026 12:00:00 +0100

Symptom

Every general-purpose sort in every standard library is $O(n \log n)$. Timsort, introsort, pdqsort, merge sort, heapsort. Decades of work by extremely motivated people, an enormous amount of money riding on it, and they all land on the same exponent.

Shannon Entropy and the Source Coding Theorem

Fri, 13 Nov 2026 12:00:00 +0100

Symptom

You gzip a log file and it drops to 8% of its original size. You gzip the already-gzipped file and it gets slightly bigger. Somebody asks why, and the honest answer you have is “because it’s already compressed,” which is a restatement, not a reason.

Rice's Theorem

Fri, 16 Oct 2026 12:00:00 +0200

Symptom

The security team asks for a scanner with no false positives and no false negatives. Every piece of malware caught, nothing legitimate quarantined.

The platform team asks whether the analyzer can flag every function that performs I/O, so the pure ones can be cached automatically.

Gustafson's Law and the Universal Scalability Law

Fri, 25 Sep 2026 12:00:00 +0200

Symptom

Two measurements that both look like they break the previous post.

The first: your Spark job takes four hours on ten nodes. You move it to a hundred nodes and it still takes four hours — but it is now processing ten times the data. T081 said the ceiling was 20×, and nobody hit a ceiling.