The Myhill–Nerode Theorem

Symptom You need to prove a language is not regular. Everyone points you at the pumping lemma, and you spend an afternoon losing to it. The statement is a nest of quantifiers: for every regular language there exists a pumping length $p$ such that for every string $w$ with $|w| \ge p$ there exists a decomposition $w = xyz$ with $|xy| \le p$ and $|y| > 0$ such that for all $i \ge 0$, $xy^i z$ is in the language. To use it you negate all of that and play a game against an adversary who picks $p$ and the decomposition while you pick $w$ and $i$. ...

Kleene's Theorem: Regexes Are Finite Automata

Symptom Your service goes down. CPU pinned at 100% on one core, no memory growth, no error logs, requests timing out. The cause is a validation regex — something reasonable-looking like ^(a+)+$, or in real life a URL or email validator with nested quantifiers — matched against a 30-character string that happens not to match. The engine is exploring an exponential number of ways to split the input among the nested groups, and it will get there eventually, some time after the heat death of everything. ...

The Exponential Time Hypothesis and Fine-Grained Complexity

Symptom You have a string algorithm. Edit distance between two sequences, the classic dynamic program, $O(n^2)$ time. It has been in production for years. Now the inputs are genome-scale. At $n = 10^5$ characters, $n^2 = 10^{10}$ operations, about 10 seconds at a billion ops per second. At $n = 10^6$ it is $10^{12}$ operations, about 1000 seconds — seventeen minutes for one pair of strings. You need to do a million pairs. ...

The No-Cloning Theorem

Symptom You have spent this whole part learning that information wants to be copied. Entropy bounds compression (T018), Huffman hits the bound (T020), redundancy buys error correction (T021). Every result so far treats “make another copy” as the free operation — the thing you do without thinking, the reason cp has no interesting failure modes. ...

The Nyquist–Shannon Sampling Theorem

Symptom Your dashboard shows CPU utilization averaging 40%, comfortably under budget. Users are reporting timeouts. You add more logging and the average stays at 40%. Eventually someone captures a one-second trace and finds the truth: the service spikes to 100% for eight seconds out of every ninety, and your monitoring samples every sixty seconds. The spikes are real, periodic, and invisible. ...

The Byzantine Generals Problem

Symptom A node in your cluster is not down. It is worse than down. Its disk is returning corrupted blocks that pass the checksum because the checksum is corrupted too. Or a NIC with a firmware bug is duplicating and reordering frames. Or a bad memory module flipped a bit in a config value and the node now believes it owns a shard it does not. Or someone has root on one machine. ...

Vector Clocks and Causal Consistency

Symptom You took the lesson from T058 and stopped trusting wall clocks. Every event now carries a Lamport timestamp, and your writes are ordered by it. The vanishing updates stopped. ...

Parameterized Complexity (FPT and W[1])

Symptom Your problem is NP-hard, and yet it keeps being easy. You are computing a minimum vertex cover to select monitoring points in a network. Twelve thousand nodes. NP-hard, per T028, so you brace for the worst — and the answer comes back in under a second, every time. The cover is small, around thirty nodes, because the network is sparse and the interesting nodes are few. ...

Approximation Algorithms and Ratios

Symptom You have accepted that the problem is NP-hard. Now what? The literature offers a wall of results with numbers attached: 2-approximation, $\ln n$-approximation, PTAS, FPTAS, 0.878. Nobody explains what these numbers buy you, how they are proved, or how they connect to the heuristic you already wrote. Meanwhile your greedy heuristic is running in production and you have no idea whether it is within 5% or a factor of 50 of optimal, because you cannot compute the optimum to compare against — that was the whole problem. ...

Streaming Lower Bounds and Sketching

Symptom Product wants the daily unique-visitor count. You have a firehose of events. The obvious implementation is a set. Add each visitor ID, report the size. At a billion distinct IDs, eight bytes each, that is 8 GB before any hash table overhead, and in practice a HashSet will cost you two to three times that. Per day. Per dimension you want to slice by. Multiply by country, by platform, by campaign, and the memory bill is absurd for a number nobody looks at past two significant figures. ...

Universal Hashing

Symptom Your service went down under a hash collision attack. Somebody noticed your web framework put POST parameters into a hash table, found thousands of distinct keys colliding under its hash function, and posted a form with 20,000 of them. Every insert walked a chain. Quadratic behaviour, one CPU pinned per request, service dead. This actually happened, across PHP, Python, Ruby, Java and .NET in 2011, and again against Rust’s default HashMap before it switched to SipHash. ...

Minimum Description Length and Occam's Razor

Symptom You fit a model. It scores 94% on training data and 71% on held-out data. So you simplify: fewer parameters, more regularization. Training drops to 88%, held-out rises to 84%. You simplify further and both drop. Somewhere in there was an optimum, and you found it by trial and error, with a validation set and patience. ...

Kolmogorov Complexity

Symptom You have two files, each exactly one megabyte. The first is a megabyte of the byte 0x00. The second is a megabyte from /dev/urandom. Gzip the first and you get a few hundred bytes. Gzip the second and you get slightly more than a megabyte, because the header costs something and there is nothing to exploit. ...

Karp's 21 Problems

Symptom Three weeks into a project, you are still trying to write an exact algorithm. The problem is yours and it looks specific: assign delivery vans to routes such that every stop is covered, no van exceeds its capacity, and the total distance is minimized. Nothing in the literature matches it exactly. So you keep going — better data structures, smarter pruning, a cleverer greedy pass with a repair step — and each version works on your test set and falls over on production data. ...

The Cook–Levin Theorem

Symptom You know reductions (T017): to prove your problem hard, reduce a known-hard problem to it. Fine. But that begs the obvious question, and it is the question a sharp colleague asks the first time you use the technique. ...

The Hamming Bound and Error-Correcting Codes

Symptom You are specifying ECC memory for a fleet. The vendor quotes SECDED: single error correct, double error detect, at 8 check bits per 64 data bits. That is 12.5% overhead, and someone in the room asks the obvious question — why not correct two errors? Or three? What does it cost? ...

Huffman Coding Is Optimal

Symptom You are compressing a log file. The symbols are wildly skewed: INFO is 90% of the lines, WARN is 9%, ERROR and FATAL split the rest. Fixed-width two-bit codes give you exactly 2 bits per symbol, and entropy (T018) says the floor is about 0.53 bits. There is a factor of four sitting there. ...

Balls Into Bins and the Power of Two Choices

Symptom You have 100 backends and a load balancer hashing request IDs to pick one. Expected load per backend is exactly 1%, and you have checked the hash is good. ...

Reductions

Symptom You have a new problem. Your build system needs to decide whether two configuration files can ever produce conflicting outputs. Or your linter needs to decide whether a regex with backreferences can match a string of a given length. Or your scheduler needs to decide whether a set of periodic tasks is feasible. ...

Godel's Incompleteness Theorems

Symptom The proof assistant is failing you. You are trying to verify a piece of concurrent code in Coq or Lean, and the tool will not accept your termination argument. A colleague says, without much conviction, “well, Gödel — you can’t prove everything anyway.” ...