Misdiagnoses, wrongful arrests, failed security screenings—a surprising number of catastrophic real-life errors share the same mathematical cause: the base rate fallacy. Part statistical trap, part cognitive blind spot, it is one of the most consequential thinking errors a human being can make.
In his book Thinking Fast and Slow, Daniel Kahneman demonstrates how easily even the most conscientious humans can fall into this trap.
Bayes’ Rule
“A cab was involved in a hit-and-run accident at night. Two cab companies, the Green and the Blue, operate in the city. You are given the following data: 85 percent of the cabs in the city are Green and 15 percent are Blue. A witness identified the cab as Blue. The court tested the reliability of the witness under the circumstances that existed on the night of the accident and concluded that the witness correctly identified each one of the two colors 80 percent of the time and failed 20 percent of the time. What is the probability that the cab involved in the accident was Blue rather than Green?”
Kahneman went on to write that “The two sources of information can be combined by Bayes’s rule. The correct answer is 41 percent. However, you can probably guess what people do when faced with this problem: they ignore the base rate and go with the witness. The most common answer is 80 percent.”
This is the base rate fallacy in its purest form: ignoring or underweighting a known population statistic because a more exciting, eye-catching piece of information crowds it out. Here it’s seen in the fleet composition; 85 percent of the cabs are Green. The fallacy happens when people latch onto the witness’s 80 percent accuracy and essentially forget the 85/15 split entirely.
Statistical Bias
Here’s another example from Kahneman’s book that might be applied to founders thinking about their customer base:
“An individual has been described by a neighbor as follows: ‘Steve is very shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.’ Is Steve more likely to be a librarian or a farmer?”
Kahneman found that when asked this question in an experiment, most people said Steve was probably a librarian. It makes sense: we often view librarians as introverted. But the majority of us fail to realize that farmers outnumber librarians in the U.S. by more than 11 to one.
Jack Murtagh made this pretty clear in his recent article in Scientific American: “This statistical bias becomes more obvious when the career possibilities have a starker difference in population sizes: Steve loves astronomy. Is he more likely to be a banker or an astronaut?”
