Originated by Thomas Bayes (An Essay towards Solving a Problem in the Doctrine of Chances, 1763) and, independently, Pierre-Simon Laplace (1774).
Revises how likely a belief is when new evidence arrives, by weighing the starting estimate against how much more likely the evidence is if the belief is true.
Three signals from a prospect, each moving the chance of a signed deal by how much likelier it is if they will sign
Set the starting chance from the base rate, then how often each signal shows up among prospects who sign and among those who do not. The starting odds are multiplied by each signal's ratio in turn, and the line shows the chance after each one. Make a signal equally common in both groups and watch it stop moving the line. Illustrative.
Chance the prospect signs, %
An example
The sales director of a business software company was reviewing the quarter's forecast, and a rep had marked a large deal as 90 percent likely after a good second meeting. The director pulled two years of closed deals from the CRM. One qualified prospect in five had signed. Nine in ten of the winners had taken a second meeting, but so had nearly half of the losers, so a second meeting only doubled the odds, from one to four to one to two: a 33 percent chance.
She did the same for the next signals. A pricing request showed up in 70 percent of wins and 25 percent of losses, which took the deal to 58 percent once it arrived. A legal review of the contract appeared in 60 percent of wins and only 10 percent of losses, and it carried the deal to 89 percent when the customer's lawyers called two weeks later. The team changed its forecast rule: a deal counts as committed only after legal review starts, and second meetings stopped moving the number much.
Why it works
People update on evidence in one of two bad ways. Some ignore how common the outcome was to begin with and jump to near-certainty on one striking sign. Others ignore the evidence and stay where they started. The rule fixes both by asking one question of every piece of evidence: how much more likely is this if the belief is true than if it is false?
That ratio is the evidence's weight. A sign that shows up almost as often in lost deals as in won ones has a ratio near one and should barely move the estimate, however encouraging it feels. In odds form the arithmetic is simple: multiply the starting odds by each ratio in turn. The starting point matters as much as the evidence, which is why a rare outcome needs strong evidence before it becomes likely.
The thinking behind it
Thomas Bayes Thomas Bayes, an English Presbyterian minister, wrote An Essay towards Solving a Problem in the Doctrine of Chances, published in the Philosophical Transactions of the Royal Society in 1763, two years after his death, by Richard Price. It solved a special case of what is now called Bayes' theorem: how to infer the probability of a cause from the events observed. Updating a forecast from the signals a deal shows is that inference.
Pierre-Simon Laplace Pierre-Simon Laplace, the French mathematician and astronomer, reached the idea independently. His 1774 memoir on the probability of causes given events, Mémoire sur la probabilité des causes par les événements, gave a general method for reasoning from observed effects back to their causes. The Bayesian interpretation of probability, as a degree of belief revised by evidence, was developed mainly by him.
How to run it
- State the belief and its alternative. Write both as plain outcomes, such as this prospect signs this quarter or does not.
- Set the starting chance from a base rate. Use how often the outcome happens in similar cases, before looking at anything about this one.
- Ask how likely the evidence is under each. Estimate how often this signal shows up when the belief is true and when it is false, from records where you can.
- Take the ratio. Divide the first by the second; a ratio near one means the evidence barely matters.
- Multiply the odds. Convert the starting chance to odds, multiply by the ratio, and convert back to a chance.
- Repeat for each independent piece of evidence. Use the updated chance as the new start, and skip evidence that only repeats an earlier signal.
- Decide separately. The updated chance is a belief; whether to act on it depends on what each outcome costs.
Where it goes wrong
The most common failure is skipping the base rate. A striking sign gets read as if the outcome were a coin toss to begin with, and a rare event looks likely. Spot it when nobody in the room can say how often the outcome happens in general; find that number before discussing the evidence.
The second is counting the same evidence twice. Two reports from the same source, or a pricing request that always follows a second meeting, are one signal, and multiplying both inflates the result. Ask whether each new item would have appeared anyway given the last. The third is picking the ratios after seeing the answer one wants. Write them down before looking at the case.
When to use something else
- Reference Class Forecasting. When the main question is the starting point: how often things like this happen at all.
- Decision Tree Analysis. When the beliefs are set and the question is which action to take given each outcome's payoff.
- Brier Score. When the aim is checking, over many forecasts, whether the probabilities a team gives turn out right.
Where it sits in SOLVE eX
- Any step: Tools that serve every step: habits of thought and checks on the thinking itself.
Go to the source
- An Essay towards Solving a Problem in the Doctrine of Chances, Philosophical Transactions of the Royal Society, 1763
- Mémoire sur la probabilité des causes par les événements, 1774