From Francis Galton's report on guessing an ox's weight (Vox Populi, 1907), developed by James Surowiecki (The Wisdom of Crowds, 2004) and Scott E. Page (The Difference, 2007).
Collects independent estimates from a varied group and takes the median, which often beats any single expert's guess because individual errors cancel out.
Two hundred private guesses at first-year sales, and how one loud guess pulls them off
Each run is one employee's guess at how many ovens will sell in the first year. Play collects 200 private guesses: three in five miss by 20 percent or more, yet the median lands near the 12,000 actually sold. Then let everyone hear the CEO's 20,000 first, or give the whole crowd a shared optimism, and Play again. Illustrative.
- An employee's private guess: around 12,000 ovens, give or take 4,000.
Guessed first-year sales, ovens
Runs so far: 200 of 200
- The crowd's median guess
- 11,727
- Guesses 20% or more too low
- 30%
- Guesses 20% or more too high
- 30%
An example
A kitchen appliance maker was about to launch a countertop steam oven and needed a first-year sales number for the production plan. In past launches the forecast had come from the head of sales, and it had missed by a third twice running. This time the product director sent a one-question survey to 200 employees across sales, service, finance, and the factory: how many ovens will we sell in the first twelve months? Answers were private, and nobody saw anyone else's.
The guesses ran from under 3,000 to over 20,000, and three in five missed the eventual result by more than 20 percent. The median was 11,700. The plan was set at 12,000, and the first year closed at 12,000. The director had tried the survey once before, after the CEO told a kickoff meeting that he expected 20,000. That time the guesses bunched tightly around 15,000, and the launch sold well short of both numbers.
Why it works
Every guess is the truth plus an error. When people guess on their own, from different knowledge, their errors point in different directions, and many of them cancel when the guesses are combined. The median keeps the cancellation and ignores the few wild answers, so a crowd of ordinary guessers can land closer than nearly all of its members.
The cancellation depends on independence. If everyone hears the same loud number first, or shares the same optimism, the errors lean the same way and little cancels; the crowd becomes one guess repeated two hundred times. That is why the method collects estimates privately, before any discussion, and why a wide spread is useful information, provided it comes from people reasoning separately.
The thinking behind it
Francis Galton Francis Galton, a polymath and statistician, reported in Vox Populi (Nature, 1907) on a weight-judging contest at a Plymouth fair, where some eight hundred visitors guessed the weight of an ox once it was slaughtered and dressed. The median of their guesses fell within one percent of the true weight. He chose the median, which a few wild guesses cannot drag far, and the method on this page uses it for the same reason.
James Surowiecki James Surowiecki, a former New Yorker business columnist, opened The Wisdom of Crowds (Doubleday, 2004) with Galton's ox and argued that groups often judge better than their best members under four conditions: diversity, independence, decentralization, and a way to aggregate. The book is a popular synthesis, and its conditions work as a checklist for any crowd estimate; independence is the one a loud first guess destroys.
Scott E. Page Scott E. Page, a University of Michigan professor of complex systems, proved the diversity prediction theorem: a crowd's squared error equals the average individual squared error minus the diversity of the predictions. His book The Difference (2007) makes the case for diverse groups. A crowd is more accurate the more its members' guesses differ, which is why an anchor that makes everyone guess alike hurts the crowd even as it makes the guesses look more confident.
How to run it
- Pose one precise question. Name the quantity, the unit, and the period, such as ovens sold in the first twelve months.
- Recruit a varied pool. Fifteen or more people who know different parts of the problem, from sales to the factory floor.
- Collect estimates privately. Do it before any meeting, presentation, or hint of what leaders expect.
- Take the median and report the spread. Give the middle guess and the range that holds the middle half of the guesses.
- Look into a wide spread. Ask a few people at each end what they know; the reason can matter more than the number.
- Score the crowd afterward. Compare the median with the outcome, and keep the record for the next estimate.
- Refresh the pool. Keep it varied by adding people from functions the last pool lacked.
Where it goes wrong
The first failure is collecting guesses after the anchor is set. A forecast meeting where the most senior person speaks first, followed by a survey, measures that person's view two hundred times. Spot it when the spread is narrow and the median sits near the loudest number, and recover by running the survey before any discussion.
The second is a crowd that shares one bias. Asking only the sales team about sales, or only the launch team about a launch, gives errors that all lean the same way. The third is trusting the median on a question nobody can know. Crowds do well on quantities people have some information about; where nobody has any, the median is only a comfortable number.
When to use something else
- Delphi Forecasts and Predictions. When the estimators are experts who should see each other's reasoning, in rounds, without meeting.
- Prediction Markets. When the question stays open for months and new information should keep moving the estimate.
- Fermi Estimation. When no crowd is available and one person must build the number from its parts.
Where it sits in SOLVE eX
Go to the source
- Vox Populi, Nature 75, 1907
- The Wisdom of Crowds, 2004
- The Difference, 2007