Thinking in Bets

Thinking Tools5.3 Validating Impact and ExecutionAlso known as Decisions as Bets

Developed by Annie Duke (Thinking in Bets, 2018).

Treats each decision as a bet on an uncertain future, with a stated probability, so the quality of the decision is judged apart from how things turned out.

Eight calls from a sales director's year, sorted by the bet on record and the result

Each call was written down as a bet when it was made: the reasoning and a chance of success. Select a call to read its record and what happened, then place it by the quality of the bet, judged from the record alone, and the result. The first is placed as an example. Show the sort to compare. Illustrative.

Items (select one, then a box)
  • Not placedPlacedEarned winBad luckDumb luckEarned loss
  • Not placedPlacedEarned winBad luckDumb luckEarned loss
  • Not placedPlacedEarned winBad luckDumb luckEarned loss
  • Not placedPlacedEarned winBad luckDumb luckEarned loss
  • Not placedPlacedEarned winBad luckDumb luckEarned loss
  • Not placedPlacedEarned winBad luckDumb luckEarned loss
  • Not placedPlacedEarned winBad luckDumb luckEarned loss
  • Not placedPlacedEarned winBad luckDumb luckEarned loss

The bet, judged from the record

Sound betPoor bet
Bad luck
Earned win
Earned loss
Dumb luck
Bad resultGood result

What happened

Hired the senior rep. Record: three references, a work sample, a 70 percent chance of success, and a backup candidate named. Result: she left after five months to care for a parent.Not placed yet.Hired the senior rep. Record: three references, a work sample, a 70 percent chance of success, and a backup candidate named. Result: she left after five months to care for a parent.Earned win: A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.Hired the senior rep. Record: three references, a work sample, a 70 percent chance of success, and a backup candidate named. Result: she left after five months to care for a parent.Bad luck: A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.Everything the record could check, it checked; the reason she left was outside what any hiring process could see.Hired the senior rep. Record: three references, a work sample, a 70 percent chance of success, and a backup candidate named. Result: she left after five months to care for a parent.Dumb luck: A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.Hired the senior rep. Record: three references, a work sample, a 70 percent chance of success, and a backup candidate named. Result: she left after five months to care for a parent.Earned loss: A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.Signed the hospital deal. Record: nothing written; the director said it felt right after one dinner. Result: it closed and renewed.Not placed yet.Signed the hospital deal. Record: nothing written; the director said it felt right after one dinner. Result: it closed and renewed.Earned win: A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.Signed the hospital deal. Record: nothing written; the director said it felt right after one dinner. Result: it closed and renewed.Bad luck: A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.Signed the hospital deal. Record: nothing written; the director said it felt right after one dinner. Result: it closed and renewed.Dumb luck: A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.No odds, no reasoning to check. The deal worked, which says nothing about the next one made the same way.Signed the hospital deal. Record: nothing written; the director said it felt right after one dinner. Result: it closed and renewed.Earned loss: A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.Cut trade-show spend. Record: two years of lead data and a 60 percent chance the pipeline holds without the shows. Result: the pipeline held.Not placed yet.Cut trade-show spend. Record: two years of lead data and a 60 percent chance the pipeline holds without the shows. Result: the pipeline held.Earned win: A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.The data supported the call and the call paid; the right lesson is the method, and 60 percent means it could still have gone the other way.Cut trade-show spend. Record: two years of lead data and a 60 percent chance the pipeline holds without the shows. Result: the pipeline held.Bad luck: A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.Cut trade-show spend. Record: two years of lead data and a 60 percent chance the pipeline holds without the shows. Result: the pipeline held.Dumb luck: A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.Cut trade-show spend. Record: two years of lead data and a 60 percent chance the pipeline holds without the shows. Result: the pipeline held.Earned loss: A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.Hired the CEO's referral. Record: work sample skipped, no chance written down. Result: missed quota three quarters running.Not placed yet.Hired the CEO's referral. Record: work sample skipped, no chance written down. Result: missed quota three quarters running.Earned win: A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.Hired the CEO's referral. Record: work sample skipped, no chance written down. Result: missed quota three quarters running.Bad luck: A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.Hired the CEO's referral. Record: work sample skipped, no chance written down. Result: missed quota three quarters running.Dumb luck: A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.Hired the CEO's referral. Record: work sample skipped, no chance written down. Result: missed quota three quarters running.Earned loss: A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.The steps that would have caught the gap were skipped. The record shows where the process broke.Raised the renewal 8%. Record: churn data by segment, a 75 percent chance the customer stays, a walk-away price set. Result: the customer left.Not placed yet.Raised the renewal 8%. Record: churn data by segment, a 75 percent chance the customer stays, a walk-away price set. Result: the customer left.Earned win: A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.Raised the renewal 8%. Record: churn data by segment, a 75 percent chance the customer stays, a walk-away price set. Result: the customer left.Bad luck: A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.A one-in-four chance of losing them was on the record. This is the one in four, and the price logic still holds for the other renewals.Raised the renewal 8%. Record: churn data by segment, a 75 percent chance the customer stays, a walk-away price set. Result: the customer left.Dumb luck: A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.Raised the renewal 8%. Record: churn data by segment, a 75 percent chance the customer stays, a walk-away price set. Result: the customer left.Earned loss: A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.Opened a second office. Record: one customer's verbal promise of volume, no estimate of the chance it arrives. Result: the volume came and the office broke even in a year.Not placed yet.Opened a second office. Record: one customer's verbal promise of volume, no estimate of the chance it arrives. Result: the volume came and the office broke even in a year.Earned win: A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.Opened a second office. Record: one customer's verbal promise of volume, no estimate of the chance it arrives. Result: the volume came and the office broke even in a year.Bad luck: A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.Opened a second office. Record: one customer's verbal promise of volume, no estimate of the chance it arrives. Result: the volume came and the office broke even in a year.Dumb luck: A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.The whole bet rested on a promise nobody tested. It paid this time; the next office should not be opened the same way.Opened a second office. Record: one customer's verbal promise of volume, no estimate of the chance it arrives. Result: the volume came and the office broke even in a year.Earned loss: A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.Dropped a weak reseller. Record: margin analysis, a 65 percent chance of keeping most accounts, a plan to call each one. Result: kept 80 percent of them.Not placed yet.Dropped a weak reseller. Record: margin analysis, a 65 percent chance of keeping most accounts, a plan to call each one. Result: kept 80 percent of them.Earned win: A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.A reasoned bet with a plan for the downside, and it paid. Copy the method.Dropped a weak reseller. Record: margin analysis, a 65 percent chance of keeping most accounts, a plan to call each one. Result: kept 80 percent of them.Bad luck: A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.Dropped a weak reseller. Record: margin analysis, a 65 percent chance of keeping most accounts, a plan to call each one. Result: kept 80 percent of them.Dumb luck: A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.Dropped a weak reseller. Record: margin analysis, a 65 percent chance of keeping most accounts, a plan to call each one. Result: kept 80 percent of them.Earned loss: A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.Bid the big tender. Record: the team flagged that the specification favored a rival; the director bid anyway, with no odds written. Result: lost, after six weeks of work.Not placed yet.Bid the big tender. Record: the team flagged that the specification favored a rival; the director bid anyway, with no odds written. Result: lost, after six weeks of work.Earned win: A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.Bid the big tender. Record: the team flagged that the specification favored a rival; the director bid anyway, with no odds written. Result: lost, after six weeks of work.Bad luck: A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.Bid the big tender. Record: the team flagged that the specification favored a rival; the director bid anyway, with no odds written. Result: lost, after six weeks of work.Dumb luck: A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.Bid the big tender. Record: the team flagged that the specification favored a rival; the director bid anyway, with no odds written. Result: lost, after six weeks of work.Earned loss: A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.The warning was on the record and ignored. The loss was predictable from the record alone.

Your sort matched 0 of 8.Your sort matched 1 of 8.Your sort matched 2 of 8.Your sort matched 3 of 8.Your sort matched 4 of 8.Your sort matched 5 of 8.Your sort matched 6 of 8.Your sort matched 7 of 8.Your sort matched 8 of 8.

Earned win
A sound bet that paid. Keep the process, and resist crediting the win with more skill than the odds gave it.
Bad luck
A sound bet that lost. Keep the process; the record shows the call was right at the odds, and the dice fell badly.
Dumb luck
A poor bet that paid. Fix the process anyway, or the win will teach the team to repeat the shortcut.
Earned loss
A poor bet that lost. Fix the process: the record shows what was skipped, and that gap is the lesson.

The sort

  • Hired the senior rep: Bad luck. Everything the record could check, it checked; the reason she left was outside what any hiring process could see.
  • Signed the hospital deal: Dumb luck. No odds, no reasoning to check. The deal worked, which says nothing about the next one made the same way.
  • Cut trade-show spend: Earned win. The data supported the call and the call paid; the right lesson is the method, and 60 percent means it could still have gone the other way.
  • Hired the CEO's referral: Earned loss. The steps that would have caught the gap were skipped. The record shows where the process broke.
  • Raised the renewal 8%: Bad luck. A one-in-four chance of losing them was on the record. This is the one in four, and the price logic still holds for the other renewals.
  • Opened a second office: Dumb luck. The whole bet rested on a promise nobody tested. It paid this time; the next office should not be opened the same way.
  • Dropped a weak reseller: Earned win. A reasoned bet with a plan for the downside, and it paid. Copy the method.
  • Bid the big tender: Earned loss. The warning was on the record and ignored. The loss was predictable from the record alone.

An example

An industrial distributor's regional sales director lost her largest customer at renewal after raising its price 8 percent. Her vice president's first reaction was to stop all price increases for the year. She asked for one meeting first, and brought her decision log.

Since January she had written every significant call as a bet before making it: what was wagered, the reasoning, and a chance of success. The renewal entry read 75 percent, with churn data by segment and a walk-away price. The same log held seven other calls. Laid out by the quality of the bet and the result, two sound bets had lost, two poor bets had won, and the hospital deal everyone praised had been made on a dinner and a feeling. The meeting kept the pricing rule, since one loss in four was what the record predicted. It changed how new offices get approved, since the office that broke even had rested on a promise nobody tested.

Why it works

Outcomes mix two things, the quality of the decision and luck, and people read them as if they were all decision. Duke calls this resulting. Its cost compounds: sound processes get abandoned after one bad draw, and lucky shortcuts get repeated until they fail at scale.

Stating a decision as a bet with a probability breaks the habit in two ways. Before the decision, putting a number on it forces the question of what would change it and what is not known. After the result, the written record gives something to judge other than the outcome: was the reasoning sound, was the information available used, and was the probability fair? A 70 percent bet that loses is not a mistake. Ten of them should lose about three times, and a log makes that visible.

The thinking behind it

Annie Duke Annie Duke, a former professional poker player who writes on decision making under uncertainty, set out this approach in Thinking in Bets (2018), a popular book. She argues for judging a decision by its quality, apart from how it turned out, and names the opposite error resulting: concluding that a decision was bad because its outcome was bad. She also argues for holding beliefs as bets, each with a degree of confidence.

How to run it

  1. Restate the decision as a bet. Write what is being wagered (money, time, a hire, a relationship) and on what future it pays off.
  2. Put a number on it. Write a probability of success, even a rough one, so the team can see how sure the decider is.
  3. Name what would move the number. List the evidence that would raise or lower the chance, and check whether any of it can be had before deciding.
  4. Record the reasoning. Keep a short entry with the date, the options weighed, the probability, and the information used.
  5. Judge the result against the record. When the outcome lands, ask whether the bet was sound on what was known then, separately from whether it paid.
  6. Sort the lessons by quadrant. Keep the process behind sound bets that lost; fix the process behind poor bets, especially the ones that won.

Where it goes wrong

The usual failure is writing the log after the fact. Entries drift toward whatever happened, and the record ends up confirming the outcome it was meant to be judged apart from. Date each entry before the result is known, and keep it where no one edits it later.

A second failure is using bad luck as a shield. Every loss becomes variance, and nothing gets fixed. The test is the record itself: if it shows skipped steps or ignored warnings, the bet was poor whatever the odds claimed. A third is false precision, arguing over 70 against 72 percent. The point of the number is a commitment that can be checked, so rough is fine; consistent is what matters.

When to use something else

  • Poker Decision Review. When the team needs a structured review of one decision after the fact, step by step, with the outcome held aside.
  • Brier Score. When the bets are numerous and resolved, and the question is how well calibrated each forecaster's probabilities are.
  • Probabilistic Thinking. When the decision is still ahead: spread the odds over several outcomes, weigh each choice's payoffs by them, and pick on expected value and the chance of a loss.

Where it sits in SOLVE eX

Go to the source

  • Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts, 2018

Back to the catalog