Skin in the Game

Thinking Tools5.3 Validating Impact and Execution

Developed by Nassim Nicholas Taleb (Skin in the Game, 2018).

Weighs advice and forecasts by what the source loses if wrong, discounts claims made at no cost, and builds personal stakes into decisions.

Four forecasts of a launch's first-year sales, weighted by what each forecaster loses if wrong

Four people forecast first-year units for a new espresso machine; the bars are their ranges. Score what each one loses if wrong, from 1 (nothing) to 10 (their own money or job). The plain average treats every forecast alike; the weighted view moves toward the people who will pay for a miss. Set every stake equal to see the two agree. Illustrative.

1
2
6
9
Plain average of the four forecasts
31.5K units
Forecast weighted by what each loses if wrong
23.2K units
Columnist: 40K to 60KAnalyst: 25K to 45KHead of product: 18K to 30KFounder: 12K to 22K0K20K40K60KPlain averageWeighted by stakeColumnist: 40K to 60KAnalyst: 25K to 45KHead of product: 18K to 30KFounder: 12K to 22K0K20K40K60KPlain averageWeighted by stake

An example

A small appliance company was deciding how many units of its first home espresso machine to order for the opening production run. Four forecasts sat on the table. A columnist at a trade newsletter had written that the category would boom and put first-year sales at 40,000 to 60,000. An outside analyst hired for a market study said 25,000 to 45,000. The head of product, whose bonus depended on the launch, said 18,000 to 30,000. The founder, who had put her own money into the tooling, said 12,000 to 22,000.

The operations director asked one question of each: what do you lose if you are wrong? The columnist lost nothing; a missed call would be forgotten by the next issue. The analyst lost a little reputation. The product head lost a bonus and some standing. The founder lost money she could not get back. Weighting the forecasts by those stakes moved the planning number from about 31,500 units to about 23,000. The company ordered 24,000, with an option on a second run, and sold 21,000 in the first year.

Why it works

A forecast costs its maker nothing unless being wrong does. People with no exposure can be bold, fashionable, or simply careless, and the claim reads the same as one from someone who will pay for a miss. Asking what the source loses if wrong separates the two. Exposure does not guarantee accuracy, but it filters out the claims made for attention or for free, and it tends to make people check their own reasoning.

The same logic applies forward, to decisions. When the people who decide also bear the downside, they weigh it. When they do not, risk drifts toward whoever cannot see it. Building stakes into the decision, through co-investment, deferred bonuses, or a public commitment, makes the decider's interest and the organization's line up.

The thinking behind it

Nassim Nicholas Taleb Nassim Nicholas Taleb, a former options trader who writes on risk and probability, set out the idea in Skin in the Game: Hidden Asymmetries in Daily Life (2018). His argument is that decision makers should bear the consequences of their decisions, and that advice from people who carry no downside deserves heavy discounting. The book is an essay for a general reader, and some statisticians have faulted his wider treatment of statistics, but the core test, asking what the source loses if wrong, needs no statistics at all.

How to run it

  1. List the sources the decision rests on. Write each forecast, recommendation, or piece of advice alongside who gave it.
  2. Ask what each loses if wrong. Name the concrete cost to the source: money, a bonus, a job, reputation, or nothing.
  3. Discount the costless claims. Give little weight to bold forecasts from people who pay nothing for a miss, however confident they sound.
  4. Weight the rest by exposure. Lean toward the people who will bear the consequences, while still checking their reasoning for bias.
  5. Put stakes on the decision makers. Tie the deciders' pay or capital to the outcome through co-investment, deferred bonuses, or clawbacks.
  6. Back strong predictions with a commitment. Ask anyone pushing a confident view to commit in writing, or to a bet, before the decision rests on it.

Where it goes wrong

The common failure is treating exposure as proof of accuracy. A founder with her savings in the company can be badly wrong, often optimistically so. It shows up as one person's view winning every argument because they have the most at stake. Weight exposure, and still test the reasoning.

A second failure is building stakes that point the wrong way. A bonus for hitting a launch number gives the product head a reason to forecast low and beat it. Check what each stake rewards, and pair upside with downside. A third is lopsided exposure: managers who share in the gains but not the losses take more risk than the owners would. Look for gains and losses landing on different people.

When to use something else

  • Moral Hazard Analysis. When the issue is a party shielded from the cost of its own risky choices by a contract or guarantee, and the work is redesigning the terms.
  • Wisdom of Crowds Estimate. When many independent estimates are available and the aim is to pool them, with no reason to favor one source.
  • Brier Score. When forecasters have track records that can be scored, so accuracy can be measured directly.

Where it sits in SOLVE eX

Go to the source

  • Skin in the Game: Hidden Asymmetries in Daily Life, 2018

Back to the catalog