Power-Law Reasoning

Thinking Tools3.4 Facts About the FutureAlso known as Fat-tail thinking

From the income distribution of Vilfredo Pareto (1896), carried into markets by Benoit Mandelbrot (1963), surveyed by M. E. J. Newman (2005), and brought to risk by Nassim Nicholas Taleb (2007).

Assumes a few extreme cases dominate the total until the data shows otherwise, so averages, ranges, and risk limits are not built on a bell curve that does not fit.

A thousand deal sizes from a fat-tailed distribution: the average is set by a handful of deals

Each run draws one deal from a power law that starts at $20,000. Play builds the histogram; watch the average jump each time a huge deal lands, while the median barely moves. Raise the tail index toward 3 and the tail thins: the average settles near the typical deal. Illustrative.

1.25
  • Deal size: deals start at $20,000, and the share of deals more than k times that size falls as k to the power of minus 1.25.
$100,000$200,000$300,000Average: $98,382$100,000$200,000$300,000Average: $98,382

Deal size, $. The end bars also hold every run beyond them.

Runs so far: 1,000 of 1,000

Median deal
$34,182
Average deal
$98,382
One deal in a hundred is above
$880,937
Deals above $1 million
1.0%

An example

A company selling supply chain data subscriptions planned next year's revenue as 220 new deals at its average deal size of about $98,000. A new finance director plotted the last thousand deals by size on log-log axes, and the upper half fell along a straight line, the mark of a power law. The median deal was $34,000. Ten deals above $1 million, one of them $15 million, had brought in 35 percent of five years' new revenue, and the smaller half of all deals only 13 percent.

The average was a fact about those ten deals more than about the typical customer, and in a year when none of them closed, the plan would miss by about a third. She rebuilt it in two parts: a base of 220 deals priced from the median and the recent run rate, and a named pipeline of eight large accounts treated as upside. Hiring and spending were tied to the base, and the large accounts got their own review each quarter.

Why it works

Many business quantities are not shaped like a bell. Deal sizes, customer revenue, project overruns, losses, and returns on bets often follow a power law: most cases are small, and a few are enormous. In that shape the average describes almost nobody, the sample average jumps every time a giant lands, and the range seen so far says little about the next extreme.

Assuming the tail is fat until the data shows otherwise protects every number built on top. Plans use the median for the base and treat the large cases as a separate, lumpy stream. Risk limits are set for the tail outcome, since a standard deviation computed from such data never settles. A log-log plot is the cheap test: a straight upper tail is the warning sign.

The thinking behind it

Vilfredo Pareto Vilfredo Pareto, an Italian economist of the Lausanne school, found in Cours d'économie politique (F. Rouge, Lausanne, 1896 to 1897) that the distribution of income across many countries followed a regular power law, with a small share of people holding a large share of income and land. Joseph Juran later generalized the observation into the Pareto principle. The regularity Pareto found in incomes is the pattern this tool looks for first in any data.

Benoit Mandelbrot Benoit Mandelbrot, the founder of fractal geometry, showed in The Variation of Certain Speculative Prices (Journal of Business, 1963) that changes in cotton prices follow heavy-tailed distributions, with far more extreme moves than the normal curve allows. The paper carried distributions of Pareto's kind from incomes into markets, and as the founder of fractal geometry he later wrote The Fractal Geometry of Nature (1982).

Nassim Nicholas Taleb Nassim Nicholas Taleb, a former options trader who writes on risk and probability, argued in The Black Swan: The Impact of the Highly Improbable (Random House, 2007) that rare, high-impact events dominate outcomes. His books are essays for a general reader with technical appendices, and some statisticians fault them for ignoring robust methods. This tool takes from them its practical rule: plans, reserves, and limits are sized for the tail.

How to run it

  1. Plot before averaging. Put the cases on log-log axes, size against how many are at least that large.
  2. Read the upper tail. A straight line there, or a few cases far beyond the rest, means a fat tail.
  3. Compare mean and median. When the mean is a multiple of the median, the mean belongs to the largest cases.
  4. Measure the tail's share. Find what share of the total the top 1 percent and the top 10 percent of cases carry.
  5. Plan the base on the typical case. Build the forecast from the median and the count of cases.
  6. Treat the giants separately. Name the large cases and give them their own pipeline or reserve, kept out of the average.
  7. Size limits for the tail. Set reserves, caps, and stress tests for the largest plausible case, beyond the largest seen.

Where it goes wrong

The first failure is planning on the average. The plan looks prudent, the typical year falls short, and the shortfall gets blamed on the sales team. Spot it when the mean is two or three times the median, and recover by splitting the plan into a base and a stream of large cases.

The second is calling every skew a power law. Many skewed quantities have tails that thin out quickly, and treating them as wild ties up reserves for nothing, so fit the tail before concluding. The third is taking the biggest case so far as the limit. In a fat-tailed world the record is broken regularly, and a reserve sized to the worst loss on file is sized for the past.

When to use something else

  • Pareto Principle. When the question is which few causes or customers account for most of the effect, as a list to act on.
  • Monte Carlo Simulation. When the plan needs a full range of outcomes from several uncertain inputs combined.
  • Tail-Risk Hedging. When the fat tail is a loss to protect against and the question is what the protection costs.

Where it sits in SOLVE eX

Go to the source

  • Cours d'economie politique, F. Rouge, Lausanne, 1896
  • The Variation of Certain Speculative Prices, Journal of Business 36(4), 1963
  • Power Laws, Pareto Distributions and Zipf's Law, Contemporary Physics 46(5), 2005
  • The Black Swan: The Impact of the Highly Improbable, Random House, 2007

Back to the catalog