Monte Carlo Simulation

Thinking Tools3.4 Facts About the FutureAlso known as Monte Carlo Method

Conceived by Stanislaw Ulam and developed with John von Neumann at Los Alamos (1946 to 1947), and first published by Nicholas Metropolis and Ulam (The Monte Carlo Method, 1949).

Runs a model thousands of times with inputs drawn at random from their likely ranges, producing the spread of possible outcomes and the odds of each.

A thousand runs of a reporting system's release plan

Three tasks with ranges of durations and one risk that sends the work back. Play draws every input at random 1,000 times and stacks up how long each run took; the dashed lines mark the day half the runs finish by and the day four in five do. Raise the risk or the longest testing could take, and Play again. Illustrative.

25%
10
  • Build the new reports: between 8 and 22 days, most likely 11.
  • Migrate the data: between 4 and 15 days, most likely 6.
  • User testing: anywhere between 3 and 10 days.
  • Security review sends it back: a 25% chance of ten more days.
203040506050%: day 3080%: day 36Promised: day 35203040506050%: day 3080%: day 36Promised: day 35

Days to release

Runs so far: 1,000 of 1,000

Average
31.0 days
Half the runs finish by
day 30
Four in five finish by
day 36
Chance of missing the day-35 promise
24%

An example

A finance team at a mid-size insurer was replacing its month-end reporting system, and the chief financial officer wanted a date to give the audit committee. The project lead added up the most likely durations: eleven days to build the reports, six to migrate the data, six or seven for user testing. That came to about twenty-four days, and she promised day 35 to leave room.

An analyst on the team asked for ranges instead. Each task owner gave a best case, a most likely case, and a worst case, and the security lead said that about one release in four came back from review for ten days of rework. The analyst ran the plan a thousand times in a spreadsheet, drawing each duration from its range. Half the runs finished by day 30. Four in five finished by day 36, and nearly a quarter missed day 35. The lead went back to the CFO with day 38 at better than eight in ten, and a request to book the security review a week earlier.

Why it works

A plan built from single best guesses adds up the most likely value of several skewed ranges, and the likely total of skewed tasks is later than the sum of their likely values. Task durations run long more often than short: a task can finish a day early but can slip a week. Adding most-likely values ignores that skew, and it ignores the rare events, such as a failed review, that hit some runs hard and leave others alone.

Simulation does the arithmetic that intuition cannot. Drawing every input at random from its range, thousands of times, and recording each result builds the actual spread of outcomes. The answer then comes as odds (a date with an eight-in-ten chance) and the inputs that move the tail are plain to see.

The thinking behind it

Stanislaw Ulam Stanislaw Ulam conceived the method at Los Alamos in 1946 while estimating the odds of winning a game of solitaire by playing it many times, and he saw that electronic computers made such sampling practical. With Nicholas Metropolis he published The Monte Carlo Method (Journal of the American Statistical Association, 1949), the first unclassified account of estimating a hard quantity from many random trials.

John von Neumann John von Neumann took up Ulam's idea and turned it into a procedure for computing neutron diffusion, set out in a 1947 proposal to Robert Richtmyer at Los Alamos. In 1948 he and Nicholas Metropolis, with others, ran the first automated Monte Carlo calculations on the ENIAC. His earlier work with Oskar Morgenstern, Theory of Games and Economic Behavior (1944), set out expected-utility theory for choices under risk.

Nicholas Metropolis Nicholas Metropolis, a Los Alamos physicist, developed the method with Ulam and von Neumann and gave it its name. In 1948 he was among the group that ran the first automated Monte Carlo calculations on the ENIAC, and in 1949 he and Ulam published the paper that first described the method in public. The Metropolis algorithm (1953) carries his name, a sign of how far the sampling idea traveled from its first problem.

How to run it

  1. List the uncertain inputs. Name every duration, cost, or rate whose value is not yet known, and the discrete risks that might or might not happen.
  2. Give each input a range. Ask the owner for a best, most likely, and worst case, and a chance for each risk.
  3. Write the calculation. Build the formula that turns one set of inputs into one outcome, the same one a single-point plan would use.
  4. Run it thousands of times. Draw each input at random from its range on every run and record each outcome; a spreadsheet handles a few thousand runs.
  5. Read the spread. Report the median, the 80th or 90th percentile, and the chance of missing the date or budget that matters.
  6. Find the inputs that drive the tail. Narrow one range at a time and rerun; the ones that pull in the 80th percentile are where to spend effort.
  7. Rerun as facts arrive. Replace each range with the actual value once a task is done, and quote the new odds.

Where it goes wrong

The most common failure is ranges that are too narrow. Owners give a worst case that is only a little worse than the likely one, and the simulation reports false confidence. Spot it by comparing the ranges with how past projects actually ran, and widen them until history fits inside.

The second is treating inputs as independent when they move together: if one supplier is late, the next often is too. The spread then comes out too tight. Model the shared cause as one input that feeds several tasks. The third is quoting the average as the plan. A date with an even chance of being missed is a coin toss, so give the percentile the decision needs.

When to use something else

  • Reference Class Forecasting. When there is no model to run, but a record of how similar projects turned out.
  • Sensitivity Analysis. When the question is which single input moves the answer most, and one change at a time is enough.
  • Decision Tree Analysis. When the uncertainty comes as a few discrete outcomes and a choice must be made between them.

Where it sits in SOLVE eX

Go to the source

  • The Monte Carlo Method, Journal of the American Statistical Association 44, 1949

Back to the catalog