From engineering design practice, where the notion is traced to Bernard Forest de Bélidor (La science des ingénieurs, 1729).
Sizes capacity as a multiple of the expected load, with the multiple set by how bad a failure would be and how uncertain the estimates are.
Two thousand possible launch-day peaks, and the capacity a factor of safety buys
The expected launch-day peak needs 400 servers; each run draws a real peak around it, wider when the forecast is shakier, with a small chance of a surge from press coverage. The dashed line is the capacity bought: 400 servers times the factor of safety. Runs to its right are outages. Set how uncertain the forecast is, then pick the factor that pushes the failure chance below what an outage would cost. Illustrative.
- Peak against the forecast: around 1.00 times the forecast, give or take 0.15.
- Press surge: a 4% chance of 0.4 times the forecast on top.
Launch-day peak, as a multiple of the forecast
Runs so far: 2,000 of 2,000
- Chance the peak exceeds capacity
- 11.7%
- Peak in the worst run in a hundred
- 1.46×
- Average peak
- 1.01×
An example
An online ticketing company was preparing for the release of a stadium tour, the biggest on-sale in its history. The traffic forecast called for a peak needing 400 servers, and the infrastructure team planned to provision 480, a factor of 1.2, which was its usual margin.
The head of reliability asked two questions the usual margin skipped. How wrong had launch forecasts been before? About 15 percent either way, one standard deviation, judging by the last ten launches. And what would an outage cost? Refunds, a front-page story, and the promoter's contract, which was up for renewal. At 1.2, the simulation of possible peaks put the chance of running out at about one in nine, a risk nobody in the room would sign.
The team settled on a factor of 1.5, 600 servers, which brought the chance to about 1 percent for an extra day of cloud spend. They wrote down the forecast error and the cost of failure beside the factor, so the next launch could start from the reasoning behind the number.
Why it works
A load estimate is a guess with an error, and the error is often larger than the people who made it believe. Capacity set equal to the estimate fails about half the time the estimate is right on average. A factor of safety buys a margin above the estimate, sized to two things: how uncertain the estimate is, and how bad a failure would be. A shaky forecast or a costly failure calls for a larger factor; a well-measured load with a cheap, recoverable failure can run lean.
The factor also covers what the model leaves out. Surges, correlated failures, and errors in the estimate's own assumptions add to the tail, and the margin absorbs some of them. Writing down why the factor was chosen keeps it tied to those reasons when someone later proposes trimming it.
The thinking behind it
Engineering design practice No single person is credited with the factor of safety; it grew up in engineering design practice. The idea is traced to Bernard Forest de Bélidor's La science des ingénieurs (1729), a treatise on the conduct of fortification and civil building works, an attribution made by Isaac Elishakoff. The phrase itself is first recorded in the 1850s, and from the middle of the nineteenth century the convention was written into structural and mechanical design codes. Wilhelm Albert's chain tests of 1837, sometimes cited as its origin, concern metal fatigue, not the factor itself.
How to run it
- Estimate the expected load. Use the best data available for the demand the system, budget, or buffer must carry.
- Measure your forecasting error. Compare past forecasts with what happened to see how far off they usually run.
- Price a failure. Write down what running out would cost: money, customers, safety, reputation, and whether it can be recovered.
- Choose the factor. Pick the multiple that brings the chance of failure below what that cost can tolerate, starting from industry norms where they exist.
- Set capacity at load times the factor. Buy or reserve that capacity, and price the margin so the trade is explicit.
- Record the reasoning. Keep the forecast error, the failure cost, and the factor together, so the margin is not trimmed by habit.
- Revisit after each peak. Compare the real peak with the forecast and adjust the factor for the next one.
Where it goes wrong
The factor is a habit. Every project gets 1.2 because the last one did, whatever the uncertainty or the stakes; spot it when nobody can say where the number came from. Tie the factor to measured error and the cost of failure each time.
The margin is spent twice. Each layer adds its own buffer, or one buffer is counted against two risks, and the total is either wasteful or thinner than it looks. Add up the margins across the whole chain.
The factor hides a bad estimate. A generous margin lets a team skip measuring the load at all. Keep improving the forecast, and lower the factor only as its error shrinks.
When to use something else
- Monte Carlo Simulation. When the load comes from several uncertain parts and the whole distribution is needed.
- Queueing Theory. When the load arrives at random over time and the question is how long waits get, more than whether the system holds.
- Tail-Risk Hedging. When the risk is a rare extreme that no affordable margin covers and needs insuring instead.
Where it sits in SOLVE eX
Go to the source
- La science des ingénieurs dans la conduite des travaux de fortification et d'architecture civile, the attribution is Isaac Elishakoff's, 1729