Words of Estimative Probability

Thinking Tools3.4 Facts About the FutureAlso known as Kent scale

Developed by Sherman Kent (Words of Estimative Probability, 1964), later fixed for United States intelligence analysis in the 2015 revision of Intelligence Community Directive 203.

A fixed vocabulary that ties each probability word, from almost certain to almost certainly not, to a numeric range, so writer and reader mean the same odds.

Kent's scale ties each word to a band of odds

Kent's 1964 scale: five estimative words, each a bar over the odds it stands for. The four squares are the numbers behind a manufacturer's board memo. Move a number and see which word's bar it lands in: that word goes in the memo, and a number between bars, such as 61 or 15 percent, is a call to decide which way the evidence leans. Illustrative.

80%
50%
30%
95%
Numbers that land between two words' bars
0 of 4
Almost certain: about 87 to 99%Probable: about 63 to 87%Chances about even: about 40 to 60%Probably not: about 20 to 40%Almost certainly not: about 2 to 12%0%25%50%75%100%RenewalRivalOverrunApprovalAlmost certain: about 87 to 99%Probable: about 63 to 87%Chances about even: about 40 to 60%Probably not: about 20 to 40%Almost certainly not: about 2 to 12%0%25%50%75%100%RenewalRivalOverrunApproval

Pick the word for each estimate, and see how the board reads it

Four estimates for a manufacturer's board memo, each with a number behind it. At each analyst turn, pick the word that goes in the memo: the board's reading shows, with Kent's range for the word. The board's turns show what the shared scale lets them do. Illustrative.

Analyst

Board

Analyst

Board

Select a stage to read it, or play the sequence.Renewal. The account team puts the chance that the largest customer renews at 80 percent. Pick the word for the memo.Pick a move to see the reply.The board reads a done deal and stops planning for losing the account.Too strong: almost certain covers about 87 to 99 percent.The board reads about three chances in four and keeps a fallback plan.Matches: probable covers about 63 to 87 percent.One director hears 20 percent, another 60, and the meeting argues about the word.No range: possible lets every reader pick their own odds.A director asks. The scale is printed on the memo's last page, so the answer takes one line: probable means about 63 to 87 percent. Without it, one director hears 60 percent and another hears 90.New rival. Strategy puts the chance that a known rival opens a plant in the region this year at 50 percent.Pick a move to see the reply.The board reads a likely entry and approves a price cut in advance.Too strong: probable starts around 63 percent.The board reads a coin toss and asks for a plan it can trigger if the rival announces.Matches: chances about even covers about 40 to 60 percent.The board reads a low threat and drops it from the agenda.Too weak: probably not covers about 20 to 40 percent.The board plans. An even chance tells the board to fund a response plan now and commit the money only when the rival announces.Overrun. The project office puts the chance that the plant upgrade runs over budget at 30 percent.Pick a move to see the reply.The board reads no real risk and releases the contingency reserve.Too weak: almost certainly not covers about 2 to 12 percent.The board reads a serious risk and delays the upgrade a quarter.Too strong: chances about even starts around 40 percent.The board reads a real but minor risk and keeps the reserve in place.Matches: probably not covers about 20 to 40 percent.The board keeps a reserve. Probably not still means about three chances in ten. The board keeps the contingency reserve and asks for a monthly cost report.Approval. Regulatory affairs puts the chance that the new product is approved by June at 95 percent.Pick a move to see the reply.The board cancels the fallback plan, and a one in twenty chance now has no plan at all.Wrong kind of word: certain states a fact, and no estimate should claim it.The board reads three in four and pads the launch plan with a costly delay buffer.Too weak: probable tops out around 87 percent.The board commits the launch budget and keeps one named fallback.Matches: almost certain covers about 87 to 99 percent.The board commits. Almost certain lets the board commit the launch budget while one person owns the fallback for the remaining one chance in twenty.

Words that match the number: 0 of 4Words that match the number: 1 of 4Words that match the number: 2 of 4Words that match the number: 3 of 4Words that match the number: 4 of 4

  1. Renewal (Analyst). The account team puts the chance that the largest customer renews at 80 percent. Pick the word for the memo.
    • Almost certain The board reads a done deal and stops planning for losing the account. Too strong: almost certain covers about 87 to 99 percent.
    • Probable The board reads about three chances in four and keeps a fallback plan. Matches: probable covers about 63 to 87 percent.
    • Possible One director hears 20 percent, another 60, and the meeting argues about the word. No range: possible lets every reader pick their own odds.
  2. A director asks (Board). The scale is printed on the memo's last page, so the answer takes one line: probable means about 63 to 87 percent. Without it, one director hears 60 percent and another hears 90.
  3. New rival (Analyst). Strategy puts the chance that a known rival opens a plant in the region this year at 50 percent.
    • Probable The board reads a likely entry and approves a price cut in advance. Too strong: probable starts around 63 percent.
    • Chances about even The board reads a coin toss and asks for a plan it can trigger if the rival announces. Matches: chances about even covers about 40 to 60 percent.
    • Probably not The board reads a low threat and drops it from the agenda. Too weak: probably not covers about 20 to 40 percent.
  4. The board plans (Board). An even chance tells the board to fund a response plan now and commit the money only when the rival announces.
  5. Overrun (Analyst). The project office puts the chance that the plant upgrade runs over budget at 30 percent.
    • Almost certainly not The board reads no real risk and releases the contingency reserve. Too weak: almost certainly not covers about 2 to 12 percent.
    • Chances about even The board reads a serious risk and delays the upgrade a quarter. Too strong: chances about even starts around 40 percent.
    • Probably not The board reads a real but minor risk and keeps the reserve in place. Matches: probably not covers about 20 to 40 percent.
  6. The board keeps a reserve (Board). Probably not still means about three chances in ten. The board keeps the contingency reserve and asks for a monthly cost report.
  7. Approval (Analyst). Regulatory affairs puts the chance that the new product is approved by June at 95 percent.
    • Certain The board cancels the fallback plan, and a one in twenty chance now has no plan at all. Wrong kind of word: certain states a fact, and no estimate should claim it.
    • Probable The board reads three in four and pads the launch plan with a costly delay buffer. Too weak: probable tops out around 87 percent.
    • Almost certain The board commits the launch budget and keeps one named fallback. Matches: almost certain covers about 87 to 99 percent.
  8. The board commits (Board). Almost certain lets the board commit the launch budget while one person owns the fallback for the remaining one chance in twenty.

An example

The finance director of a mid-sized auto parts maker used to write board memos full of words like likely, possible, and a real chance. Then one quarter the directors read "likely" on a supplier failure as a near certainty and paid to qualify a second source early. She adopted a fixed scale: every estimate in the memo now carries one of five words, each tied to a range, from almost certain down to almost certainly not, with the table on the memo's last page.

The next memo had four estimates. The largest customer renewing, at 80 percent, was probable. A rival entering the region, at 50, was chances about even. The plant upgrade running over budget, at 30, was probably not. Regulatory approval by June, at 95, was almost certain. The board spent its time on what to do about each risk, and two directors asked to see the numbers too, which she now gives in brackets.

Why it works

Probability words feel precise to the writer and vague to the reader. The writer knows the number behind "likely"; each reader supplies a number of their own, and different readers supply very different ones. A board can agree on the wording and disagree on the odds without knowing it.

Tying each word to a range closes that gap at almost no cost. The writer still writes in words, which suit judgments too rough for a single number, but the words now carry a shared meaning. The ranges also bind the writer: picking a word commits to a band of odds, so the forecast can be checked later against what happened. Words such as possible drop out, since they fit any odds and so commit to none.

The thinking behind it

Sherman Kent Sherman Kent, a Yale historian, led the CIA's Office of National Estimates from 1952 to 1967 and is called the father of intelligence analysis. In Words of Estimative Probability (Studies in Intelligence, 1964) he proposed tying words such as probable and almost certain to fixed ranges of odds, after finding that readers took the same phrase to mean very different chances. His scheme was not adopted in his time. Estimative lexicons spread through the intelligence community only after 2005, and the 2015 revision of Intelligence Community Directive 203 fixed a set of likelihood terms with ranges.

How to run it

  1. Adopt one scale. Pick a published set of words and ranges and use it in every memo, forecast, and risk register the group writes.
  2. Put a number on each estimate first. Write the probability before choosing a word, so the word follows the judgment.
  3. Choose the word whose range holds the number. If the number sits on a boundary, decide which way the evidence leans and say so.
  4. Strike the words with no range. Possible, could, and a real chance fit any odds; replace each with a word from the scale or a number.
  5. Print the scale with the document. A short table on the last page lets every reader check what a word means.
  6. Give the number where precision matters. For a decision that turns on a few points of probability, add the figure in brackets.
  7. Check the words against outcomes. Each quarter, see whether the events called probable happened about three times in four.

Where it goes wrong

The common failure is a scale the writers use and the readers never learn. Analysts pick words carefully and directors read them the old way, because no one showed them the ranges. Print the scale in every document, and spend five minutes on it in the first meeting that uses it.

A second failure is letting the word stand in for the judgment. A writer who reaches for probable out of habit has moved the vagueness from the reader to the writer; ask for the number first. The third is false precision at the edges: 62 and 64 percent fall either side of a boundary, yet the difference is noise. When an estimate sits near a boundary, say so and give the number.

When to use something else

  • Brier Score. When the estimates are already numbers and the question is how good the forecaster has been over many calls.
  • Reference Class Forecasting. When the number behind the word is itself in doubt, and the first job is anchoring it to how often such things happen.
  • Calibration Training. When people need practice making their stated odds match how often they turn out right.

Where it sits in SOLVE eX

Go to the source

Back to the catalog