From Karl Popper (Logik der Forschung, 1935; in English as The Logic of Scientific Discovery, 1959).
Pairs every claim with the specific observation that would prove it wrong, then sets a date for the test, exposing beliefs that cannot fail and sharpening those that can.
Four claims from a strategy plan, each paired with what would prove it wrong
At each of the lead's turns, pick the test for the claim: the reply shows whether any result could still count against it. A good test names an observation, a threshold, and a date; a claim no observation could refute is marked a value. Illustrative.
Strategy lead
The plan's claims
Strategy lead
The plan's claims
Claims made testable or marked as values: 0 of 4Claims made testable or marked as values: 1 of 4Claims made testable or marked as values: 2 of 4Claims made testable or marked as values: 3 of 4Claims made testable or marked as values: 4 of 4
- Customers love the brand (Strategy lead). The plan's first claim, and the one it leans on hardest: our customers love the brand.
- Track brand mentions on social media and report the positive ones. Mentions rise in a good month and fall in a quiet one, and either can be read as love. Cannot fail: it looks only for evidence that agrees.
- If fewer than 40% of last year's buyers buy again by June 30, the claim is false. On July 1 the reorder rate will either clear 40% or it will not. Falsifiable: an observation, a threshold, and a date.
- The brand test (The plan's claims). Write the refuting result beside the claim, with the date, before anyone sees the data.
- Price lifts margin (Strategy lead). The plan raises prices on the bedding line by 8 percent and says margin will rise.
- Watch whether margins improve over time. Any rise, in any month, can be credited to the price. Cannot fail: no threshold and no deadline.
- If bedding gross margin is not above 34% at the end of Q2, the price change did not work. The Q2 close will settle it. Falsifiable: a named number on a named date.
- The price test (The plan's claims). The test also says what happens next: if it fails, the old price returns in Q3.
- Redesign cuts calls (Strategy lead). The plan says the redesigned checkout will reduce calls to support.
- Survey customers on whether the new checkout feels easier. Most say yes. The call volume is not in the survey. Tests a different claim: what people say about ease, not what they do.
- If calls per 1,000 orders are not under 40 within 60 days of launch, it did not. The call log will show it, sixty days out. Falsifiable: the claim's own measure, a threshold, and a date.
- The checkout test (The plan's claims). Today's rate is 55. The claim is bold enough to fail, which is what makes passing worth something.
- We put customers first (Strategy lead). The plan's last line: we put customers first.
- Test it against the customer satisfaction score each quarter. A low score is explained as a lag, a high one as proof. Any result fits, so the test is decoration.
- Mark it a value: it guides choices, and no observation could show it false. It moves to the values page, and the plan stops citing it as evidence. Right: a commitment worth keeping, kept apart from the hypotheses.
- A value, not a test (The plan's claims). These four claims now stand as three dated tests and one value, each labeled as what it is.
An example
A home goods retailer's strategy plan rested on twelve claims, among them that customers loved the brand, that an 8 percent price rise on bedding would lift margins, and that a redesigned checkout would cut calls to support. At the review, the chief financial officer asked one question of each claim: what would we see if this were wrong, and by when?
Four claims became tests. If fewer than 40 percent of last year's buyers bought again by June 30, the brand claim was false. If bedding margin was not above 34 percent at the end of the second quarter, the price had not worked. If calls per 1,000 orders were not under 40 within sixty days of launch, the redesign had not done its job. Three claims, such as putting customers first, moved to a values page, since nothing could refute them. Five were cut as wishes. By July, the margin test had passed and the call test had failed: calls fell from 55 to 49. The team reworked the returns flow, which drove most of the remaining calls.
Why it works
A claim that no result could contradict feels strong, because nothing ever counts against it. In fact it says nothing, since it permits every outcome. Most strategy claims are written this way by accident: no threshold, no date, so any result can be read as support and the plan never learns.
Pairing each claim with the specific observation that would refute it forces the claim to say something. It also exposes the claims that cannot be tested, which are often values: worth holding, but not evidence for a forecast. The date matters as much as the threshold, because a test with no deadline can always be extended. A bold claim that survives a real chance of failing is worth far more than a safe one that could never have failed.
The thinking behind it
Karl Popper Karl Popper, an Austrian-British philosopher of science, set out falsifiability in Logik der Forschung (1935), published in English as The Logic of Scientific Discovery (1959). He held that a theory is scientific only if some possible observation could refute it, and that science advances by bold conjectures and attempted refutations, not by piling up confirmations. A theory that fits every outcome is, on his account, outside science. The business use of the test keeps his core move: name the refuting observation before the evidence comes in.
How to run it
- List the claims the plan rests on. Write each one as a plain sentence, including the ones everyone takes for granted.
- Ask what would prove each wrong. Name the specific observation that would contradict the claim if it were false.
- Set a threshold. Put a number on the observation, such as a reorder rate under 40 percent.
- Set a date. Fix when the result will be read, so the test cannot be quietly extended.
- Mark what cannot be tested. Move claims no observation could refute to a values page, and stop citing them as evidence.
- Write down the response to failure. Agree in advance what changes if the test fails, such as restoring the old price.
- Read the results on the date. Report each test as passed or failed, without rewording the claim afterward.
Where it goes wrong
The most common failure is rewriting the claim after the result. The redesign did not cut calls to 40, so the team says it was really about satisfaction. Write the test down before the data arrives, and read it as written.
The second is a test with no teeth: a threshold so low that it cannot fail, or a measure that only counts the good news. If everyone is sure the test will pass, raise the bar or pick the claim's own measure.
The third is calling every value untestable to protect it. Some claims that sound like values, such as customers loving the brand, do have measures. Mark a claim as a value only when no observation at all could count against it.
When to use something else
- Pre-Registered Predictions. When the forecasts themselves need writing down and scoring, with their probabilities, before the results come in.
- Randomized Controlled Trial. When the claim can be tested by assigning the change at random against a control group.
- Assumption Audit. When the plan's hidden assumptions need finding first, before any of them can be turned into tests.
Where it sits in SOLVE eX
Go to the source
- Logik der Forschung, Julius Springer, Vienna, released late 1934 with a 1935 imprint; English edition The Logic of Scientific Discovery, Hutchinson, 1959, 1935