Causal Diagram

Thinking Tools4.1 Root Causes and the Case for Going BackAlso known as Causal DAG, Directed Acyclic Graph

Developed by Judea Pearl (Causality, 2000) from the path analysis of Sewall Wright (Correlation and Causation, 1921).

Maps assumed causes as a directed graph of variables and arrows, showing which factors to control for, which to leave alone, and whether an effect can be estimated at all.

Did the sales training raise sales? The diagram of what causes what

Each arrow says one thing directly causes another. Switch views to trace each path from training to sales: the direct one is the effect to measure, two back doors run through common causes, and the path through promotion is closed unless someone controls for it. Illustrative.

Select a box to read it.Tenure. Years with the company. Managers picked veterans for the training first, and veterans sell more anyway.Region. The training rolled out in the two strongest regions first, and those regions sell more anyway.Took the training. The treatment: 120 of 400 reps took the course this year.Sales per rep. The outcome: bookings in the six months after the course.Promoted. Promotion follows both: managers promote reps who took the training and reps who sell well. A common effect of the two.The effect. The arrow to measure: how much the training itself changes a rep's sales. The raw comparison of trained and untrained reps mixes it with every other path below.Tenure. A back door: training comes from tenure, and so do sales. Left open, it makes the training look better than it is. Controlling for tenure blocks it.Region. A second back door, through where the training ran first. Controlling for region blocks it.Promotion. Not a back door: both arrows point into promotion, so the path is closed as it stands. Controlling for promotion (studying only promoted reps, say) opens it and biases the answer.
Tenure
Years with the company. Managers picked veterans for the training first, and veterans sell more anyway.
Region
The training rolled out in the two strongest regions first, and those regions sell more anyway.
Took the training
The treatment: 120 of 400 reps took the course this year.
Sales per rep
The outcome: bookings in the six months after the course.
Promoted
Promotion follows both: managers promote reps who took the training and reps who sell well. A common effect of the two.
The effect
The arrow to measure: how much the training itself changes a rep's sales. The raw comparison of trained and untrained reps mixes it with every other path below.
Tenure
A back door: training comes from tenure, and so do sales. Left open, it makes the training look better than it is. Controlling for tenure blocks it.
Region
A second back door, through where the training ran first. Controlling for region blocks it.
Promotion
Not a back door: both arrows point into promotion, so the path is closed as it stands. Controlling for promotion (studying only promoted reps, say) opens it and biases the answer.

Choose what to control for, and see whether the training effect can be measured

Each row is a variable from the diagram: control for it or leave it alone. The analyst's first try is set at load: tenure and promotion. The lines under the rows say which paths are open; find the set that blocks every back door and opens nothing. Illustrative.

Control for tenure

Control for region

Control for promotion

  • Pick a choice in every row.The trap: promotion is a common effect of training and sales. Comparing only promoted reps links training to sales through promotion, and the estimate is biased whatever else is controlled.Tenure still confounds: veterans were trained first and sell more anyway.Region still confounds: the strong regions were trained first.Every back door is blocked and nothing is opened. Comparing trained and untrained reps of the same tenure in the same region measures the training's effect, if the diagram is right.
  • Pick a choice in every row.The trap: promotion is a common effect of training and sales. Comparing only promoted reps links training to sales through promotion, and the estimate is biased whatever else is controlled.Tenure still confounds: veterans were trained first and sell more anyway.Region still confounds: the strong regions were trained first.Every back door is blocked and nothing is opened. Comparing trained and untrained reps of the same tenure in the same region measures the training's effect, if the diagram is right.
  • Pick a choice in every row.The trap: promotion is a common effect of training and sales. Comparing only promoted reps links training to sales through promotion, and the estimate is biased whatever else is controlled.Tenure still confounds: veterans were trained first and sell more anyway.Region still confounds: the strong regions were trained first.Every back door is blocked and nothing is opened. Comparing trained and untrained reps of the same tenure in the same region measures the training's effect, if the diagram is right.
  • The trap: promotion is a common effect of training and sales. Comparing only promoted reps links training to sales through promotion, and the estimate is biased whatever else is controlled. (P: Yes)
  • Tenure still confounds: veterans were trained first and sell more anyway. (T: No)
  • Region still confounds: the strong regions were trained first. (R: No)
  • Every back door is blocked and nothing is opened. Comparing trained and untrained reps of the same tenure in the same region measures the training's effect, if the diagram is right. (T: Yes, R: Yes, P: No)
Back door through tenure
Pick every rowOpenBlocked
Back door through region
Pick every rowOpenBlocked
Path through promotion
Pick every rowClosedOpened by the control
Training effect can be measured
Pick every rowNoYes

An example

A software company's sales enablement team reported that reps who took its new negotiation course booked 30% more in the next six months than reps who did not. The vice president of sales wanted the course made mandatory. An analyst in revenue operations asked how the 120 reps had been chosen.

She drew the diagram before running any numbers. Managers had sent their veterans first, and veterans sell more anyway; the course had also run first in the two strongest regions. Both were back doors, paths from the course to sales that did not go through the course's effect. Her first attempt controlled for tenure and compared only reps who had since been promoted, and the diagram showed why that was a mistake: promotion depended on both the training and sales, so filtering on it opened a new path. Comparing trained and untrained reps of the same tenure in the same region, with promotion left alone, put the course's effect at about 8%. The course stayed, optional, aimed at reps in their first two years.

Why it works

Data alone cannot say what causes what: the same correlation between training and sales fits a world where training works and a world where managers simply trained their best people. A causal diagram writes down which of those worlds the analyst believes in, arrow by arrow, so the assumptions can be argued over openly.

Once drawn, the diagram answers a precise question: which variables must be controlled for to measure one effect? Every path from the cause to the outcome that starts with an arrow pointing into the cause is a back door and must be blocked. A variable that two arrows point into, a common effect, closes the path it sits on, and controlling for it opens that path. The rules are mechanical: two analysts with the same diagram reach the same answer, and if no measured set works, this data cannot answer the question.

The thinking behind it

Judea Pearl Judea Pearl, a UCLA computer scientist and 2011 Turing Award recipient, developed Bayesian networks and then a formal theory of causal inference built on directed acyclic graphs, set out in Causality: Models, Reasoning, and Inference (2000). His framework adds the do-operator for interventions, which separates seeing a variable at a value from setting it there, and structural models for counterfactuals. The rules for reading back doors and common effects off a diagram come from that work.

Sewall Wright Sewall Wright, an American geneticist and a founder of population genetics, invented path analysis in Correlation and Causation (Journal of Agricultural Research, 1921). It was one of the first methods to draw assumed causal relations among variables as a diagram of arrows and to read the strength of each path from the data. Path analysis is a precursor of today's causal graphs.

How to run it

  1. Name the cause and the outcome. One treatment and one result: the course, and bookings in the six months after it.
  2. List the other variables that might matter. Anything that could affect who got the treatment or how the outcome came out.
  3. Draw an arrow wherever one directly causes another. From knowledge of how the business works; leave an arrow out only when confident it is absent.
  4. Check there are no cycles. No variable may cause itself through a chain of arrows; if one seems to, split it by time.
  5. Trace every back door. Each path from cause to outcome that begins with an arrow into the cause.
  6. Choose controls that block them all. Never include a common effect of the cause and the outcome, or anything the cause itself produces.
  7. Add the suspected unmeasured confounders. If one opens a back door that no measured variable can block, say the effect cannot be measured from this data.

Where it goes wrong

The diagram is drawn to fit the analysis. Arrows that would complicate the answer get left off, and the diagram blesses the controls already chosen. Draw it before looking at the data, and have someone who knows the business check the arrows.

Everything gets controlled. The analyst adds every available column on the theory that more controls are safer, and picks up a common effect or a consequence of the treatment. Control only for what blocks a back door.

The answer is trusted beyond the diagram. A clean estimate is only as good as the arrows behind it. State the diagram with the result, so anyone who doubts an arrow knows what to test.

When to use something else

  • Randomized Controlled Trial. When a randomized test is possible, which removes the back doors by design.
  • Difference-in-Differences. When the treatment started at a known time for one group and the question is how its trend changed against another group's.
  • Propensity Score Matching. When the back doors are known and many, and the job is building comparable treated and untreated groups.

Where it sits in SOLVE eX

Go to the source

  • Causality: Models, Reasoning, and Inference, Cambridge University Press; second edition 2009, 2000
  • Correlation and Causation, Journal of Agricultural Research 20, 1921

Back to the catalog