Weighted Decision Matrix

Thinking Tools5.2 Selecting the Right Decision ToolsAlso known as Pugh matrix, Decision matrix

From weighted scoring going back to the prudential algebra of Benjamin Franklin (1772), formalized by C. West Churchman and Russell L. Ackoff (1954), brought to managers by Charles Kepner and Benjamin Tregoe (1965), with the baseline variant of Stuart Pugh (1981).

Scores each option against weighted criteria and sums the results into a ranking, with a variant that rates options better, same, or worse than a baseline.

Four CRMs, one must, four weighted criteria, and a test of the lead

Each score, 1 to 10, is multiplied by its criterion's weight and summed. The open-source CRM fails the must, so it is out despite the top total; the mid-market CRM leads the rest by 3 points. Now test that lead: cut the weight on reporting by a fifth, from 5 to 4, and the lightweight CRM takes over. A lead that flips on a small shift is a close call. Illustrative.

Syncs with the billing system out of the box

MustEnterprise suiteMid-market CRMLightweight CRMOpen-source CRM

Ease of use for the sales team

Weight6Enterprise suite4Mid-market CRM7Lightweight CRM9Open-source CRM7

Reporting and forecasting

Weight5Enterprise suite9Mid-market CRM7Lightweight CRM3Open-source CRM6

Cost over three years (10 is the cheapest)

Weight4Enterprise suite4Mid-market CRM6Lightweight CRM8Open-source CRM10

Vendor support

Weight3Enterprise suite9Mid-market CRM6Lightweight CRM5Open-source CRM5

Weighted total

Enterprise suite112OutMid-market CRM119OutLightweight CRM116OutOpen-source CRM127Out

Comes out on topNo option passes every must: rescope or find another.Enterprise suiteMid-market CRMLightweight CRMOpen-source CRM

In order

  1. 119Enterprise suiteMid-market CRMLightweight CRMOpen-source CRMOut: fails a must (Syncs with the billing system out of the box).
  2. 116Enterprise suiteMid-market CRMLightweight CRMOpen-source CRMOut: fails a must (Syncs with the billing system out of the box).
  3. 112Enterprise suiteMid-market CRMLightweight CRMOpen-source CRMOut: fails a must (Syncs with the billing system out of the box).
  4. 127Enterprise suiteMid-market CRMLightweight CRMOpen-source CRMOut: fails a must (Syncs with the billing system out of the box).

An example

A software company's forty-person sales team needed a CRM, and the sales operations manager had four candidates and a week to recommend one. She held a first meeting only to agree on the criteria and weights, before anyone saw a score: ease of use at 6, reporting at 5, three-year cost at 4, and vendor support at 3, plus one must, a working sync with the billing system. The open-source option failed the must and was out, despite the highest total. Of the other three, the mid-market CRM came to 119, the lightweight one to 116, and the enterprise suite to 112.

Then she moved each weight by a fifth. Most shifts left the order alone, but cutting reporting from 5 to 4 put the lightweight CRM on top. The decision turned on one question: how much the team relied on reporting. The vice president checked, found reporting fed every weekly forecast review, kept its weight at 5, and approved the mid-market CRM with that reason written on the matrix.

Why it works

A group comparing options in conversation tends to weigh whatever was said last, and each person carries a private ranking of the criteria. The matrix brings both into the open: the weights say what matters and by how much, the scores say how each option does, and the arithmetic does the combining that people do badly in their heads. Agreeing on the weights before scoring keeps them honest, since no one yet knows which weight helps which option.

The sensitivity check turns a total into a judgment. A lead that survives every weight moving by a fifth is robust. A lead that flips on one small shift is a close call, and it names the exact disagreement worth the rest of the meeting.

The thinking behind it

Benjamin Franklin Benjamin Franklin, the printer, scientist, and statesman, described the method's oldest well-known form in a letter to Joseph Priestley in September 1772. Asked for advice on a hard choice, he called it moral or prudential algebra: list the reasons for and against in two columns over several days, estimate the weight of each, strike out reasons on opposite sides that balance, and see where the balance lies. The weighted matrix keeps his two essentials, a weight on every reason and a written record, and sets several options side by side.

C. West Churchman C. West Churchman, a philosopher and systems scientist at the University of California, Berkeley, and a pioneer of operations research, formalized the idea with Russell Ackoff in An Approximate Measure of Value (1954), in the Journal of the Operations Research Society of America. They proposed measuring the value of alternatives by combining weighted scores on several objectives, an early form of the weighted decision matrix.

Stuart Pugh Stuart Pugh, a British professor of engineering design, added the baseline variant in Concept Selection: A Method That Works (1981), given at the International Conference on Engineering Design in Rome. Each candidate design is compared with a chosen baseline, criterion by criterion, as better, the same, or worse; the team then strengthens the leaders and runs the comparison again. He later placed concept selection inside a whole approach to product development in Total Design (1991).

How to run it

  1. List the options and the criteria. Three to five options and four to seven criteria, each stated so a score can be argued from evidence.
  2. Screen on the musts. Name any absolute requirement and drop every option that fails it before scoring starts.
  3. Agree on the weights first. Weight each criterion from 1 to 10 in one meeting before anyone sees a score, and record the reason for the top two.
  4. Score every option on every criterion. Use a 1 to 10 scale with its ends defined, such as 10 for setup in a day, and note the evidence behind each score.
  5. Multiply, sum, and rank. Each score times its weight, summed per option; let a spreadsheet do the arithmetic while the meeting reviews it.
  6. Shift each weight by a fifth. Move one weight at a time up and down by about 20 percent and watch whether the leader changes.
  7. Settle what the shift reveals. A lead that holds is the choice; a lead that flips points to the one weight the group must settle first.

Where it goes wrong

Weights set after the scores are the classic failure: everyone knows which option each weight helps, and the matrix dresses up a decision already made. Spot it when weights change between meetings, and freeze them before scoring.

The second is criteria that overlap. Ease of use, training time, and adoption measure much the same thing, so one quality gets counted three times and the option strong on it wins by repetition. Merge criteria that move together.

The third is false precision. A gap of 119 to 116 is noise when every score is a judgment. Run the sensitivity check, and when the lead flips, say so and settle the weight in question before declaring a winner.

When to use something else

  • Analytic Hierarchy Process. When the weights themselves are contested and the group needs pairwise judgments with a check on their consistency.
  • Kepner-Tregoe Method. When the choice sits inside a tangle of problems and risks that also need sorting out.
  • Plus, Minus, Interesting. When the options are still forming and a quick list of pluses, minuses, and open questions will do more than a scored grid.

Where it sits in SOLVE eX

Go to the source

  • Letter to Joseph Priestley on moral or prudential algebra, 1772
  • An Approximate Measure of Value, Journal of the Operations Research Society of America 2, 1954
  • The Rational Manager, McGraw-Hill, 1965
  • Concept Selection: A Method That Works, 1981

Back to the catalog