Two good estimates, two different numbers

When two respected organizations publish different numbers for the same thing, the instinct is to ask which one is right. In global health that's usually the wrong question. Both are estimates, built from different data and different assumptions, and the gap between them tells you something.

Why they differ

Take maternal mortality. The UN agencies publish estimates through an inter-agency group, and IHME publishes its own through the Global Burden of Disease study. For many countries and years the two don't match, sometimes by a lot.

That isn't because someone made an error. In a lot of places there's no complete registration of births and deaths, so nobody directly counts every maternal death. What exists is a patchwork: household surveys asking about sisters' deaths, hospital records, a census here and there, verbal autopsies. Each group decides which sources to trust, how to adjust for known biases, how to fill the years with no data and how to borrow strength from similar countries.

Make different reasonable choices and you get different reasonable numbers.

Modeled is not a dirty word

People sometimes hear "modeled estimate" and assume it means made up. It's closer to the opposite. A good model is an explicit, documented way of combining imperfect sources, and it comes with uncertainty intervals that say how sure it is.

The alternative is usually worse. A single reported figure can look precise and still be missing most of the deaths it's meant to count. At least a model admits what it doesn't know.

How to read two numbers that disagree

When I see two estimates that don't line up, a few questions help more than picking a side:

What data went in? Is one using a survey from last year that the other hasn't included yet?
Do the uncertainty intervals overlap? Often the point estimates differ but the ranges overlap a lot, and the honest answer is "somewhere in here".
What's being measured, exactly? Definitions vary. Age ranges, time windows and what counts as a cause can all differ.
When was each one released? Estimates get revised backwards as new data arrives. Last year's number for 2015 might not be this year's number for 2015.

There's a standard for this

The field has a reporting guideline for exactly this problem. The GATHER statement, published in 2016 in The Lancet and PLOS Medicine, is an 18-item checklist for studies that combine sources to estimate health indicators. It asks producers to document their data sources, how they were adjusted, the model, and how uncertainty was calculated, and to make the inputs available where they can.

It's a good checklist even outside global health. If you combine messy sources into a single number at work, ask whether someone could tell from your write-up what went in and what you assumed.

What this means for decisions

If a decision changes depending on which estimate you use, that's worth knowing before you make it. It means the evidence isn't strong enough on its own and you need judgment, or more data, or a decision that works across the range.

And if the decision is the same either way, the disagreement doesn't matter much, and you can stop arguing about it.

The disagreement is information. It tells you where the data is thin.