Survey weights are not optional

If you analyze a household survey without its weights, you get a precise answer to the wrong question. An unweighted average describes the people who were interviewed, not the population they were chosen to represent.

This comes up constantly with the Demographic and Health Surveys, which are some of the most used data in global health. They're carefully designed and freely available, which also makes them easy to misuse.

Why the sample isn't the population

Surveys like the DHS don't pick households completely at random from a national list. They usually sample in stages: first clusters (small areas like villages or city blocks), then households within those clusters. And they often deliberately sample more households in small regions so there's enough data to report on each one.

That's good design. But it means a household in a small, oversampled region had a higher chance of being picked than one in a big region. If you just average everyone, the small region counts for more than it should.

The weight corrects for that. Roughly, it says how many people in the population each respondent stands in for.

The DHS details that trip people up

A few things worth knowing before you open the files:

The weight has implied decimals. In the women's file the weight is v005, stored as an integer with six implied decimal places, so you divide it by 1,000,000 before using it. The variable documentation says it should be used for all tabulations.
Each file has its own weight. Household, women's and men's files are weighted separately. Using the household weight on individual women is a common slip.
The weights are normalized. They're scaled so the weighted number of cases equals the unweighted number. That's fine for percentages and means, but it means you can't sum them to get a population total.
The design affects uncertainty, not just averages. Clustering means people in the same village are more alike than people chosen at random, so your standard errors need the cluster and strata variables too. Ignoring them gives confidence intervals that are too narrow.

The DHS Program's own Guide to DHS Statistics shows how each standard indicator is calculated, including which weight and which denominator. It's worth reading the definition before calculating an indicator yourself, because "the same" indicator computed slightly differently is how two reports end up disagreeing.

Why this matters more now

It used to be that working with survey microdata meant a statistician who already knew these rules. Now anyone can load a file and ask an assistant for a chart. That's a good thing. But an LLM will happily compute an unweighted mean if you don't ask for a weighted one, and the result will look completely reasonable.

So when I review survey analysis, the first questions are simple. Which weight did you use? Did you account for clustering? Does your number match the published report for the same indicator? That last one is the best test there is. If you can reproduce the official figure, you probably have the setup right, and then you can go further.

The takeaway

Before trusting a number from a survey, try to reproduce one the survey itself published. If you can't, stop there and figure out why.