What I ask before I review someone's analysis
The most useful part of reviewing someone's analysis usually happens before I open their notebook. A few plain questions about what it's for and how it was checked catch more problems than reading code line by line, and they teach the habit instead of just fixing the instance.
Why not start with the code
Reading code first pulls you toward style. Variable names, a loop that could be a join, a chart color. Those are fine to mention, but they're rarely why an analysis is wrong. Analyses are usually wrong because of the question, the inputs or a step nobody checked.
And starting with the code puts the reviewer in charge. Starting with questions keeps the author in charge of their own work, which is where it should be.
The questions
What question does this answer, in one sentence? If it takes a paragraph, the analysis is probably trying to do several things. That's the most common issue and the easiest to fix early.
What decision or person is it for? This sets the bar. A quick number for an internal conversation needs less than a figure that will be quoted in a report.
Where did the data come from, and when? Which extract, which version, which date. Half the disagreements between two analyses come from two slightly different pulls of the same data.
What did you exclude, and why? Filters are where most of the judgment hides. Dropped rows, date cuts, "invalid" records. Each one is a decision that should be written down.
How do you know it's right? I'm looking for at least one independent check. Does a total match a published figure or another system? Does a known case come out the way it should? Did you look at a few individual rows, not just the summary?
What surprised you? A surprising result is either the interesting finding or a bug, and the author usually has a feeling about which. This question gets that feeling said out loud.
What would you do with another week? This tells me what they think is weak, which is usually more accurate than what I'd guess.
When AI wrote part of it
More and more analysis is partly written by an assistant, and I don't think that changes the questions. It makes "how do you know it's right" more important. Generated code tends to run cleanly and look confident, so the check has to come from somewhere other than "it ran". I'd rather someone tell me plainly that an assistant wrote the join and they verified it against a known total than leave it unsaid.
How I try to give the feedback
- Separate "this changes the answer" from "this is how I'd write it". Only the first kind is required.
- Ask before telling. "What happens to rows with no date?" lands better than "you dropped the nulls".
- Say what's good. People need to know what to keep doing, not just what to fix.
The takeaway
The goal of a review isn't to find what I would have done differently. It's to make sure the answer is right and that the person who wrote it can explain why. If those questions become the author's habit, the next review gets shorter.