Least privilege, least complexity: keeping AI-built data solutions simple

LLMs will happily design a data solution with more pieces than the problem needs, and explain confidently why every one of them is necessary. Most of the time the better answer is the simplest one that still follows good practice.

They can sketch the whole thing in a few minutes now. Pipelines, tables, orchestration, a dashboard on top. That's useful, but sounding reasonable isn't the same as being right for the problem in front of you.

To be clear, this isn't an argument against enterprise data practices. Governance, lineage and good architecture matter as much as ever. It's a caution about trusting AI designs too much.

Don't take the design on faith

LLMs will make decisions and then justify them, and the justification often reads better than the decision. An extra layer for scalability. A new tool for flexibility. A second copy of the data for performance. Sometimes that's right. Often it's solving a problem nobody has.

So before accepting a design, it's worth asking a few questions about each piece:

What problem does this actually solve?
What happens if we take it out?
Could someone else on the team explain it, or fix it on a bad day?
Is there a simpler way that does the same job?
Does it fit our architecture, governance and lineage standards?

If the honest answer to the second question is "nothing much", it probably shouldn't be there.

Borrow an idea from security

Security has the principle of least privilege: a user or a process should get only the access it needs to do its job, nothing more. It sounds obvious and still gets ignored all the time.

I think the same philosophy works for data solutions. Call it least complexity: build only what the problem needs. A view before a new pipeline. A scheduled query before a streaming setup. One table before five. A tool you already have before a new one.

Simple, not sloppy

Least complexity works inside your standards, never instead of them. Data governance, access controls, risk mitigation, clear lineage, documented architecture, data quality checks and privacy requirements all still apply. They're what make data trustworthy at scale, and they're often the very things an AI-generated design leaves out or treats as an afterthought.

So the test for each piece is simple: is it required by the problem, or by our standards? If it's neither, question it. The combination is what works: enterprise best practices plus the simplest solution that meets them.

Why over-engineering costs more now

When building was slow, it made some sense to build for every future need up front. That's changed. We can build quickly now, so speculative pieces tend to cost more than they save.

That's not true of foundations. Governance, lineage and core architecture are much harder to add after the fact, so they're worth getting right from the start. What isn't worth it is building for needs nobody has yet.

And things move fast. Requirements change, tools change and the models themselves change every few months. An elaborate design locks in assumptions that may not hold for long. The extra complexity is also what's hardest to hand off, explain to the business or debug at the worst possible moment.

What this looks like when you lead a team

For me it mostly comes down to defaults. The simplest thing that works is the starting point, and additions are what need defending, not removals. If a design can't be explained to a peer or a business partner in a few sentences, it's probably doing too much.

None of this means using AI less. It means staying the person who decides. The goal isn't the most impressive design. It's the one you can defend.