Build in Public: What I Learned Building 12 AI Systems for Real Clients

Why I'm Writing This

There's a lot of noise in the AI space right now. Tools that promise to automate everything. Consultants who've never shipped a production system. Frameworks that look elegant in demos and fall apart in the real world.

I've built 12 AI systems for real clients, across real industries, with real operational constraints. These are the lessons that actually matter.

Lesson 1: The Problem Definition Is the Product

The most common mistake I see is jumping to the solution before the problem is fully understood. "We need an AI agent for client onboarding" is not a problem definition. It's a solution hypothesis.

The real work is upstream: What specifically breaks in your current onboarding process? Where does the client experience degrade? What decisions are being made manually that follow a consistent pattern? What data exists that isn't being used?

The quality of the AI system is almost entirely determined by the quality of the problem definition. I spend more time on this phase than any other.

Lesson 2: Start With the Highest-ROI Automation Target

Not every manual process should be automated. The right target has three characteristics:

1. It's high-frequency (happens multiple times per week)
2. It follows a consistent pattern (the same inputs produce the same outputs)
3. The cost of getting it wrong is recoverable

ESG compliance mapping for a healthcare procurement client was a perfect target: it happened weekly, followed a strict methodology, and errors were caught in review before they caused problems. The result: 62 hours per month of manual work automated at under £10/month in API costs.

Lesson 3: Document Everything Before You Build

Before writing a single line of code, I document the architecture. Every input. Every decision point. Every output. Every edge case. This document becomes the test specification — and it's what I show the client before we start building.

This step alone eliminates most of the scope creep and expectation mismatches that kill AI projects.

Lesson 4: Test Against Real-World Scenarios

I test every system against a minimum of 30 real-world scenarios before it goes live. Not synthetic test cases — actual examples from the client's operations. Edge cases. Unusual inputs. The situations that happen once a month but matter enormously when they do.

This is where most AI projects fail. The demo works. The production system doesn't.

Lesson 5: Measure Everything

Every system I build has a performance dashboard. Time saved. Accuracy rate. Error rate. Cost per operation. These numbers matter for two reasons: they prove ROI to the client, and they tell me where to improve.

If you can't measure it, you can't improve it. And if you can't prove the ROI, the client won't renew.

Lesson 6: Build for Handoff

The best AI system is one that the client can understand, maintain, and improve without me. I document every system to the same standard as a paid engagement: architecture diagrams, decision logic, test results, maintenance procedures.

This isn't just good practice. It's the difference between a project and a product.

What This Means for You

If you're considering AI automation for your business, the questions to ask aren't about the technology. They're about the problem: Is it well-defined? Is it high-frequency? Does it follow a consistent pattern? Can you measure success?

If the answer to all four is yes, you have a strong automation candidate. If the answer to any is no, that's where the work starts.