Lesson 27 of 30
Scoping a POC That Leads to Production
The three commitments to insist on before a line of code is written, the exit criteria that let a POC end rather than drift, and the demo trap that turns a successful-looking POC into a cost.
Discovery told you what to build and for whom. Scoping is the colder conversation that decides whether the POC leads to production or to another POC.
Most POCs do not die in a demo. They drift. The customer liked it, nobody was sure what to do next, the champion got moved to another project, and six months later the system is a diagram in someone's deck. An FDE's job is to make that outcome mechanically harder than the alternative.
The three commitments
Before you write a line of code, insist on three things. They are uncomfortable to ask for, and that is the point. The customers willing to make them are the customers whose POCs convert.
An eval set
A small labelled dataset — 50 to 200 examples — with ground-truth answers that someone on the customer side signs off on. It defines what 'working' means in numbers you can both read. Without it, 'it works' is a vibe you will relitigate every Friday.
A named production owner
A specific person who will carry the pager when the agent misbehaves. Not a team, not a committee — a name. If no name is forthcoming, the system has no owner, and a POC without an owner lands and dies.
A written path to production
A short document that says: if we hit these numbers, here is how the POC becomes a production system — the budget, the deployment shape, the integration points, the sign-offs. Not a plan, a path. Without one, success at demo day leads to the same conversation again a quarter later.
The three commitments do the same job together: they turn the POC from an exploration into an experiment. Experiments have a hypothesis and a stopping condition. Explorations have neither, and every week the customer asks whether they should still be running.
Exit criteria written down
What counts as "done" needs to exist on paper before you start. The shape to aim for is a single page with four lines:
- What we are building. One paragraph. The specific workflow being changed, not the technology being used.
- What we are measuring. The eval set, named. The metric the customer's owner cares about on their dashboard. The latency and cost ceilings you will not cross.
- What makes this a yes. The number on the eval set, the number on the dashboard, by what date. All three. Any two of them is a loophole.
- What makes this a no. The honest version. "If accuracy is below X by week three, we stop and recommend not pursuing this." Writing down how the POC fails is the single most alignment-forcing move in the whole engagement.
Customers resist the fourth line. They want the POC to always be rescuable. Your job is to explain, kindly, that "always rescuable" is the same thing as "never conclusive," and that the whole purpose of the POC is to let both sides decide with information they did not have in discovery.
Timebox honestly
Four weeks is the number most serious POCs converge on. Shorter and you cannot install the measurement; longer and the POC becomes a product build, the organisation around it shifts, and the sponsor who signed off in January has moved to a different team by March.
- Step 1
Week 1 — install measurement
Eval set agreed and loaded, baseline numbers recorded, a thin end-to-end path running. No polish, no demo.
- Step 2
Week 2 — reach signal
Iterate the prompt, retrieval and tool surface against the eval set. If you cannot beat the baseline by a margin the customer finds meaningful by end of week 2, that is useful early evidence.
- Step 3
Week 3 — make it real
Integrate against the actual data source, not a copy. Put a real user on it, watch where it breaks, fix the three things that break most often.
- Step 4
Week 4 — decide
Run the final eval, write the one-page memo, hold the decision meeting with the named owner and the economic buyer in the room. The POC ends — either as a go, or honestly as a no.
The hardest parts of this shape are weeks one and four, which is why teams without an FDE discipline skip them. Week one is the boring work that makes the next three honest. Week four is the meeting that forces a decision, which organisations are strangely willing to avoid forever if nobody runs it.
The demo trap
A POC that wows and never ships is not a neutral outcome. It is a cost. You spent four weeks of an engineer on it, the customer spent time, the expectation it set influences how the customer talks about AI internally for the next year — and if they talk about it as "impressive but didn't go anywhere," every future conversation starts further back than this one did.
The demo trap has a signature. Signs you are falling into it:
- You keep showing it, and nobody asks when it will be live.
- The eval set never got agreed, and nobody chases you about it.
- "One more feature" has been said three times.
- The named production owner stopped attending the weekly.
- You have started polishing the UI to make the demo land harder.
Each of these individually looks fine. Collectively they are a POC that has quietly re-categorised itself as marketing. The move is not to push harder on the demo — it is to reopen the exit-criteria conversation, with the people who signed them, and either restart the clock or end the engagement.
What to take away
Scoping decides whether the POC leads to production. Insist on three commitments before building: an eval set with ground truth, a named production owner on the customer side, and a written path to production. Write down what counts as a yes and, harder, what counts as a no. Timebox to four weeks, because shorter cannot install measurement and longer dissolves the question. Watch for the demo trap — a POC that wows and never ships is a cost, not a win, and its signature is the quiet disappearance of exit criteria. A good FDE is the person who will reopen that conversation rather than ride the demo into a quarter.
Next: the FDE interview loops at the top labs — what each one tests, what they read for, and the rounds candidates most reliably fail.
Check yourself
5 questions · pass 4/5 to unlock The Interview Loops at the Top Labs
1.Which commitment matters most before you start building a POC?
2.A POC "succeeds" in the demo but the customer cannot articulate what would have to be true for it to go live. What have you built?
3.Your customer wants to timebox the POC at twelve weeks. What should you push back with?
4.What is the single hardest thing to add to a POC after week one?
5.The customer asks for one more feature before they will decide. What should you look at?
5 left to answer
Discussion
Sign in to postNo comments yet. Be the first to say something.