We Thought We Knew What Good Looked Like. Then AI Started Writing the Code.
Experimenting, Evolving, Never Settling
Six years ago I joined the Payments team. Coming from a traditional QA background, it was a different world — CI/CD pipelines, fast delivery, metrics I'd never heard of. But I got stuck in and started thinking about how to bring quality into that space properly.
The first thing I pushed for was shifting quality left. I introduced Three Amigos sessions with the product owners — getting QA, dev, and product aligned before any work kicked off. It changed how the team thought about quality, and it stuck.
From there we just kept evolving. DORA metrics, elite-level delivery, zero-touch releases, code going straight from a developer's local environment to live with no manual gates. Just the pipeline doing its thing.
Through all of it, one thing never changed — quality was always built in, never bolted on. Risk was always assessed and standards were always maintained.
And now? We're exploring spec-driven development. Same team, same mindset, always learning, always trying something new.
So, What Is Spec-Driven Development?
Before any code gets written, the team puts together a detailed specification — business requirements, test cases and acceptance criteria. Tools like SpecKit or Repomix, used alongside Claude Code, take that specification and drive the entire implementation.
Here's how it works in practice using SpecKit, which follows three key stages:
📋 Specification
You pass your requirements into SpecKit and it transforms them into a much more detailed and refined specification document. It takes your raw inputs and shapes them into a clean, well-structured blueprint that the AI can work from confidently.
🗺️ Plan
From that specification, SpecKit generates a technical implementation plan. This is where you define your tech stack, architecture choices, and technical standards — for example, specifying TDD, writing tests before code, or any other technical approaches your team follows. SpecKit produces a structured, milestone-driven roadmap of everything that needs to be built.
✅ Tasks
Finally, SpecKit breaks that plan down into smaller, manageable tasks. Rather than generating everything at once, it works through the implementation piece by piece. You can see exactly what the AI is doing at each step, review the changes it's making, and course correct if needed. Small tasks, steady progress, full visibility. That's the model.
Where QA Fits — And Why Domain Knowledge Still Matters
AI can generate test cases, and sometimes they're solid. But for complex, high-risk, or compliance-heavy releases, that's not enough on its own. Domain knowledge — understanding the regulations, the edge cases, the things that have bitten the team before — that's not something you can fully prompt your way to.
QA needs to be at the table when the spec is written, making sure quality is designed in from the start, not reviewed in at the end.
In practice, this is already making a difference for us. Because of my deep domain understanding of our systems, I've been supplying test cases directly to the developers to include in the spec. Those test cases then feed into what the AI uses to drive its implementation — and the impact on test coverage has been significant.
The AI isn't just generating generic tests; it's working from real, domain-specific scenarios that reflect how our systems actually behave. That's the difference between coverage that looks good on paper and coverage that actually gives you confidence.
Spec-driven development doesn't remove QA from the process, it positions us right at the beginning of it, where our input matters most. We're not the whole picture, but we're an important piece of it.
How Has It Actually Gone?
🚀 For simpler, well-scoped work — the results have been impressive.
The vast majority of what was generated was correct and working, with very little needing to be corrected or reworked. The AI delivers. And it delivers quickly. Testing live hidden gave us the confidence we needed, and we were able to enable those features to all users faster than ever before.
Where things get more challenging is when you're working with larger, more complex services. When the requirements are complex, the spec has to be really detailed and precise. Without that level of clarity, the AI can start making assumptions that don't fit, or confuse itself within the complexity of the codebase.
This is where more validation on our side becomes important — to catch anything that hasn't landed quite right and make sure the outcome is what we intended. A future step for us is to look into modularising our largest services — breaking them down into cleaner, more defined units so that spec-driven development becomes easier to apply.
Shifting How We Test, Without Compromising Quality
Here's the uncomfortable truth. Code is shipping faster than ever, and if QA is sitting at the end of the pipeline waiting to sign off before anything goes live, we're the ones slowing things down — and that defeats the purpose of everything above.
The harder shift isn't the tooling — it's letting go of the old playbook:
- Exhaustive manual testing on environments
- Automated end-to-end regression packs that can be flaky and slow down release cycles
- Waiting for everything to be perfect before a release
With spec-driven development, AI generates extensive test coverage within the codebase — following the test pyramid approach, covering unit, integration, and functional tests — guided by the test cases QA provides upfront. That gives us a strong foundation of confidence without the overhead of exhaustive manual validation. What makes more sense on top of that is a small, focused set of automated end-to-end tests that cover your critical and compliance flows — fast, reliable, and targeted at what actually matters.
Here's how we approached our first spec-driven release:
- Ran through key business scenarios on staging
- Did some exploratory testing
- Moved straight to testing live hidden — only internal team members and stakeholders could see the feature
- Enabled for a small set of users once we were happy
- Monitored closely, then rolled out to all members
We could do this because the test coverage within the code was strong, and going live hidden meant we were protected.
Feature switches give you control over who sees what and when. And looking ahead, canary releases and automated rollback are the natural next step — gradually shifting traffic to new releases and automatically rolling back if failure rates spike. That's where we're heading.
This isn't about doing less — it's about doing the right things, in the right places, at the right time.
Live Monitoring: The Foundation of All of This
Since we started testing closer to live, observability became non-negotiable for us. Real-time dashboards, alerting, and visibility into how new releases are behaving — that's our safety net.
Monitoring and alerting was already part of how we operated, but with AI now writing our code, it becomes even more critical. When something goes wrong, we want to know in minutes. That's what makes everything else in this model actually work.
⚠️ For a team already operating at elite delivery performance, this isn't a nice-to-have. It's our last line of defence — and we treat it that way.
Final Thoughts
Spec-driven development has been one of the most fascinating experiences of my career so far. The idea that you can provide a well-defined specification and have AI write the code — and write it well — is something I'm still wrapping my head around. And the honest truth is, we're still early in this journey. There's a lot more to learn, and it's only going to keep evolving from here.
But beyond the technology, what I'm most grateful for is the team I get to do this with. A team that never stands still, never gets comfortable, and always finds a way to learn and grow together. Being given the opportunity to explore something like this — and to contribute to it meaningfully as a QA — is something I genuinely value.
If there's one thing I hope you take away from this article, it's that the specification is everything. Get that right, involve your QAs early, and you'd be surprised just how far AI can take you.
At the end of the day, this is just the nature of our industry. It never stops changing — and neither do we.