Chapter 10: An AI-Driven Development Process
To codify this, the purpose of an effective AI-Driven development process is to utilize the implementation capabilities of agentic coding tools as efficiently as possible while maintaining compliance with project specification. In an ideal scenario, which I do think is coming, we go from a loose spec to product in one prompt, and quickly and collaboratively revise the solution from there until it is in complete alignment with project goals. That being said, at the current state of agentic capabilities we can move from a multi phase approach that tackles this.
The Three-Phase AI Development Process
PHASE 1: PROTOTYPE PHASE 2: DEV COMPED PHASE 3: REFACTOR + QA ┌────────────────────┐ ┌────────────────────┐ ┌────────────────────┐ │ │ │ │ │ │ │ 75-85% of spec │ │ Complete │ │ Audit, polish, │ │ │ │ alignment with │ │ secure, test │ │ Complex features │ │ full Spec Tree │ │ │ │ = dummy impl │──────►│ │──────►│ Code organization │ │ │ │ Spec by spec, │ │ │ │ Design + all │ │ incremental │ │ Prod readiness │ │ flows present │ │ │ │ │ │ │ │ Version control │ │ Manual + QA │ │ │ │ is key │ │ testing │ │ │ │ │ │ │ └────────────────────┘ └────────────────────┘ └────────────────────┘ Client: Weekly Client: Bi-weekly Client: Delivery sync meetings progress reports presentation AI Efficiency: Human Review: Macro Audit: MAXIMUM CRITICAL COMPREHENSIVE
Phase 1: Prototype
As mentioned earlier, a prototype sacrifices completeness for the sake of efficiency. When agentic tools are able to tackle completeness with efficiency the following phases will collapse to only this phase, and we enter into an awesome realm of all products being prototypes. Multiple prototypes can be built concurrently to determine project feasibility, product market fit etc. and the focus will continue to trend upwards back into discovery, but I digress. As of right now, a good prototype is the min / max of agentic efficiency that still meets the client goals and portrayal of project direction. A couple features that only encompass 10% of the project spec is not a prototype, in our opinion, a prototype is at least 70-85% of the project spec in which complex features are still represented but are supplemented with dummy features or data. Design and feel of the project should be represented and all user flows should also be present. The prototype should allow the client to materially understand what their end project will be with a high degree of certainty.
In terms of developing a prototype efficiently, it should be downstream from the Spec Tree. Using the Spec Tree as a master reference point for the agent, one or multiple agentic runs should be initiated that instruct the coding agent to generate a prototype specific Spec Tree in which complex features are represented with dummy implementations. A developer should review the new, prototype specific, Spec Tree to ensure that the derivative is correct. Common patterns that should be dumbed down are intensive security or compliance requirements, third party integrations, legal requirements (ToS and Privacy Policy), intensive analytic layers, data pipelines, complex branding or design requirements - any feature that the developer is not confident that the agent could implement in one agentic run. What should remain are CRUD functionality, general design guidelines, colors, aesthetics, branding and all user flows - anything that is completable in an agentic run. A simple deployment pipeline should be set up for presentation purposes. Most importantly, the prototype should be constructed in a way that can be built upon, and provides context for future agentic implementation of the complete project. It’s also important to build the prototype in a way in which sweeping revisions can be implemented quickly.
When presenting the prototype to stakeholders, expect a level of revisions, however, utilizing your definition assets and the spec tree, you can cross reference any requested changes against scope to determine if the requests should be handled as change orders and will impact timelines and cost. Most revisions at this phase will be related to design, which is where the prototype should be optimized for sweeping changes. Utilizing proper component structures, templates, and design inheritance will be key here so design revisions can be implemented quickly. One thing that we at Benmore are testing is if, at the prototyping phase, we should allow the clients to prompt and make their own changes in order to speed up the revision cycle. In a low risk prototyping phase, this could expedite the revision cycle so long as client changes are isolated from the core developer codebase. Additionally, clear expectations surrounding expanding scope is a key concern when handing over implementation capabilities to the client so they don’t “run away with the ball” here and prompt their way into a completely new product that they are in love with. Regardless, once prototyping is complete, the client is on board, the prototype doesn’t introduce any additional features - or said features are accounted for, we can move into the completeness phase of the project.
Prototype Phase Checklist
Building the Prototype:
Features to Simplify in Prototype:
| Keep (Full Implementation) | Simplify (Dummy/Placeholder) |
|---|---|
| CRUD operations | Intensive security / compliance |
| All user flows & navigation | Third-party API integrations |
| General brand / design | Legal documents (ToS, Privacy) |
| Color scheme & aesthetics | Complex analytics / dashboards |
| Core UI components | Data pipelines |
| Basic form validation | Payment processing (use test mode) |
Presenting to Client:
Phase 2: Dev Comped
The purpose of this phase is to completely bring the project into alignment with the Spec Tree. At this point, the prototype should be in alignment with the prototype Spec Tree and the developer should utilize AI to cross reference the two specs to key in on outstanding differences between the two trees. Once differences are identified, these should be documented and specified so both developer and agentic tools can begin bringing the prototype into state with the Spec Tree. Utilizing the differential document, each non-compliant spec in the tree should be incrementally tackled with success criteria being utilized as a check against a spec being complete. In this process, the developer will need to be more involved with the progression of the agentic coding tool. When specs are marked as completed by the agentic tool, the developer should verify that success criteria is actually met and other specs within the application are still functioning as intended before moving on. Proper versioning is going to be key here, as if a rogue agentic run messes up a spec, the ability to revert to a previous version is needed. At the very least, a commit should be made after each completed spec. It’s important to recall that the Spec Tree is a living document, and if the leaf nodes of the Spec Tree cannot be completed with one agentic context session, it’s likely an indication that the node needs to be split to agentically palatable specs. More information or context may be needed at the spec level as well - but the key thing to remember is that each spec should be able to be implemented and tested within one agentic context session. Ideally, it is able to be implemented and tested within one agentic run - but in a worst case one agentic context epoch.
Let’s talk about a couple of the common edge cases in this phase. Oftentimes, as mentioned, when working with third party systems, specs become impossible to complete due to lack of context. In your differential document, it’s important to ensure that these knowledge gaps are closed. Another common issue in this process is rogue agentic runs that end up modifying other parts of the solution. Here is where versioning control is important, as well as proper change visibility instructions for the agentic tool. Specifically, in the Spec, or in instruction to the agent, outlining that all changes and modifications to the solution should be presented to you after implementation. Context management is also key here. Make sure that each new Spec starts from a blank context session within the agentic tool, and if the context session is not sufficient to complete a Spec then the spec must be broken down into a consumable spec. Foregoing testing criteria is another common issue in this phase. Ensure that the agent writes tests to validate the spec completion and documents those tests in a separate testing document or in the spec tree itself. The mandate to break each spec into a separate context session can create friction in overarching agentic understanding of the project. To combat this, ensure that the agent orients itself at the beginning of each context session by investigating the spec tree and differential document. This phase is definitely more of a review oriented process, and the general attitude towards it should be one that is alike a senior developer reviewing a junior’s work. Approach it with a healthy level of skepticism and meticulously review each spec.
Dev Comped Phase Checklist
Setup:
Per-Spec Development Loop:
Common Issues & Remedies:
| Issue | Remedy |
|---|---|
| Spec can’t be completed in one context session | Break spec into smaller sub-specs |
| Agent modifies unrelated parts of codebase | Enforce change visibility instructions; revert via version control |
| Third-party integration lacks context | Close knowledge gaps in differential doc before attempting |
| Agent skips testing | Mandate tests in spec instructions |
| Macro project understanding lost between sessions | Agent reads Spec Tree at start of each session |
Ok, now when it comes to the client management here, we recommend reducing weekly syncs to bi-weekly meetings as, especially for non technical clients, there is not much to present. Progress can be given on a weekly basis in relation to the percentage of the spec tree that has been completed in a report. At this point, trust in the relationship should be established and the client, from the prototype should know what their product is going to look like and when it will be done. In our experience higher meeting frequency at this phase just gives the project a higher likelihood of being derailed or changed due to revision requests or new ideas from clients so limitation of client interaction during this phase can actually be a benefit. A bi-weekly presentation of completed specs is typically all that is needed at this phase. Rounding this out though, a successful completion of this phase is when each spec is implemented and the entire Spec Tree is in sync with the software that is built.
Phase 3: Refactors, Production Readiness, and Testing
The purpose of this phase is to essentially, audit, polish, secure and test the software at a macro level. The software should be in compliance with the spec tree at this phase, but that spec tree was implemented at spec by spec level and there are likely inconsistencies with the project as whole. Now is our time to discover and fix these issues. The first component to investigated within the codebase is proper code organization. When moving from spec to spec, as mentioned, there can be a lack of macro project understanding. Some spec level code may not be organized correctly within the code base, so it’s important to work down the Spec Tree and organize the logic throughout the code base. There should not be multiple different utilities, scattered feature code, redundant components etc. What this can turn into is a refactor, but this can be handled relatively efficiently as the mandate to the agentic tool is to maintain functionality, but reorganize. At this phase, it’s especially important to retest any success criteria as too ambitious of a refactor can mess things up. As always, versioning control plays a key role, and refactors should be split into separate agentic context windows. It may be beneficial to update the spec tree, with new refactor specs as well if a large amount of project organization is determined.
Once the codebase is organized and “cleaned”, it’s time to perform a similar exercise for production readiness. Utilizing your success criteria, scalability goals, and security/compliance requirements, use agentic tools to investigate and try to find anything that is out of spec. Start from the top of the Spec and move downwards to the feature and then atomic level. Prompt the agent to be hypercritical of the current spec and to ruthlessly check it against your production criteria. It may be helpful to audit each component of production readiness at a time. Start with scalability, then move to security, etc. Once we have performed these agentic sweeps we can move to more traditional methods of testing.
Of course, make sure that you, yourself, physically navigate through the entire application’s UI and try to break workflows. Make sure to investigate every spec works through the UI. During UI investigation, the agentic technology to do this is right around the corner, if not here already. If possible, instruct AI to perform similar testing in relation to UI testing against the Spec Tree. Once you have manually tested each spec, agentically tested each spec, and performed your codebase sweeps, we can mark the project as NQA comped. Meaning you have tested it, but we need to involve a QA team member or other developer to perform another round of manual testing against the spec tree, and another round of agentic investigation. Once another member also gives their stamp of approval, the project is marked as QA comped. The last step is consolidating everything together and presenting the client with their completed project.
Phase 3: Refactor, Production & QA Checklist
Code Organization:
Production Readiness Sweeps:
Testing Ladder:
┌───────────────────────────────────────────────────────────────────┐ │ │ │ STEP 1: Developer Manual Testing │ │ Navigate every screen. Test every workflow. Try to break it. │ │ │ ├───────────────────────────────────────────────────────────────────┤ │ │ │ STEP 2: Agentic Testing │ │ AI tests specs against Spec Tree. AI attempts edge cases. │ │ │ ├───────────────────────────────────────────────────────────────────┤ │ │ │ STEP 3: NQA Comped │ │ Developer gives stamp of approval. │ │ │ ├───────────────────────────────────────────────────────────────────┤ │ │ │ STEP 4: QA Team / Second Developer │ │ Independent manual testing against Spec Tree. │ │ Independent agentic investigation. │ │ │ ├───────────────────────────────────────────────────────────────────┤ │ │ │ STEP 5: QA Comped │ │ Project cleared for delivery. │ │ │ └───────────────────────────────────────────────────────────────────┘
When delivering the completed project to the client, it’s ok to hype it up a bit and have a celebratory tone toward the completion. However, remember that the client has been viewing progress at fairly distant intervals at this point - so it’ll be important to remind the client of the original scope of the project, the success criteria, brand kit etc. Then you present documentation on how project scope was matched, how success criteria was met, and how the application was tested. Remember that your success criteria should be independently verifiable, this is why, if the success criteria was approval based, there could be virtually objective end to the project at all. Regardless, after a project recap is delivered you can do a quick demo of the project to the client, explain to the client where to access their completed software, and start discussing support and maintenance expectations or plans for future work. At this step it’s important to take advantage of the heightened sentiment of the relationship and ask for a case study, a review, or anything else that would assist in social proof or future sales. Yay, you just completed a software development project!
Project Delivery Checklist