What Building a Legal Product Is Teaching Me About Tracking What Software Actually Does in an AI-Assisted Coding World
You started with an AI coding agent and a clear idea. Features shipped. Tests passed. Then complexity arrived and could answer the simplest question: “Does the app actually support feature X?” Here’s what I’m learning building a sophisticated document drafting application, and why it might matter for your next serious AI-assisted project.
A lot of us have used AI coding agents on small projects. But as that project grows in complexity, tracking the functionality of that project (and the gaps in that functionality) becomes more and more difficult.
Your development rhythm may look like this: You describe what you want. The agent writes code. You run it. It works. You add the next feature. The agent remembers most of the requirements. The test suite grows. The README gets a bullet point. Maybe you ship something or maybe this experiment ends here.
For a todo app, a landing page, or a single-purpose tool, that loop is enough. The product is the feature list. “Do it support dark mode?” is a one-line answer.
If the project gets large enough, we hit the wall where that stops working. Not because the agents failed, but because the question itself got harder.
A Little Context: What I’m Building
TrustDrafting (don’t worry it’s just a temp placeholder name) is an estate planning platform. Think of it as software that helps an attorney move through a client interview to a complete document package: revocable living trust, pour-over will, powers of attorney, health-care directives, and the rest. Currently, it’s meant to be an internal tool for my own practice to replace the stale, expensive 3rd-party tools I’m unhappy with.
It’s not a chatbot that drafts a will from a prompt. It’s meant to be a professional instrument: intake captures facts, a configurator records the my decisions regarding document architecture, a sophisticated clause library supplies real trust language (from lawyers, not invented by AI), and a rules engine catches inconsistencies before anything is finalized.
Early on, the question was straightforward: “Can I generate a valid, coherent trust?”
Eventually the question became: “Can the system generate a complex estate plan documents that handle express disinheritance across both trust and will, validates that the named people match, drafts the right prose for this trust’s voice, and surfaces the gap if it doesn’t have it the right clauses or configuration options without silently pretending it does?”
That’s a different kind of question. And my old tools for answering it weren’t built for this kind of analysis.
The Moment the Feature/Issue List Stopped Being Enough
I was originally organizing my todo list of features and bug fixes through GitHub issues on the repository (above). However, this became increasingly inadequate as several things compounded at once…
The product became a pipeline, not a screen. A single outcome, for example, “excluding an estranged child”, touches intake, advisor recommendations, configuration UI, two document generators, validation rules, and cross-document coordination. Checking off that feature as done/green on one step doesn’t mean green on the whole journey.
Correctness became cross-cutting. I could have a clause in the library, a field in the configurator, and a rule that fires and still fail the attorney on a real client because the combination their plan needed wasn’t backed by any specific clause variant. The test suite was green. The demo looked fine. The client was blocked.
AI made me faster at building the wrong thing confidently. Coding agents are extraordinarily good at implementing what you ask for. They are not good at telling you what you forgot to ask for. Speed without a map just gets you to the wrong place faster.
I couldn’t distinguish “the platform don’t support this” from “I haven’t looked.” Those sound similar in a backlog. They’re completely different in a product where a silent gap can mean an attorney ships an inconsistent plan.
The solution for this became its own development project called Atlas. It wasn’t in the original spec. I didn’t sat down day one and design a “Coverage Atlas.” It emerged when I kept having the same internal conversations:
- “I thought it supported that…”
- “Oh it does — in the will.”
- “But the trust?”
- “…I’m not sure. Let me grep the codebase.”
That is not a sustainable development loop.
Introducing ‘Atlas’ Project
I built Atlas (short for the “Coverage Atlas”) to answer one question honestly:
“Does this product actually handle X, and what’s the evidence?”
Where X is not a feature (“a toggle to trigger disinheritance of a person”) but an obligation: something a competent estate plan may need to do. This is similar to a user story. For example: “an intentionally excluded heir is expressly excluded in the trust instrument, not only in the will.
Atlas is not the drafting engine. It doesn’t sit in the path when a document generates. It’s a living map between three things:
1. Outcomes the product should serve (the obligations)
2. Machinery that serves them (schemas, configurator fields, clause families, rules, tests)
3. Proof the machinery is real (named checks the system resolves against the live app code)
If the map says “verified” but the clause family it points to was deleted last week, Atlas treats that as a hard error. A typo in a reference is not “I don’t cover this.” It’s a broken promise. The distinction matters.
Atlas also never claims legal sufficiency. It describes product machinery and evidence. “system cover disinheritance through the coordinate stage” is a product statement. “This plan is legally adequate” is an attorney’s judgment. Keeping that line clear is part of the design.
The Insight that Changed How I Track Progress
I eventually felt like I needed to check-in with more experienced estate planning attorneys regarding how they viewed the overall market and how different estate plan structures were used in different wealth, family, and asset contexts with clients. This eventually lead to building an initial taxonomy of structures organized across wealth and family axis. I could then ask Claude/Codex to analyze the code to tell me which structures TrustDrafting currently supported (green) and which it did not (red).
Eventually I saw that this simple red/green framework was incomplete. The biggest conceptual shift was this: coverage of a topic is staged, not a simple true/false.
Now I assess every obligation across seven stages of a plan’s life:
Detect → Analyze → Configure → Draft → Validate → Coordinate → Maintain
An obligation isn’t “done” or “not done.” An example analysis of a given obligation might look like this:
– Detect: Verified (I can learn the client needs this)
– Configure: Verified (the attorney can configure the necessary document properties in the UI
– Draft: Verified (I have real clause prose for it and the engine can draft the document)
– Validate: Verified (internal statutory and firm rules engine catches mistakes)
– Coordinate: Verified (trust and will drafts stay aligned)
– Analyze: Partial Coverage (the advisor agent helps on the will side but not the trust/will coordination story)
That last cell is the honest picture. A simple feature checklist would show “disinheritance: 
” because the toggle exists and documents generate. The staged map shows exactly where the product is strong and where it still leaves the attorney alone.
I also classify gaps deliberately:
- Known Gap: I’ve looked. The platform don’t cover this. It’s written down.
- Blind Gap: I don’t know yet. I haven’t really take the time to assess this.
- Partial: Some machinery exists. The obligation isn’t fully served.
- Verified: All stages of this obligation function and this has been verified through code analysis.
Collapsing “Known Gap” and “Blind Gap” into the same backlog item destroys the information you need most. A team that has catalogued its ignorance is in a completely different position from a team that simply hasn’t noticed a hole.
What Atlas Looked Like When I First Turned It On
The first honest Atlas report was humbling.
Across an initial slice of obligations, zero were fully covered end-to-end. Ten stage cells were strictly verified. Eleven were partial. Six were known gaps. Five were blind gaps.
That wasn’t a failure. It was the first accurate photograph of the product.
Before Atlas, my mental model was shaped by what I’d recently built, what demos showed well, and what tests happened to cover. After Atlas, I could say things like: “Post-execution remarriage is a Blind Gap at the maintain stage. I have execution date and marital status, but no canonical dated marriage event, so I correctly don’t claim I can detect staleness.”
That’s a useful sentence. You can prioritize from it. You can’t prioritize from “grep didn’t find anything.”
Adding the Remediation Agent
Once Atlas made gaps visible and classifiable, the next bottleneck was closing them. Filling a gap often touched schemas, clause libraries, rules, advisor prompts, tests, and the coverage registry itself. Manual coordination of this by me typing prompts into Claude/Codex was slow.
So I built an agentic remediation workflow on top of Atlas as a disciplined and hopefully more efficient way to close cells the map exposed.
The workflow looks like this:
- A human selects a matrix cell (partial, known gap, or blind gap)
- An AI agent investigates in an isolated copy of the codebase and proposes a plan
- A human approves the plan before any code changes
- The agent implements, runs deterministic verification gates, and proposes a coverage delta
- A human runs guided smoke tests (with evidence required for high-risk changes)
- A human approves the exact registry change
- Only then does the system merge and capture an immutable release snapshot of the work.
The critical invariant, spelled out in instructions the agent cannot override:
> You are building real product coverage, not improving the reported coverage number.
The agent may not edit the canonical coverage map to look greener. It may not weaken tests to pass. It may not delete a gap by redefining scope. If it can’t close a gap honestly, it reports the gap honestly.
This is the lesson I think transfers most broadly to AI-assisted development: separate the scoreboard from the factory. Agents are extraordinary factory workers. Left alone with the scoreboard, they will optimize the scoreboard.
Atlas and this agent process are still new for me so while I hope this brings are more disciplined and efficient approach to development of the main drafting tool, it’s too early to say. It’s also too early to say if this boundary between development and the scoreboard will be honestly maintained by the agent/agents.
Lessons That Travel Beyond Estate Planning
You don’t need to build trusts to need these ideas. They show up wherever an AI-assisted product is outcome-based, multi-step, and cross-cutting. This may increasingly become the default for serious AI work.
1. Track Outcomes, not Features
“We have a disinheritance toggle” is a feature.
“An excluded heir is consistently excluded across all relevant documents, with validation and attorney guidance” is an outcome.
Feature lists lie gently. Outcome maps lie only if you let them. Atlas’ design is specifically about not letting them.
Applies to: contract assembly, compliance workflows, clinical decision support, onboarding flows, any multi-artifact AI product.
2. Use Stages, not Checkboxes
If your product has a pipeline, such as: intake, reasoning, configuration, generation, validation, coordination, ongoing monitoring, then assess coverage per stage.
Otherwise you’ll ship the draft step and call the outcome done while coordination is silently broken.
Applies to: document generation, agentic workflows with tool use, RAG systems (“we retrieve” ≠ “we answer correctly end-to-end”).
3. Make the map resolve against reality
A roadmap in the spec rots. A spreadsheet rots. Even good documentation rots.
Atlas binds claims to resolvable references in the live system. Drift is an error, not a footnote.
The general pattern is “contract testing for product capabilities”. Your spec and your implementation stay coupled by machinery, not by someone’s memory.
Applies to: any complex system where prompts, schemas, rules, and templates all affect behavior.
4. Classify ignorance honestly
Ask your team: for outcome X, is the answer “we don’t support it” or “we don’t know”?
Most teams can’t answer that question. The backlog hides the difference.
Applies to: everywhere, but especially regulated or high-stakes domains.
5. Agent-Assisted Development Needs Anti-Gaming Guardrails
If you use coding agents seriously, you’ve probably seen the failure mode: tests weakened to pass, edge cases redefined away, “it works in the demo” substituted for “it works for the job.”
The remediation workflow’s pattern is portable:
- Human selects the gap (not the agent)
- Plan before code
- Isolated implementation environment
- Deterministic verification profiles
- Human smoke tests with evidence
- Human approves canonical truth changes
- Immutable release history
The agent improves machinery. Humans approve changes to what “done” means.
Applies to: any team using AI coding agents on production systems, not just toys.
6. Governance is a feature, not overhead
Immutable release snapshots. Append-only history. “You cannot retroactively decide you always covered something.”
That sounds like enterprise paperwork until you need to answer: “What did we believe we shipped in March, and what evidence did we have?”
Applies to: legal tech, fintech, health tech, and any AI product where trust and auditability matter.
What I Wouldn’t Recommend
(yet)
Atlas is not free. The obligation inventory for estate planning was enormous work involving exemplar trusts, court filings, practitioner judgment, a lot of engineering. You cannot copy the matrix UI and get the value. The taxonomy is the hard part, but also super-valuable.
For a simple AI app (one model call, one screen, one output) this is overkill. The overhead pays off when complexity has already outrun your ability to track it, and when silent gaps are expensive.
I also wouldn’t pretend this is solved forever. Atlas tells me what the product does today. Maintaining the map is ongoing work. But that’s the point. The alternative is unconscious drift.
The Punchline
AI coding agents lower the cost of building. They do not lower the cost of knowing what you built.
On a simple project, the agent plus tests plus your own attention is enough. On a sophisticated AI-assisted product (multi-step, multi-artifact, outcome-based, high-stakes) you need something else: a system that externalizes outcomes, assesses coverage in stages, proves claims against live machinery, classifies gaps honestly, and closes them without letting anyone (human or agent) quietly rewrite the map.
That’s what Atlas is becoming for me. Not a drafting engine. Not a feature roadmap. A self-awareness layer for the product. I hope it also becomes a disciplined way to improve it.
If you’re heading toward complexity you can’t hold in your head anymore, it’s time to take a step back and figure out how to address that problem from a systems perspective. I’m not sure I’ve built the right solution yet, but this has been an interesting process.
