The Understanding Gap: Why AI Modernization Still Stalls on Legacy Code

TL;DR: AI coding agents have made legacy modernization dramatically faster — teams now reverse-engineer and rebuild decades-old applications in weeks instead of years. But look closely at the fast wins and the same pattern appears every time: most of the effort went into understanding the old system, not writing the new one. Extracting business logic, feeding agents guardrails, validating rules with domain experts, catching edge cases the model missed, and managing prompts before context windows degraded. The coding was never the bottleneck. Understanding was. This post argues that understanding legacy behavior is the durable constraint on AI modernization — and that treating it as persistent infrastructure, rather than a one-time manual push, is what separates a lucky project from a repeatable program.

The wins look effortless. They aren't.

There is a genre of modernization story making the rounds right now, and it goes like this. A team points an AI coding agent at a twenty-year-old, undocumented, business-critical application. Five weeks later they have a modern, containerized rebuild running on a supported stack, security vulnerabilities down by orders of magnitude, and a maintenance bill that just got a lot smaller. Vendors had quoted a year. The team did it in a month.

These stories are real, and they matter. They put to rest the question of whether AI agents can do serious modernization work on serious systems. They can.

But if you are a modernization or engineering leader deciding how to run your next project — not reading a case study, but planning the actual work — the headline number is the least useful part of the story. The useful part is buried in the footnotes: the cautions, the "here's what we had to do to make it work," the quiet admission that the first AI-generated build had all the right parts wired together wrong.

That is where the real lesson lives. And the lesson is not about coding speed.

There is also a scale caveat hiding in plain sight. The applications in these stories are small by enterprise standards — on the order of 100,000 lines of code or less. That is a size two developers can very nearly hold in their heads, which is exactly why the manual approach works: there's little enough system that a couple of people can reconstruct its behavior and check the agent by hand. Real enterprise estates run to millions of lines, and the largest run past a hundred million. The hand-built understanding process that succeeds at 100,000 lines doesn't survive two more orders of magnitude. There aren't enough experts, enough hours, or enough context window to do it by hand at that size.

What the fast projects actually spent their time on

Read past the timeline and a consistent shape emerges in every successful AI modernization effort. The work that made it succeed happened before and around the code generation, not during it:

  • Extracting domain context up front. Someone — human or agent — had to read the legacy code and reconstruct what it actually did: the business rules, the data flows, the dependencies, the edge cases nobody documented in 2004.
  • Feeding the agent guardrails. Standardized frameworks, target-architecture constraints, and explicit guidelines, so the agent rebuilt the system correctly instead of faithfully reproducing the old system's flaws.
  • Validating business rules with people who knew them. AI agents miss or invent edge cases, especially in undocumented systems. Every extracted rule had to be checked against a domain expert before anyone trusted it.
  • Managing the context window by hand. Long agent sessions exhaust token budgets and lose the thread. Developers had to break analysis into pieces, watch for degradation, and step in when the agent drifted.
  • Running the old and new systems in parallel for weeks to confirm the rebuild behaved like the original before cutting over.

Notice what all of that is. It is not writing code. It is establishing and verifying an understanding of the system — and then keeping the agent tethered to that understanding so it doesn't wander.

The code generation, the part everyone points to, was fast precisely because the understanding had already been assembled. Strip that away and you don't get a five-week rebuild. You get a confident agent producing plausible code against a system it fundamentally does not comprehend.

Speed cuts both ways

There's a second edge to that speed, and it's the one that bites. Point an ungrounded agent at a system it doesn't understand and it doesn't stall — it confidently produces a large volume of plausible, wrong code. In the fast case studies, the first AI-generated build routinely arrived with all the right components wired together incorrectly: the pieces were present, but the application didn't flow, and developers had to make cleanup passes before business users could even begin to test it.

The same dynamic shows up on greenfield work, where agents can generate sprawling new code faster than any team can understand what was just created — leaving organizations with piles of code nobody has a mental model of. Volume was never the constraint. Correctness you can trust is. Every line of confident-but-wrong output is rework waiting to happen, and rework is exactly where grounded context earns its keep: in controlled testing, agents querying a verified model produced 73% less of it.

Why coding agents hit a wall on legacy — and it isn't the model

It is tempting to file this under "the models will get better." They will. But the wall coding agents hit on large legacy systems is structural, not a matter of raw capability.

A coding agent working on a legacy estate faces four problems at once. It struggles to navigate past system boundaries, because real enterprise applications span multiple disconnected systems and follow business rules into places the agent can't reach. It can't hold enough of the codebase in view, because context windows will never be large enough to trace a complicated flow through millions of lines of spaghetti. It doesn't recognize the frameworks — COBOL, RPG, AS/400, PowerBuilder, sprawling .NET and C# — because they don't look like the modern, web-scraped code the models trained on. And it has no reliable way to know whether the conclusion it just drew about a business rule is actually correct.

None of those are fixed by a bigger model. They are fixed by giving the agent something it currently lacks: a complete, verified, external representation of how the application actually behaves — one that doesn't have to fit inside a context window, doesn't stop at a system boundary, and comes with proof.

A comparison showing an agent failing when using its context window and succeeding when using a persistent context layer
When operating within its own context window, an agent will strain to recall facts over a growing corpus. By delegating those facts to a persistent context layer, the agent retrieves facts more easily and frees up reasoning capacity.

Understanding is the bottleneck — and it evaporates

Here is the part the fast case studies rarely dwell on: the understanding they worked so hard to build was mostly disposable.

The domain context lived in the developers' heads and in a pile of prompts. The validated business rules were confirmed once, verbally, in a conversation with a business user. The mental model of the system existed for the duration of the project and then dissolved into the new codebase. When the next application comes up — and in a real estate there are dozens — the team starts the understanding work over from zero.

This is the trap. AI has made the coding phase of modernization cheap and fast. It has done almost nothing for the understanding phase, which is where the time, the risk, and the cost actually concentrate. And because that understanding is rebuilt by hand each time and never persisted, the speed doesn't compound. Each project is its own heroic effort.

For a one-off rebuild, that's tolerable. For a modernization program across a large estate, it's the thing that quietly kills the initiative.

Treating understanding as infrastructure

CoreStory exists to close exactly this gap. It is not a coding agent — it doesn't compete with Claude Code, Copilot, or Codex. It is a context layer that sits underneath them.

It was built for exactly the scale where the manual approach breaks — codebases north of 100,000 lines and up into the millions, with the largest customers running on the order of 100 million lines of code at once.

CoreStory reverse-engineers an application, from source and from its running environment, into a persistent, high-fidelity model of how the system actually behaves. Under the hood that model is a normalized graph representation — informally, a digital twin — that captures every layer of the application: the view layer, the data structures and database, the architecture and deployment patterns, the underlying business logic, the inferred user types, the data flows, and the calls out to third-party systems and dependencies.

Three things make that representation useful rather than just large:

It goes where coding agents can't. Instead of grepping and searching a single repo, CoreStory uses what we call polymorphic agents — agents that pick the right tool to understand each part of a system, follow execution traces across system boundaries, and reverse-engineer artifacts like running screens even when source code isn't available. On large estates this typically covers around 98% of the application; the exceptions are genuinely closed artifacts like binaries.

It's normalized against a real ontology. The behavior of every application is mapped into a standard, industry-independent ontology, then cast into the customer's own language. That normalization is what lets a coding agent talk to CoreStory the way two engineers talk. The agent asks, in plain business terms, "how do I change the behavior for beneficiaries aged 50 and older when they hit one of these life events?" — and CoreStory answers with exactly where and how to make the change, plus a pointer back into the code.

It's verified, and it persists. Every business rule CoreStory asserts is checked by independent agents that follow an audited trail from the specification back into the underlying code. Every answer carries a pointer to its source, so it can be confirmed rather than trusted. And the model stays live — it updates on each commit, so understanding accrues over time instead of evaporating at the end of a project.

The canonical way a coding agent uses all this is over MCP. The two agents pair-program: the coding agent asks how to resolve a ticket, CoreStory returns the impact analysis, the exact locations to change, and the acceptance criteria to test against. Human specs and documents can be exported too — some regulated customers need them — but the machine-to-machine path is where the leverage is.

What changes when the agent is grounded

When a coding agent stops guessing and starts querying a verified model of the system, the numbers move in the same direction every time.

In joint research with Microsoft and GitHub's research arm, coding agents using CoreStory's context showed a 51% accuracy uplift on software-engineering benchmark tasks — with a ~70% average reduction in token spend and roughly 2x faster issue resolution.

The token reduction is the counterintuitive one, so it's worth explaining. Agents don't burn tokens because the work is hard; they burn tokens searching down dead-end trails, re-crawling the application, and rebuilding a mental index inside a context window that can't hold it. Give the agent the right answer and the acceptance criteria in one or two turns, and all that wasted exploration disappears. Lower spend, better output — at the same time.

The quality gains show up just as clearly in controlled testing. On a public benchmark against an open-source Java monolith, running the same agent, model, and prompt scaffold with and without CoreStory's context, the grounded runs delivered:

Controlled benchmark
Metric Improvement with CoreStory context
Time to validated change (wall-clock) 58%faster
Hallucinations 81%fewer
Rework required by the agent 73%less

Same agent, model, and prompt scaffold run with and without CoreStory context against an open-source Java monolith. Results held within a few percent across every agent tested.

These held within a few percent across every agent tested. The effect isn't tied to one vendor's model — it's what happens when any capable agent is given grounded, verified context instead of being asked to reconstruct it on the fly.

"Can't we just do it ourselves with Claude Code?"

This is the fair question, and the honest answer is: yes, for one application, if you assemble the understanding by hand — which is exactly what the fast case studies did. Smart people, a capable coding agent, weeks of extracting context, writing guardrails, validating rules, and babysitting the context window.

The problem is what that costs across an estate. You are signing up to rebuild the understanding from scratch for every application, to maintain the entire prompt-and-guardrail stack around your agents yourself, and to keep skilled people on the loop validating rules the agent can neither reach nor verify. The build-versus-buy decision was never really about whether the coding agent is good enough. It's about whether understanding should be a repeated manual heroic or a persistent, verified layer your whole organization can build on — one that already handles the frameworks legacy estates are actually made of, validated across 40+ of them dating back to COBOL and RPG.

The teams getting outsized results — one insurer reported roughly a 30x return modernizing an 11-million-line COBOL estate with CoreStory and a team of coding agents — aren't winning because their coding agents are faster. They're winning because their agents finally understand what they're working on, and that understanding doesn't disappear when the project ends.

Where this leaves modernization leaders

AI has quietly moved the constraint. The coding phase of modernization is no longer the hard part. Understanding the system you're modernizing — accurately, verifiably, and in a form your agents can actually consume — is. The fast wins you've been reading about didn't skip that work. They did it manually, once, and threw it away.

The opportunity is to stop throwing it away. Treat understanding as infrastructure, keep it grounded in the code and current with every commit, and the speed you saw in a single lucky project becomes something you can run across the whole estate.

Ready to close your understanding gap?

If you're planning a modernization program and don't want to rebuild the understanding by hand for every application, talk to an expert about putting a persistent context layer underneath your coding agents.

FAQ

Is CoreStory a coding agent that replaces Claude Code, Copilot, or Codex?

No. CoreStory is a context layer, not a coding agent. It works alongside the coding agents you already use, feeding them verified context about your systems over MCP. The two agents pair-program: yours asks how to resolve a ticket, CoreStory tells it exactly where and how, with the tests to prove it.

How is this different from feeding the agent AGENTS.md or skill files?

Those files provide local, static context and are still bounded by the agent's context window — and someone has to write and maintain a growing pile of them. CoreStory maintains a complete, verified graph of the system underneath and serves the right slice of it on demand. It can generate and maintain the interface files for you, but the value is in the grounded model, not the files.

How does CoreStory avoid the hallucinated business rules that plague AI on undocumented code?

Every rule it asserts is checked by independent verification agents that trace an audited path from the specification back into the source code, and every answer carries a pointer to that source. Nothing has to be trusted on faith — it can be confirmed. That's why grounded agents in testing showed 81% fewer hallucinations.

Does it work on the old frameworks our estate is actually built on?

Yes. Understanding has been validated across 40+ frameworks, including COBOL, RPG, AS/400, and PowerBuilder, alongside modern languages. Coverage on large estates typically reaches around 98%, with fully closed artifacts like binaries as the main exception.

Does the understanding stay current as we keep shipping?

Yes. The model is persistent and updates on each commit, so it evolves with the application instead of going stale the moment a project ends. That's the point — understanding compounds instead of evaporating.

CoreStory
CoreStory Editorial Team