The DIY Context Trap: Why You Can't Build Your Own Codebase Context Infrastructure

Part 4 of 5 · Previous: Why are You Still Putting Codebase Context in Markdown Files? · Next: Your AI Coding Tools are More Capable Than Your Process

The previous post made the case that search is the wrong paradigm for codebase context. What agents need is compiled, structural understanding, the kind that comes from analyzing the code ahead of time, exhaustively and deterministically.

So the natural follow-up is: we’ll build that ourselves.

We hear this regularly. A platform team or a senior engineer gets excited about the problem, spends a few weeks building a proof of concept. Single repo, single language, hooked up to an LLM. The agent generates descriptions of the code, produces some documentation. The output is reasonable. The demo looks great.

It looks like the prototype works.

What the Prototype Doesn’t Show You

The prototype handles one repo in one language. Production needs all repos, all languages, every commit, every branch, every merge. This is where teams discover what the problem actually is.

Even if your organization is primarily one language — you’re a Java shop, you write everything in TypeScript, whatever. That won’t be 100% of all of your software. You’ll have at least a handful if not a dozen languages that really make up the system. Infrastructure is code. Backends in one language, frontends in another. Configuration in YAML. Build scripts in Bash. Terraform. What we see is most teams have heterogeneous code, and they don’t realize the scope of it until they try to build a system that has to handle all of it.

Now you have to resolve the language-specific challenges. Each language has its own uniqueness. In Ruby, you have metaprogramming. In C++, namespaces. In C, compile-time flags. TypeScript 3.x is different from TypeScript 5.x. Python 2.7 and Python 3.12 are basically different languages wearing the same name. COBOL ‘85 versus ‘89. Thirty years of C spec.

Each language in the long tail takes as much work as the primary language but covers a fraction of the codebase. What we had to build at Driver is a language-specific front end for each one, and we solve real insidious problems in those front ends before we ever go to working with large language models. Most teams discover this almost as soon as they try to start building out automated context. You have to be very knowledgeable about the structures of the languages you’re working with.

The Graph Problem

Then you want to do smart things. Follow call chains — who are the callers, who are the callees. Understand the connected relationships between the software.

The syntax tree, if you produce the complete syntax tree from a codebase, is a very complicated object. It’s not a directed acyclic graph. It has all kinds of cycles in it. You’re going to end up in loops as you try to follow the way the codebase actually works. You need to simplify these things to the well-connected sub-graphs before you can really start working with these graphical objects. Then you have to figure out: what are we actually trying to know about what this software does? What’s important to describe?

This isn’t a linear project. It’s a compounding one. Each layer of understanding depends on every layer beneath it working correctly. Call graph analysis depends on symbol resolution. Cross-repo synthesis depends on call graph analysis. Architectural understanding depends on all of it. If your symbol resolution is wrong, everything downstream is wrong too.

Reliability at Scale

When your context system produces wrong output, the agent produces wrong code. At small scale, engineers catch errors in review. At enterprise scale — millions of inference calls per day — you need many-nines reliability. The edge cases that only appear in production. The language constructs that are technically valid but that your parser has never encountered.

Each reliability failure erodes trust in the entire system, and trust is hard to rebuild. Aaron, our head of sales, puts it simply: every customer who’s tried to build this internally reaches the same conclusion. It’s too hard.

We’ve been building Driver’s transpiler for over two years with a team that thinks about nothing else. Language-specific parsers across 30+ years of language specifications. Error handling for edge cases that search-based systems never encounter because they never process deeply enough to find them. And features built on this foundation compound — they enable capabilities that are impossible to build on a retrieval foundation. Symbol-complete documentation. Architecture-aware navigation. Change impact analysis across repo boundaries. Each feature strengthens the foundation and widens the gap.

The Real Cost

The direct cost of building this internally is significant. But honestly, it’s not the real cost.

The real cost is what those engineers could have been building instead. Your best engineers, the ones with the deepest codebase understanding and the most expertise in language tooling, spending their time on internal infrastructure rather than the product your customers pay for. And the system they build will never be done. Ongoing maintenance proportional to the rate of change in your codebase and the number of languages it supports.

When a customer connects their repos to Driver, setup takes minutes per developer. The transpiler processes the codebase and generates symbol-complete context — the initial compilation can take hours for large codebases, but that cost is paid once. After that, updates happen quickly and automatically on every commit. Languages are already supported. Edge cases are already handled.

The Pattern

If you’re at the point where you’re considering building codebase context infrastructure internally, I’d genuinely recommend talking to teams that have tried it first. The story is remarkably consistent. The prototype works. Production is a different problem entirely. And the engineers who end up maintaining it are always the ones you can least afford to pull off product work.

Context infrastructure is infrastructure. It should work the way version control works, the way CI/CD works. You adopt it, you configure it, and it runs. Building it from scratch is solving a problem that’s already been solved, at the expense of the problems only your team can solve.