Work AI Can Speed Up - And Work It Cannot

While AI Agents are causing numerous problems in the field of Software Engineering, but nothing is more harmful than the delusion it places in peoples' minds that making sweeping changes to existing applications is trivial. This perspective, growing more and more amongst the non-technical members of engineering organizations and others on the outside of the software engineering community looking in, shows a considerable ignorance in regards to what software engineers actually spend the majority of their time on. Pure code creation, what AI Agents are celebrated for the most, is by far the simplest part of the day to day work of being a software engineer.

Pure Code Creation

Adding a button to a page has been simple since the creation of HTML, and is even easier today in modern browsers:

<button onclick="fetch('https://mysite.com/some-endpoint')">Click Me</button>

One could argue this isn't really demonstrating any functionality, isn't actually doing anything with the request and response, etc. But beyond even that are considerations of the entire code base:

• What are the standard button components, classes or styles already present in the projects, and which one(s) apply to this use case?
• What considerations need be made for mobile vs desktop (if not baked into the styles and components)
• What considerations need be made for security? (CSRF, Auditing, Logging, etc...)
• How does this affect existing automated testing?
• How should this function be affected by users or tenants level configuration?

These are questions that an AI is not going to automatically take into account just because you ask it to add a button.

Greenfield Development

There is a reason why the majority of demonstrations of "AI Agents changing the landscape of software engineering" are the agents writing some service or application completely from scratch. In the industry this is referred to as "greenfield development".

Greenfield development in software engineering refers to creating a new software system from scratch, without any constraints from existing systems or legacy code. This approach allows for greater innovation and flexibility, enabling developers to implement modern technologies tailored to specific project needs.

This simple and concise definition gives us a template to work from to break down the real time cost in software engineering projects.

Working with Existing Systems

"...without any constraints from existing systems or legacy code."

Software engineering organizations, by their very definition, already have codebases they are developing, maintaining, and running. Sometimes massive codebases that have grown organically over years, with hundreds of engineers hands and voices lent to their creation. Real software engineers have to take into account this existing code base for every change they make, ensuring that new functionality, behavior, and components align with the existing system for consistency and to reduce the chance of bugs or issues later on.

Challenges of Legacy Codebases

Working with existing codebases can be challenging for modern AI Agents due to several factors:

1. Discoverability - It is not always obvious when looking purely at a specific service or repository what all is actually occurring inside of it.
2. Proximity - As codebases grow and mature, layers are built on top of previous code as new enhancements are made, separating concepts that once were local to one another into puzzle pieces that fit into a number of separate scenarios.
3. Context Windows - The hard and firm restriction to the maximum scope an AI Agent can incorporate into it's work.

Discoverability

The discoverability of features and integrations in a software engineering organization can be inconsistent. In small organizations this is often approached for engineers the same way as for an AI Agent; scouring through the limited codebase until you understand all of the moving parts and how you should utilize them piece after piece.

In more mature organizations however this information can be clouded by the organic growth of the codebase over time. There may be a handful or more of implementations that achieve the same goal, and it may not be immediately clear which of those are the "current" or "final" method to doing something. This decision may be completely external to the code, in documentation or in the comments of a staff level or higher engineer who is more responsible for the overall architecture of the org, or is simple decided by the whims of the founder's current vision. Code may be spread across a wide range of languages and frameworks of varying ages and maturities.

There may also be behavior occurring that is fully transparent to your code and the code surrounding it. Filters being automatically applied to RESTful endpoints, authentication occurring completely separately from the service itself, compiled in functionality like tracing, approaches that were taken to make a software engineer's life easier but serve to completely hide discoveries from an AI agent whose primary approach consistent of the mass consumption of the code it sees right in front of it's face.

Proximity

While proximity and discoverability are related concepts, they become more pronounced when working with AI Agents compared to software engineers.

For a Software Engineer, the proximity of something matters less than it's pure discoverability; a found variable or function reference that is not local to the current code is very obvious and only adds a minor delay in retrieval and understand of the behavior you are investigating. AI Agents on the other hand are extremely sensitive to proximity, as they tend to take in large contexts in the immediate aware of the work they are taking on naturally, and use this behavior naturally to establish their short term memory for the action they are currently taking. (This also leads AI Agents to be enforce local consistency regardless of the local vs global consistency of a project or application, but that is another topic entirely.)

The proximity factor is also the key reason why AI Agents do better when working on code written by other AI Agents, or the same Agent simply in a different session. Because of the proximity problem, it is common for AI Agents to keep all of their work for a specific feature or solution in a small proximity when it is initially created. This means that other agents coming back later to add additional functionality or to correct bugs have immediate access to all of the relevant code.

Context Windows

The critical factor of all these models and tools that puts us in this position is the Context Window. Context Window restrictions are the real measure that matters when it comes to the long term usability of AI tools; all the performance on whatever nonsense benchmarks they have out there don't matter at all if the window size is too low to encompass the scope of a larger project.

"...new software system from scratch..."

These scenarios are free of the constraints of the context window because you don't typically have enough code to make a difference. Very few are arguing against AI's use as a code generation tool to quickly bootstrap a new project; it is absolutely a powerhouse of a way to get work off the ground quickly.

As your codebase grows, or as you incorporate existing legacy code bases, the context window restriction will cause more issues. Making repository or system-wide changes, a use case that seems to lend itself very well to AI tooling, can quickly become infeasible.

To be continued...