Should You Fix AI-Generated Code or Rebuild It?

There’s a good chance your organization is running code in your production environment right now that nobody on your engineering team wrote. AI-generated code is finding its way into codebases from every direction, and plenty of it ships without an engineer ever looking at it.

A company in the media space I’ve been helping recently contacted me about a Rails application that had grown substantially over the past six months. Their application has a custom-made administration section that serves internal team members. Tired of waiting for someone in the 4-person engineering team to have enough bandwidth to handle feature requests, the non-technical team members from departments like marketing and management learned how to use Claude Code and began developing what they needed and submitting new functionality on a weekly basis.

Given that the engineering team barely had time to work on their tasks and most of the work was not customer-facing, the Claude-generated code would get merged and shipped directly into production without as much as a second glance. By the time I got the call, two of the four engineers had spent a large chunk of their last two sprints putting out fires caused by those changes. It left the team behind on their schedule.

Their team lead wanted to know one thing: would it make more sense to continue maintaining the code added by the non-engineering team, or would it make sense to throw it away and rebuild?

Where AI-Generated Code Usually Breaks Down

When I get called in to look at an application built this way using AI assistance, I don’t start by combing through the code line by line, since it’s way too time consuming. However, there are a handful of areas that often tell me most of what I need to know about how expensive it’ll be to manage and maintain.

Check the automated test suite

One of the first places I look at when working with any new code I’m unfamiliar with is the automated test suite. A lack of a comprehensive automated test suite for anything other than a small change shows that there was little to no thought made about maintaining the codebase long term. Even if the plan was to only do manual testing, the act of having automated tests will help with understanding what the code is supposed to do. Without this, there’s a good chance that the app will not behave as expected and add to the cost of long-term maintenance.

Surprisingly, a large automated test suite is also a red flag. AI agents are amazing when it comes to building an automated test suite, creating hundreds or even thousands of test cases in a single prompt. But more tests doesn’t always equal better testing. A large chunk of AI-generated tests are slow, highly inefficient, and repetitive. The media company mentioned in this article suffered from this problem, with their test suite run times running at least three times slower, from 90 seconds to over five minutes.

As another example, one app I worked on for another client had over 3,400 automated test cases, but on closer review almost half were testing framework behavior instead of business logic, which is a waste of resources. Even worse, three of the tests were false negatives, where the test was written to make them pass while masking legitimate bugs in the code. These kinds of issues make any updates to the code problematic since the test suite is going to be a hindrance rather than helping. A test suite is supposed to be one of the levers that lets you ship quickly without sacrificing quality, and a bloated one takes that lever away.

Look for inconsistent code patterns

A telltale sign that application maintenance is going to be an issue is to pay attention to how consistent the newly-introduced code is across the board. For brand-new applications from scratch, it’s paying attention to how the code is organized into different modules, classes, or whatever the underlying programming language and framework uses. For code added to existing apps, it’s checking if the new code looks like what’s already in place (assuming the team already had guidelines in place).

There are a couple of ways to spot this when scanning the codebase. One common way this surfaces in AI-generated code is when existing functionality is re-implemented in a different way in a second spot, such as using a new library to handle authorization when the code already has a way of doing it. Another way it appears is in the form of code duplication, which can easily sneak in undetected when not paying attention. This is exactly the kind of drift that static code analysis tools are good at surfacing without needing to read every file yourself.

For instance, I recently used Claude Code to refactor a shared method that was a bit lengthy, and it did. But it also duplicated a lot of logic in each place the method is called instead of keeping it in the shared method itself. It’s easy to overlook these changes, and over time they can create an expensive maintenance burden on the team.

Measure performance under production load

Admittedly, many software engineers don’t fully consider how their code will behave in production environments. The code works well in development and even staging when there’s fewer resources used at any given time. Once the code is in the real world, using databases containing millions of records and hundreds or thousands of users on the servers at any given time, the bottlenecks start rearing their ugly head.

In my experience, LLMs are notorious for writing code that has zero consideration for performance. Even when explicitly asking the tools to consider performance, it often forgets to cover some of the more common regressions, like missing database indexes or N+1 queries. For example, an N+1 query in the media company’s admin section made it so a page needed to run 500 small queries to load everything, adding seconds to the page load. I made a single line change in the codebase to eager-load those records, leaving the page with two database queries that ran in milliseconds.

These issues can be mitigated by adding context to the prompts about what to expect in production environments, but non-technical users usually can’t articulate this properly, leading to sub-optimal code getting pushed through.

Would you know if something breaks in production?

The key to keeping applications running smoothly in production is to have visibility into what’s happening in that environment at all times. Logging, monitoring and observability all play essential roles in letting teams know how an app is behaving. More importantly, it tells them when something is going wrong and needs immediate attention. An application with sufficient logging and real-time metrics lets developers address problems quickly. Without it, you’re risking hours of downtime and unhappy users.

Logging is something else I’ve noticed current AI tooling skip frequently. I can understand since I consider logging to be somewhat subjective, where logging the wrong thing can be wasteful and too much can make it difficult to track anything. However, AI tools love to write excessive code, tests, and comments (I recently saw a pull request from the client in this story where 75% of the code changes were AI-generated comments) yet still do minimal logging unless prompted, where it can go overboard and cost you a lot of money since some services charge by the amount of logging it receives.

Applying Guardrails for Everyone Who Ships Code

Of course, non-programmers won’t have the time or the desire to take care of the code that they introduce into the organization. That’s fine, as long as an actual owner gets named before the code goes live instead of during an incident. Before this leads into problems across teams, it’s better to handle these issues early by introducing guardrails around the codebase that apply to every single person across the organization—no exceptions.

Software engineering teams often follow guidelines and rules around their projects to help keep the codebase working as smoothly as possible. The more common ones are requiring code reviews before merging code into the main branch of the repository and running automated checks through a CI service (linting code, passing the automated test suite, and so on). If you don’t have that in place yet, automating a linter into your development workflow is one of the cheapest guardrails to add, and it catches a good chunk of the consistency problems described above before a human ever reviews the change.

Anyone who wants to contribute to the technical side of things must follow the same steps before their work gets pushed forward. We shouldn’t expect non-engineers to fully understand how these systems work, but they should know the process for acceptance of new code so they’re not blindsided by a rejection due to a failing test or code that doesn’t follow the project’s standards.

Doing this can be tricky, as some people like to pull rank and ignore the process. I’ve experienced this first-hand at a different startup I worked at. The CEO knew how to code a bit, but it wasn’t his strength. One Monday the CEO arrived at the office to let us know he spent all weekend building a new interface for a section of the app. The changes broke that section’s core functionality. When we pointed it out, the CEO immediately used his position and told us to figure out the issue and fix it, which led me and another engineer to lose an entire day of our planned work. Had we gotten the CEO accustomed to our process, we wouldn’t have lost two engineering days, which are precious at an early-stage startup.

So, Should You Fix AI-Generated Code or Rebuild It?

For the media company, we kept the code. The generated code had a lot of shortcomings, such as not following all coding standards the team had settled upon. But rebuilding that admin section would have taken at least a month of engineering time that the team couldn’t spare. In addition, the small fires that needed to be put out were contained in non-public areas, so it didn’t affect any of the company’s customers.

Making this decision usually comes down to determining the blast radius of the changes. When the problems caused by AI-generated code are clustered in a handful of spots, the cleanup work becomes simple to scope and address. On the other hand, when the issues affect more than just a small segment of the codebase and can affect the long-term maintenance of the application as a whole, many times it’s cheaper to rebuild than to attempt to fix a shaky structure.

At the media company, we determined it was the first scenario. The data models for the work done for the administration section were mostly isolated from the business-critical areas, and the risk of any changes leaking into customer-facing pages was minimal. The initial work took two weeks to get to a point where everything was under control and the test suite went back to its original 90-second run times. We also made sure the cleanup could be done alongside new feature work, so the existing roadmap wasn’t affected.

Regardless of the path you choose to take, it’s worth having the “maintain vs. rebuild” conversation early. Most teams I talk to don’t take the time to assess the situation to make a valid choice. Instead, they end up keeping the code that’s already there since it’s easier to keep around, absorbing the maintenance cost for months without realizing that the alternative would have been quicker and cheaper.

What to Do If AI-Generated Code Is Already in Production

Going back to my opening story about the media company that was experiencing problems thanks to AI-generated code, the team is still allowing non-technical team members to ship changes to the application. After spending a sprint fixing the initial batch of issues, the main difference is that now all changes go through the same pull request reviews and CI checks as everyone else on the engineering side.

In addition, any new code is assigned an owner responsible for reviewing and maintaining those changes before shipping to production. I notice this part is often skipped in teams, which is what leads to not knowing who needs to manage the new code. By being clear about responsibilities, it’s kept costs down by saving at least one full-time engineer’s salary from being spent on maintenance, and it’s made everyone aware about the importance of keeping an application running in peak condition even when using AI to build things.

This is the kind of work that I take on as a Fractional Rails Engineer. I’ve been adding extra capacity to teams to help them manage with the ever-increasing workload generated by AI, helping them clean up and maintain inherited code and putting guardrails in place so that the next time someone from another department wants to build new functionality, they’ll be covered.