A CTO I worked with years ago used to tell a story about 30 monkeys in a room. Bananas are hanging from the ceiling. Every time a monkey goes for the bananas, all of them get blasted with freezing water. Eventually, they stop going for the bananas. Then you start swapping in new monkeys, one at a time. A new monkey walks in, spots the bananas, and goes for them. The others immediately stop him, even though they've never been sprayed themselves. Keep going until every original monkey is gone. Now you have a room full of monkeys that have never been sprayed, have no idea why, but absolutely will not touch the bananas.
That story was about what can happen to a company's code, processes, and business rules over time. People follow patterns without understanding where they came from, and eventually, nobody in the room knows why things are done the way they are. The pattern just gets passed down. And when that pattern has a flaw in it, nobody catches it, because nobody questions it. It quietly gets baked into every decision, every new feature, every hire that comes after. By the time the flaw causes a real problem, it's so deep in the foundation that fixing it is enormously expensive, if it's even possible.
The same thing is now happening to AI models, except the cycle is faster and the scale is much larger. Instead of a room of 30 monkeys, you have models training on models training on models, and the flawed assumptions from three generations back are now just... the way things are done. No one sprayed them with water. They just learned it that way.
What Is Model Collapse?
Right now, AI tools help write nearly half of all new software code. That's a massive productivity win, no argument there. But it has quietly introduced a serious problem called model collapse.
Here's how it works. Model A writes some code. Model B learns from that code. Model C learns from Model B. If Model A had a subtle mistake baked in, that mistake doesn't just get repeated, it gets treated as correct behavior. It becomes the foundation, and every model that comes after builds on top of it.
The Army of Juniors Problem
Part of what makes this worse is how AI writes code today. It's fast, and it's capable, but it writes code the way a talented junior developer does. It follows the rules and meets the requirements, but it lacks the architectural judgment and security instincts that come from real-world experience. When the internet fills up with that junior-level code, future AI models start treating it as the gold standard. The result is code that's bloated, harder to maintain, and full of what researchers call phantom bugs, logic added to handle imaginary scenarios that don't actually exist. Nobody knows why that code is there. Nobody questions it. The monkeys just know not to touch the bananas.
The Edge Cases Are the First to Go
The most dangerous part of model collapse is something called tail erosion. In software, the most important code is often the stuff that handles rare events, a sudden traffic spike, an unusual security threat, an edge case that only shows up once in a thousand transactions. That's not glamorous code to write, but it's what keeps systems standing when things go sideways.
AI models naturally focus on the most common patterns, because that's what most of the training data reflects. The rare stuff, the edges, gets treated as noise and fades out over time. Lose enough of that, and the digital infrastructure we all depend on becomes brittle. It works great until it doesn't, and when it fails, nobody quite understands why.
Why You Should Care
You don't have to read code to feel the effects of this. In studies of unreviewed AI-generated code, there were roughly 2.74 times as many security vulnerabilities as human-written code. We've already seen cases where AI-generated "temporary" fixes caused hours of outages because they weren't reviewed closely enough. And as models converge on the average way of doing things, software starts to feel samey, less capable of the creative solutions that actually move things forward.
The Fix: Keep a Human in the Loop
The good news is this isn't inevitable. Researchers have found that the best defense against model collapse is maintaining what they call a real-data anchor. A permanent collection of high-quality, human-written, and human-verified code that keeps the models grounded in reality. Keeping a human in the loop ensures that this happens. They can tweak and tune what the AI generates.
AI is a powerful collaborator. But a collaborator still needs oversight. If we just let it write the world and train on its own output, we end up with a room full of monkeys that won't touch the bananas, and no one is left to remember why.