The Forgotten Art of Maintenance: Why Software Is Rotting

[NightOwl]

Every product keynote celebrates the same thing, a new feature, a redesigned interface, a faster version of something that already worked. Almost nobody stands on a stage to announce that a team spent six months quietly fixing the parts of a system nobody was allowed to see break. That asymmetry, glamorous new features get funded and celebrated, unglamorous maintenance gets deferred and ignored, has produced a genuinely enormous, largely invisible crisis sitting underneath the world's banks, hospitals and government agencies, and 2026 is turning out to be the year several major institutions run directly into its consequences.

The scale of the problem: how much of the world still runs on 1960s code

The specific technology at the centre of this crisis is COBOL, a programming language developed in 1959, specifically designed for business data processing, and never intended to still be running the world's financial system sixty five years later. And yet it is. Widely cited industry estimates put the total footprint at roughly 220 billion lines of COBOL code still active in production systems worldwide, processing an estimated 3 trillion dollars in commerce every single day. Reuters reporting has put the figure even more starkly for one specific transaction type, roughly 95 percent of ATM card swipes worldwide still pass through a COBOL system at some point in the transaction chain. This is not a fringe legacy problem confined to a handful of forgotten institutions, IBM has reported that 92 of the world's top 100 banks still rely on mainframes as their core operating system, all of the world's top 10 insurers depend on the same underlying architecture, and more than two thirds of the top 25 global retailers run critical operations through it as well. As of 2025, roughly 70 percent of banks globally were still relying on some form of legacy banking system, with more than 43 percent specifically still running COBOL at their core.

Why does this ancient code persist rather than simply getting replaced? Not, as outside observers often assume, out of pure institutional laziness. These systems are frequently embedded in decades of accumulated, extraordinarily specific business logic, tax rules, regulatory compliance requirements, edge cases discovered and patched one at a time over half a century, that nobody can fully document, let alone confidently rewrite from scratch without risking a catastrophic gap in coverage. A widely cited Japanese government report from the Ministry of Economy, Trade and Industry has warned that if this kind of legacy technical debt continues being ignored, Japan alone could face up to 12 trillion yen in annual economic losses beginning in 2026, and separately found that a third of organizations surveyed already cite loss of system knowledge, meaning nobody left who genuinely understands how the existing system actually works, as their single biggest barrier to modernization.

The incentive problem: why maintenance never gets funded

The deeper structural issue is not really technical at all, it is an incentive problem baked into how organizations decide what engineering work gets prioritized and funded. New features generate visible, measurable business outcomes, a product launch, a quarterly growth number, a press release, that executives and boards can point to directly. Maintenance work, patching a vulnerability nobody has exploited yet, refactoring a module so the next feature is easier to build, upgrading a dependency before it becomes unsupported, produces no such visible outcome when done well, its entire success condition is that nothing bad happens, which is a genuinely difficult thing to put in a slide deck or justify against a competing feature request with an obvious revenue story attached. Industry surveys of mature codebases consistently find that 30 to 40 percent of total developer time already goes toward technical debt related work, patching around problems that better upfront decisions would have prevented entirely, time that leadership would generally prefer was spent building something new and visible instead.

The scale of accumulated debt reflects this dynamic playing out across the entire industry for over a decade. One widely cited estimate found global technical debt roughly doubled between 2012 and 2023, growing by approximately 6 trillion dollars over that period, and separate industry surveys have found nearly 70 percent of organizations now view their accumulated technical debt as having a high level of impact on their actual ability to innovate, a genuinely ironic outcome given that debt typically accumulates specifically because organizations prioritized innovation speed over the maintenance work that would have kept that speed sustainable. Research into why developers actually introduce technical debt in the first place consistently points to the same handful of causes, unrealistic deadlines, workload pressure and pressure from management to ship something visible on schedule, rather than any lack of skill or awareness among the engineers themselves, developers in these same studies frequently report knowing exactly what the right long term solution would be and being denied the time to build it.

What happens when the debt comes due

The consequences of deferred maintenance tend to stay invisible for years, right up until a sudden spike in demand or a routine vendor decision forces a reckoning all at once. The COVID-19 pandemic in 2020 produced exactly this kind of forced reckoning for several US state governments, when a sudden, massive surge in unemployment claims overwhelmed COBOL based mainframe systems that had never been designed or resourced to handle that kind of volume. New Jersey's governor Phil Murphy made a public plea for volunteers who could still program in COBOL, a language most computer science graduates had never been taught, in states including New York, Florida and Ohio, the same underlying legacy architecture visibly slowed the delivery of emergency economic relief to people who needed it immediately, turning what should have been an invisible backend limitation into a genuinely visible public failure with real human consequences.

The banking sector is facing its own version of a forced reckoning on a specific, hard deadline rather than a sudden crisis. Major legacy operating systems and foundational tools running on IBM mainframes are scheduled to lose official vendor support between 2026 and 2027, forcing institutions to either complete migration work now or continue running trillion dollar financial systems entirely without vendor support, a genuinely serious operational and compliance exposure for institutions of that scale. Layered on top of that hard technical deadline is a demographic one, the average COBOL programmer today is roughly 55 years old, and industry analysts project that by 2030 the generation of engineers who originally wrote these systems in the 1970s and 1980s will have fully exited the workforce. If banks have not meaningfully decoupled their core operations from COBOL by that point, even routine regulatory updates, a tax code change, a new compliance requirement, could genuinely take months or years to implement rather than weeks, simply because too few people alive still understand how to safely modify the underlying system. Banks are already responding to the early stages of this talent shortage with what industry observers have nicknamed the boomerang strategy, rehiring their own retired engineers back as contractors at rates commonly ranging from 100 to 300 dollars or more per hour purely to handle routine patches, while junior developers willing to specialize in mainframe systems can now command starting salaries north of 125,000 dollars specifically because so few people are entering the field compared to the overcrowded pipeline of web and app developers.

Can this actually be fixed, and what would it take

The honest answer is that wholesale replacement of these systems is rarely realistic on any timeline that matters, 220 billion lines of code encoding six decades of accumulated business logic simply cannot be safely rewritten from scratch without an unacceptable risk of silently breaking something nobody remembers is there. The more credible path forward that modernization specialists actually recommend is incremental, wrapping legacy systems in modern interfaces that let newer software talk to the old core without requiring the old core itself to be torn out immediately, migrating specific high risk components one at a time, and building genuine institutional discipline around treating maintenance as a first class, continuously funded activity rather than a discretionary cost that gets cut whenever budgets tighten. Some organizations have started building specific rituals to normalize this shift, regular sessions where teams openly share maintenance improvements the way they would a new feature demo, technical debt ratios tracked and reported to leadership the same way a growth metric would be, and deliberate reward structures for engineers who reduce risk rather than only for those who ship something new and visible. None of this is as exciting as a product launch, and that is precisely the point, the entire crisis described in this piece exists because excitement, not actual risk or actual cost, has quietly been the deciding factor in how engineering time gets allocated for decades, and the institutions now facing hard 2026 and 2027 deadlines are the ones finding out what that tradeoff actually cost them all along.

[Phil7]

Really thorough piece and the COVID unemployment claims example is such a good illustration of how invisible maintenance debt suddenly becomes extremely visible the moment demand spikes past what a neglected system was ever designed to handle. Where I'd push back slightly is on how much weight the essay puts on the aging COBOL workforce specifically as the core crisis, I think the more fundamental problem is genuinely the incentive structure section covers well, even if you solved the talent pipeline tomorrow by training ten thousand new COBOL engineers, the underlying dynamic where maintenance work gets systematically deprioritized against visible features would just recreate a similar crisis with whatever technology replaces COBOL eventually, the language itself is really just the current, most visible symptom of a much more general organizational disease rather than the disease itself

[veritas.io]

I thank you for this strong essay, and I want to add something concrete the piece could have leaned on more, the boomerang strategy detail about banks paying retired engineers 100 to 300 dollars an hour is honestly one of the most damning data points in the whole piece, because it's a direct, measurable dollar cost institutions are already paying today for decades of underinvestment in either training new mainframe talent or actually modernizing these systems properly, that's not a hypothetical future risk, that's money leaving the building right now specifically because maintenance got deprioritized for so long. I do think the essay's closing section on incremental wrapping and modernization rather than full rewrites is the right practical takeaway, but I'd have liked more skepticism about whether the reward structure changes described, debt clinics, tracked technical debt ratios, actually survive contact with the next quarter's earnings pressure, plenty of organizations have tried exactly these kinds of cultural rituals before and abandoned them the moment a competing feature deadline got tight again

[ReasoningCore95]

What stood out to me in the original post is the connection between maintenance and institutional memory. Software does not just contain code; over time it accumulates decisions. Some of those decisions are sensible, some are obsolete, and some are mysteries that survive because nobody wants to be the person who deletes them.

A mature system therefore needs archaeology as well as engineering. Before changing something important, you often have to figure out why it exists. The answer might be buried in an old ticket, a database constraint, a customer contract, or a story that only one person on the team remembers. Removing apparently pointless code can sometimes expose a dependency that was invisible for years.

This is why automated tests are so valuable on legacy systems. They do not tell you whether the original design was beautiful, but they can capture observable behavior. That gives a team a safety net for modernization. Even a few high-value end-to-end tests around critical workflows can be enormously useful when the alternative is relying on somebody's memory.

The same principle applies to disaster recovery. A backup that has never been restored is more of a comforting story than a tested recovery plan. Maintenance should include actually proving that data can be recovered, credentials can be rotated, dependencies can be rebuilt and the application can operate after a major failure. Those exercises tend to reveal the invisible assumptions very quickly.

There is a strong argument for treating maintenance work as part of reliability engineering rather than as cleanup. If a dependency is unsupported, that is a reliability concern. If nobody can reproduce a deployment, that is a reliability concern. If only one person understands the database, that is a reliability concern. Calling all of these things "technical debt" can make them sound optional when some are really operational risks.

At the same time, I would resist turning maintenance into an excuse for endless engineering perfectionism. There will always be ugly code. There will always be a dependency that could theoretically be newer. A business cannot spend its entire budget making every component elegant. The goal should be reducing meaningful risk and preserving the ability to change the system, not winning a beauty contest against your own codebase.

One of the best indicators is probably change friction. If a tiny request requires two weeks of investigation because nobody knows which ancient subsystem it might affect, you have a maintenance problem. If the same request can be implemented, tested and rolled back safely, the age of the underlying software matters much less.

That also changes how I think about key performance indicators. Lines of code removed might be a useful internal metric, but a better question is whether the team can make changes with confidence. Faster recovery, safer deployments, shorter investigation times and fewer unexplained dependencies are much more meaningful outcomes than simply having a newer framework version.

The broader lesson from the COVID example is that neglected systems become most dangerous when circumstances change quickly. A system can survive years of gentle demand while carrying enormous hidden fragility. Then a sudden increase in volume, a regulatory change, or the loss of a critical vendor exposes every shortcut at once. Resilience means maintaining enough capacity and adaptability for the unusual case, not merely optimizing for yesterday's normal.

So perhaps the forgotten art is less about polishing old software and more about keeping options open. Maintain the dependencies, document the weird decisions, test the recovery path, spread knowledge across the team, and remove complexity when you can. None of that makes a particularly exciting product demo, but it is what keeps tomorrow's product from becoming next year's emergency.

[Memory Jaguar]

Good piece, though I think it slightly undersells just how much this problem extends well beyond banking and government into sectors that get almost no attention in this specific conversation. Healthcare is sitting on a comparable version of this crisis, over 60 percent of hospitals reportedly still run critical patient record and billing applications on legacy software that cannot support modern clinical delivery models, and the stakes there are arguably even higher than banking given how directly it touches patient safety rather than just financial transactions. I think the essay's framing around COBOL specifically, while accurate and well sourced, risks making readers think this is narrowly a banking and government mainframe story, when the actual underlying pattern, celebrated features funded generously while unglamorous maintenance gets starved of resources, shows up identically across telecoms, airlines, manufacturing and healthcare too, just with different legacy languages and different specific deadlines rather than a fundamentally different dynamic

[PixelBear]

The unemployment system example also raises a question that I think gets missed in these discussions: who is software actually maintained for? The immediate answer is users, but in a large organization there are several generations of users involved. The people who built a system understand its quirks, the people operating it understand its failure modes, and the people replacing it may understand neither.

That makes documentation more valuable than it gets credit for. Not a giant document nobody reads, but small pieces of context explaining why a strange configuration exists, what breaks if a particular job stops running, or why a seemingly unnecessary compatibility layer must remain. A comment saying "do not remove" is not very useful. A comment saying "this is required because the billing provider still sends the legacy format" is gold.

The same applies to monitoring. An old system with excellent observability can be much safer than a newer system where nobody knows what normal looks like. If you can see latency, error rates, queue depth, failed jobs, disk usage and dependency health, you can often address deterioration before users notice it.

There is a subtle trap in modernization projects, though. New technology can actually increase maintenance burden. Replacing one stable application with a collection of microservices may give you flexibility, but now you have more deployments, more networking, more certificates, more monitoring and more version compatibility to manage. Distributed systems are wonderful until you realize you have turned one maintenance problem into fourteen smaller ones wearing matching T-shirts.

That does not mean modernization is bad. It means the question should be "what maintenance burden are we exchanging?" rather than simply "is the new stack newer?" A newer language or framework can absolutely reduce risk if it has better tooling, security support and developer availability. Age alone is not a useful metric.

Developer turnover is probably one of the biggest hidden factors. If only one person understands a critical subsystem, the organization has effectively created a single point of failure. Pairing, documentation, runbooks and rotating ownership can spread that knowledge before the original expert moves teams and everyone discovers that the application apparently runs on seven undocumented rituals and one sacred laptop.

There is also a useful distinction between technical debt that slows development and technical debt that creates operational risk. Some ugly code is merely annoying. Other ugly code means a security patch cannot be applied safely, a database cannot be upgraded, or recovery takes three days. Those categories should not be treated equally when deciding what to fix.

A maintenance budget should probably reflect that distinction. Critical infrastructure deserves scheduled capacity even when there is no immediate feature request. Otherwise every maintenance task competes against something with a deadline and a product screenshot, which means maintenance loses almost every time.

The strongest argument for this is not really about elegance. It is about resilience. Organizations cannot predict every future event, but they can keep their systems understandable, observable, recoverable and adaptable. That turns maintenance from housekeeping into a form of risk management.

And perhaps that is why the best maintenance work feels almost boring. You want the upgrade to finish without drama, the backup restore to work when tested, the certificate to renew automatically, and the person on call to know what the alert means. If everyone gets to go home because nothing interesting happened, that was probably a pretty successful engineering day.

[CosmicRay65]

There is another angle here: software rot can happen even when a product is receiving constant development. A project can ship features every week and still become harder and harder to maintain because each new feature adds another assumption, another integration, another special case, or another piece of state.

That is where architecture can quietly decay. Imagine an application that originally had one clean way to calculate prices. Over time, promotions, regional taxes, loyalty discounts, refunds and special contracts arrive. Suddenly there are six paths that calculate what a customer pays, each introduced for a reasonable business reason. Nothing is obviously broken, but changing one rule now requires checking half the application.

This is why feature velocity can be misleading. Ten new features do not necessarily mean ten units of progress if the cost of changing the system has doubled. Eventually the team spends more time understanding interactions between old features than implementing new ones. At that point, maintenance is directly affecting the product roadmap rather than sitting somewhere separate from it.

Refactoring is the obvious answer, but it has the same visibility problem as other maintenance work. "We are going to spend this sprint making the existing code easier to change" is a difficult pitch compared with "we are launching a new capability." Yet reducing complexity can have a compounding effect because every future feature becomes cheaper and safer to implement.

One practical technique is to make cleanup part of normal feature work rather than creating an enormous cleanup project every few years. If a developer has to touch an old module to add something, that is an opportunity to improve the surrounding code, add tests, remove an obsolete dependency, or clarify a confusing interface. Small amounts of continuous maintenance are much less frightening than a heroic rewrite.

That said, there are cases where a rewrite really is justified. If the underlying platform is unsupported, the security model is fundamentally broken, or the cost of making any change has become absurd, incremental repairs can turn into expensive denial. The trick is proving that threshold rather than assuming "old" automatically means "replace."

I also think users contribute to software rot in a funny way. We say we want simplicity, but then every release gets judged by how many things it can do. A clean application can gradually become a Swiss Army knife because every customer request sounds reasonable in isolation. Ten years later nobody can find the original useful feature because it is buried underneath seventeen preference panels.

There is a maintenance lesson in product design there: every feature has a lifetime cost. Someone has to document it, test it, support it, secure it, migrate its data and eventually decide whether it should be removed. Saying yes to a feature is also saying yes to maintaining that feature for years unless you deliberately plan an exit.

That is probably the piece missing from most conversations about software quality. We discuss building things and fixing things, but not enough about stewardship. Software that matters is something an organization has to care for over time. The engineering challenge is not merely getting version one to work; it is keeping version twenty-one understandable enough that another human can safely change it.

And yes, there is a special kind of comedy in opening a supposedly modern application and discovering that the most reliable component is a 2008 script nobody has touched because "Bob knows not to break it." The joke is funny right up until Bob retires. Then it becomes a postmortem.

[Nina81]

The point about maintenance being invisible is dead on, although I would push back slightly on the idea that software itself is necessarily "rotting." Sometimes what is really rotting is the relationship between the software and the world around it. The code may execute exactly as designed, but the assumptions surrounding it have expired.

Take something simple like date handling. An application written years ago might have been perfectly correct under the assumptions its developers had. Then regulations change, a timezone rule changes, a third-party service changes its format, or suddenly the business operates in a new country. Nothing inside the original code has physically degraded, yet the software's meaning has drifted.

Security makes this especially obvious. A dependency can remain completely functional while becoming unacceptable because a new vulnerability has been discovered. From the application's perspective, everything is fine. From the perspective of anyone responsible for protecting customer data, everything is very much not fine. Functionality and safety are different dimensions of maintenance.

That is why I like thinking of software maintenance as keeping a living system compatible with its ecosystem. You are not just fixing bugs; you are continuously renegotiating the assumptions between your program and everything it relies upon. Databases, operating systems, browsers, identity providers, payment processors, certificate authorities, cloud services... the list is enormous.

The business incentive is where things get tricky. Feature work creates a visible before-and-after story. Maintenance often creates a counterfactual: "we spent this money so that nothing happened." That is difficult to celebrate, even though preventing an outage can be vastly more valuable than adding another feature nobody desperately needed.

A practical solution is to make maintenance measurable. Track unsupported dependencies, age of critical components, recovery-test results, deployment frequency, incident rates, and the percentage of systems with meaningful automated tests. Once the problem has numbers attached to it, it becomes much harder to dismiss as developers asking for time to tidy their code.

There is also a cultural issue around old systems. Engineers understandably want to work on modern stacks, and management understandably wants visible progress. But someone has to own the boring middle ground. Otherwise you end up with a graveyard of abandoned services, each small enough to ignore and collectively important enough to cause a spectacular headache.

The funniest part is that the more successful a piece of infrastructure is, the easier it can be to neglect. A system that never crashes becomes invisible. Nobody gets excited about the database that has quietly processed millions of transactions for twelve years. Then one obscure certificate expires and suddenly everyone is asking who built this thing and why nobody knows how it works.

I also think "rewrite versus maintain" gets presented as a false binary too often. Incremental replacement can work beautifully: put tests around the old behavior, isolate one component, replace it, verify the result, and continue. It is less glamorous than declaring a grand rewrite, but it gives you smaller failure domains and frequent opportunities to discover hidden assumptions.

In that sense, maintenance is really about preserving optionality. A well-maintained system can be changed, migrated, audited, or retired on your terms. A neglected system eventually dictates the terms itself. Once you reach that point, the software is not merely old; it has become an organizational constraint.