SPYDERMAN360, CC BY 2.0 https://creativecommons.org/licenses/by/2.0, via Wikimedia Commons

Originally posted at Clear Measure

The Nurture Bit

When I was growing up, I spent part of my summers with my grandparents in Oklahoma. I was inculcated into the value of maintenance early on. My grandfather did most of his own car maintenance and repair. I'm not talking about just an oil and filter change. Back then, a determined car owner could virtually rebuild an entire automobile.

My grandfather had a car lift in his garage.

Imagine if Danica Patrick or Jeff Gordon was told just before a race "sorry, we didn't fix your brakes because the boss said to paint a new sponsor's logo on the car. Maybe next time." I personally don't have to imagine what happens if you don't replace your tires. I did a 360 on the highway because of balding tires in snow.

To me, paying down technical debt, refactoring, cleaning up documentation, is just part of what a development shop must do. Likewise, operations must keep computers updated. However, I've seen many organizations where maintenance is treated by management as optional, or worse, as a profit obstacle. And I've seen just as many developers and network engineers avoid maintenance. Why? What are the consequences? Do they matter?

The Nature Bit

There are several reasons humans metaphorically--and actually--don't change their oil or replace their Access 97 applications. Among them,

  • It's still working, and "if it ain't broke" . . . well, you know the rest.
  • We're not good at long-term planning and "tend to opt for immediate rewards."
  • We're adaptable. Very. Hedonic adaptation, for example, has its good points, but not when it keeps us from fixing those security holes that we "really should get to someday."
  • We think it's too expensive in dollars, time, effort, or any combination of those. As Mac Smith puts it in this article, "One of the things that has NOT changed very much is the age-old management perception that maintenance is just a necessary evil and unfortunate cost center."

In my experience, maintenance such as paying down technical debt is treated as second-class, something you might have time to do after you're done writing your software and fixing bugs. Management too often sees it as similar to employees reading about business improvement between support calls; "you can do that on your own time." And while we software developers know refactoring is important, how often do we not do it just because (we feel) there isn't time?

Technical Debt is Really Debt

Measuring and reporting technical debt can be challenging. You'll find articles saying it's harder than measuring financial debt. But they all agree that technical debt translates to real dollars. We're talking potentially big numbers.

Based on this definition [the cost of fixing the structural quality problems in an application that, if left unfixed, puts the business at serious risk] and the analysis of 1400 applications containing 550 million lines of code submitted by 160 organizations, CRL estimate that the Technical Debt of an average-sized application of 300,000 lines of code (LOC) is $1,083,000. This represents an average Technical Debt per LOC of $3.61. --Technical Debt Estimation - CAST

Technical debt is unavoidable. Like buying an office building, as soon as you create software you have dollars of technical debt. Business owners pay for regular building maintenance, why not regular software maintenance? The question for organizations is, "does not cleaning up code really affect my profit?" I think unless shown otherwise, you should assume the answer is yes. Here's a list of impacts.

  • Technical Debt increases time-to-market and development costs. How?
    • Increased security risks The best way to increase the chances of your data being stolen every day? Don't prioritize auditing and daily maintaining your software.
    • Time to understand code The "dirtier" the code base, the longer it takes developers to understand an already complex system. I've experienced code that regularly took me three times longer than it should have to figure out.
    • Increased likelihood of introducing bugs The harder code is to understand, the more likely you'll break something when adding a new feature. Automated Testing helps with this, but older code often isn't written to make testing easy.
    • Decreased maintainability Ever tried to find a part for a 1980s appliance you dearly love? Multiply that by a thousand and that's what it's costing your business each day your developers struggle with old or poorly architected code.
    • Harder to replace Software components eventually have to be replaced. But it's not like replacing your car's tires. It's more like replacing all the galvanized pipe in a thirty-story five-star hotel filled with Hollywood stars before the Emmys.
    • Prone to outages The authors of Meltdown found that highly complicated, tightly-coupled systems will inevitably, unpredictably fail. Software is--you guessed it--by its nature prone to complexity and coupling.
    • Employee depression leading to apathy and increased turnover An overlooked cost of technical debt is the health toll it takes, nicely expressed in the article Technical Debt is Soul-crushing
    • The inability to quickly respond to changing requirements and priorities Consider the above points. If it takes longer to understand, maintain, fix, and introduce new features to existing code, how can the developers pivot and respond quickly to new market conditions? It's hard to turn a submarine. Now imagine trying to turn a fleet of subs. Managing technical debt transforms those submarines into James Bond sports cars.

I'm Convinced. I Think. What Do I Do?

Like buying an old house that you learn has plumbing, electric, and foundation problems . . . but you gotta live there so you can't just tear it down . . . technical debt can be overwhelming. The answer to overwhelm is chunking down the work load and choosing tasks wisely.

1. Forget the Blame

It's not anyone's fault. Or it's everyone's fault. Regardless, the way forward is to act like scientists and colleagues. Get curious. Fix the problem, not the blame.

2. Accept Change, Especially at the Top

Technical debt has been created as much from the top of the organization as from the coders. Every time a project was made "top priority," and teams were asked "how do we work around that to release faster?" without also saying "but then it needs to be done right, immediately," executives reduced the abililty to stay fleet-footed. Execs and devs must both change. It'll be for the better.

3. Measure For Reality

How will you define technical debt? How will you discover it? How will you measure it? The answers to these questions lead to facts, and facts often aren't comfortable things to face. You can do this yourself, but I recommend partnering with an outside company such as Clear Measure1, who doesn't share the organization's culture or biases. Be sure they use evidence-based metrics such as the DORA group's four key metrics of high-performing teams.

4. Improve a Little, a Lot

With facts in hand, you have ways of measuring progress. Choose an easy win that the developers agree will help them. Not the executives. Not project managers. Developers. This gives them ownership of their purpose. I again recommend they accept coaching just like a top racer or athlete does. Utlimately, though, they'll find their solution through their autonomy and mastery. Next, repeat. A one-percent improvement each day is realistic and sustainable.

5. Celebrate

Every success needs a reward, especially the little ones. Repeated behaviors with rewards become habits, and that's what we want when it comes to software maintenance and paying down technical debt.

References and Further Reading

  1. Disclosure: I'm an employee of Clear Measure.↩