Tech Debt: Oblivious, Calculated, and Natural

#NoBugs

Ward Cunningham came up with the Technical-Debt metaphor to describe the natural ebb and flow of product development. The notion is that you sometimes incur debt (as you would on a credit card) to achieve some useful end, but that you must retire that debt in order not to be buried under its weight.

To me, there are three sorts of technical debt: Oblivious (or Negligent), Calculated, and Natural.

Oblivious debt is caused by general sloppiness, lack of skill, or bad process. You can also look at it as Negligent Debt because the things that make people oblivious can be remedied by things like learning, working collaboratively, approaching the work with professionalism and a desire to do the best job possible. Oblivious debt can also be addressed by process improvement (e.g. a no-known-bugs-on-release policy) and a culture of quality. Not doing those things is a choice—negligence.

Calculated debt is debt created deliberately in order to achieve some outcome (e.g. out the door sooner). For example, agility is all about frequent releases and fast feedback loops—hours or, at most, a couple days. Deliberately lengthening your feedback loop in pursuit of the chimera of perfection does active damage, often exceeding development costs by a large factor. (Paradoxically, trading perfection for fast feedback usually gets us closer to perfection—or, at least, very good—faster because the things we learn by releasing sooner make the product better.)

Calculated debt, then, improves agility by shortening feedback. It's a Calculated debt to initially implement a microservice system as independent components in a monolith and, later, split them out into actual services when the system grows to the point that makes sense. It's Calculated debt to use a JSON in a flat file instead of a database in the early stage of development because you only have a handful of users and don’t want the added complexity of a database to slow you down. It’s calculated debt to quickly cobble together a clunky UI on order to free up the time to work on core functionality so you can guarantee that your system will be able to do your taxes by April 15,

However, one sort of Calculated debt never works: Negligent Calculated debt: "Let’s write quick-and-dirty code to get it out the door faster." This sort of Calculated Debt usually achieves the opposite of the intended effect. Work moves fastest in clean, high-quality code—you don’t have to double check everything and deal with unnecessary complexity as you build. Unexpected bugs don’t slow you down for a day while you struggle to fix them so that you can get real work done. In fact, sloppy thrown-together (usually buggy) code doesn’t get out the door faster at all. Quite the contrary,. You are calculating your own destruction.

Finally, there’s Natural debt: you do the best job you possibly can, only to learn later that you could have done it better. Natural debt is inevitable. Ward Cunningham desccribed paying off that debt as “[the team changed the code] so it looked like they knew what they were doing to begin with.”

Many people classify bugs as natural debt because “bugs are inevitiable.” I don’t buy that. Bugs are code that doesn’t behave the way you expect. Natural Debt is code that behaves exactly as you expect, but your expectations are incorrect. Natural debt has nothing to do with bugs. You deliver the best code you can, given what you know now. You are not deliberately ignoring things about how the code should work that you do know. The released code is high quality, fully tested, and with zero known defects. It’s wrong, though, because you don’t know something, and there’s no way to know exactly how it’s wrong because you don’t know what you don’t know.

The three sorts of debt all must be retired so that you down drown under its weight, but in different ways. Tolerating Oblivious debt considerably slows your development process and damage your agility. The code eventually gets to the point where simple 5-minute changes take weeks to implement. Agility is impossible under this debt load. Like credit-card debt, there may be no way to get out from under this load short of the equivalent of bankruptcy: replace the entire system, salvaging what you can.

If the Oblivious-debt load is light, then retiring it is just a natural part of the programming process. When you see a problem, fix it. Some call this the “Boy-Scout rule:” leave the campsite cleaner than you found it. To me, not refactoring constantly as you work is the worst sort of neglegence, and no, it does not “slow you down,” becuase work goes much faster in the cleaner code.

Retiring Calculated debt is a natural part of the development process, but the ease of doing so depends a lot on architecture. For example, in Domain-Driven-Design based systems like micrservices, every small architectural component (service) is responsible for it’s own data managment. There is no large shared database; every component has it’s own database. Swaping out that flatfile for a more capabable database in that system is trivially easy—it might not take longer than a few minutes. Doing the same in a big-ball-of-mud sphaghetti-code system is another matter entirely. You may be looking at months of work. Architecture matters.

Finally, there’s Natural debt. Not retiring that can bury the company. Nokia, for example, died under the weight of a technical architecture that wouldn’t let their developers make changes fast enough. However, sometimes just leaving things alone is the best approach. It may not be worth updating or improving your front-end framework if it’s good enough and isn’t slowing back-end development, for example. There is a judgement call, of course. The longer you put off the fix, the harder the fix will be, so you’re gambling.

So, there are three sorts of Technical Debt, all handled differently. The important takeaway, however, is that they all must be handled so that you don’t end up burried under your debt.