As part of the agile process, each development sprint has a planning prerequisite. During this planning process, user stories are scored using a consensus complexity and risk estimations. It is common that some stories are deemed “to risky” and either scaled-back, re-evaluated or hit the chopping block. Minimizing risk and defects is something desirable to all - management, developers and stakeholders. This mix of risk and defects is sometimes referred to as “software entropy”. From my thermodynamics class, I vaguely remember that entropy has something to do randomness, or was it heat - or is that enthalpy?

After a quick google + wiki session, I found that entropy is the amount of disorder in a system. And there is a principal that says in a closed system, one can’t decrease the amount of entropy, it either increases or could possible stay the same. So that doesn’t look good if we apply this to software. This basically says that each iteration the very best we can do is have the same unpredictability, but most likely our software will continuously get worse! Not good news for the software industry and not good news for agile practitioners.

However, we know software doesn’t always become more unpredictable, software can become more stable. So what’s the factor and how can being agile, which embraces change and uncertainty, be used to minimize software entropy?

The factor is actually refactor, as in software refactoring. So we follow the laws of thermodynamics in this house, our system is not a closed one (it all depends on how you draw your boundaries). Whenever we add or removed lines of code, we are doing so as an external influence. This can actually reduce software entropy. For instance, if we have a defect rate of say 3 defects per 1000 lines of code, and we remove 500 lines of code in an iteration, we’d have 1.5 less defects, right? Well that was pretty theoretical, but the concept does work in a very tangible, and common occurrence - bug fixing.

We consider any bug fix that turns out to be a software defect (i.e. one that is not misimplementation of some business requirement or feature) a refactoring effort. When a defect is addressed, the existing code is altered as we add or remove bits - and we are external influence. Fixing defects can have 3 possible outcomes: 1) the system is more stable, software entropy reduced (ah!, the fix works!), 2) the system is not any more or less stable, software entropy unchanged (the fix didn’t work, but at least you didn’t break anything), or 3) the system is less stable, software entropy increases (regression! whack a mole! uh-oh!). So if we are good bug fixers, we can reduce entropy.

Now for the agile part. As we work through our iterations, defects land on our plate. And properly fixing them does reduce entropy. But another way to more drastically reduce entropy is some good ole’, heads-down refactoring effort. Pure refactoring specifically targets minimization of software entropy. When we refactor, we are DRYing up methods, eliminating lines of code, cleaning and making our software more stable and manageable. We strongly recommend that each iteration ought to have some overhead earmarked for refactoring.

It is sometimes difficult to justify refactoring to stakeholders and non-developers, as the business value is difficult to measure. However, if a new feature is deemed “too risky”, a trade-off could be to add a pure refactoring effort to the iteration targeting the potential risk of the feature. Then build the feature.

So the moral is to minimize entropy and uncertainity in software, try to have refactoring a part of each development iteration.