Bad Unit Tests are Worse than no Unit Tests

Unit tests represent our best intentions as developers. They are non-functional code that take time and effort to write and maintain. But what they give us is confidence in the quality of the code, and confidence that modifying code won't break existing functionality.

Poorly written unit tests can make developers forget their best intentions. If unit tests are poorly written, involve heavy mocking, or are tightly coupled with the implementation, unit tests become an impediment.

Because effort has to go into understanding those tests, and the tests are divorced from their original context, it's understandable developers will choose to sacrifice the test rather than the time for maintaining it.

Because of the role unit tests play, it's 5 best to avoid this situation. This means spending time writing good unit tests and testable code.

Good unit tests are decoupled from the solution. They are functional without being prescriptive as to the details. It's not about making sure every line of code is covered, but making sure the code does what is expected.

To achieve good unit tests, it requires writing testable code. Unit tests shouldn't be an afterthought, but part of the development. If heavy mocking is required, that's a sign the code needs refactoring. If it's hard to see how unit tests could be made to test the code, that's another sign.

Unit tests are like any other aspect of a codebase. At all times, we are writing to be understood long after we have left the project. This is just as true for code where other developers come in cold, or even ourselves once we have context switched away from the problem at hand.

Purposes of Unit Testing

1. As a development process
   Test-driven development is a useful skill to learn. It encourages a clean design, as writing unit tests for messy design is both painful to write and painful to maintain. It also gives a target to code against, by setting what what the function ought to achieve before the function is even written.


2. Proving the code behaves correctly
   Writing unit tests is the simplest way to show that code does what it is meant to do. Unit tests allow easy checking of boundary conditions, to ensure that the function behaves as expected.


3. Demonstrating robustness
   In complex systems, well-written unit tests demonstrate that functions behave as expected. As more features are added, or as code is reused, unit tests demonstrate that these changes do not affect the existing functionality. Conversely, if unit tests easily brake when code changes, it's a good indicator of fragility.


4. Testing
   Some tests algorithms are better for than humans. When testing a function involves a large amount of data, unit tests are best equipped for this job. This is also true for time-dependent functions, or testing algorithms.


5. Refactoring
   Unit tests provide the assurance needed that refactoring the codebase hasn't accidentally changed functionality. Without unit tests, refactoring is fraught with risk.

When Algorithms Test

Code needs to be tested. Sometimes that testing is best done by an algorithm.

We recently had a user story that involved removing particular characters from the application number string. The purpose was to address a readability issues - at a glance certain characters and numbers are indistinguishable, so removing the offending characters from the string they were generated from. A simple enough coding exercise.

This card, however, sat in the testing column for a long while. It didn't need to, as we had built a unit test that did what a regular tester would do - throw lots of data at the generation and see whether those offending characters were present.

An automated test, in this case, could do what a human could not: quickly generate and accurately assess the functionality. It's checked 1000 times every time the code is compiled - overkill as far as what's needed to be tested, but that overkill is computationally negligible.

Writing the unit test first (TDD) gives the developer the target to aim for, as well as creating the before and after test artefacts. At the very least, the unit tests results could supplement the functional spot-checking that would constitute normal testing for such a story.

Writing code that verifies functionality is not simply for helping developers to write better code, but can be the best tool at the disposal of the testers for doing testing. It's leveraging technology at tasks it excels, which is what we ought to be using technology for.

Following the Boy Scout Rule

"Always leave the campground cleaner than you found it."

In all my years as a software developer, the only times I've ever gotten to start from scratch is when I've worked on personal projects. And even then, starting from scratch is a last resort.

My experience of being a programmer is working in someone else's architecture. I get to expand it out, fix it up, and sometimes refactor it away. And each time I end up having the same question - why is it built that way? And each time there's the same answer - it's because the code needed to be delivered quickly.

It feels at times that all we can do as programmers is try not to make things worse. One thing I've done is tried to fit in with the existing way of doing things. That way the code is consistent for those who maintain it in the future. But it doesn't solve the problem, merely prevents it from getting any worse.

The answer is meant to be refactoring, though it's really not an answer for fragile code. It's a sad irony that the code in most need of refactoring is the code least able to be refactored...

Any complex system is going to seem impenetrable at first glance, but there are ways to improve it. A system doesn't need to be overhauled to be fixed, and something as simple as tidying up the code that's being worked on is enough to make things better.

For this project, we have pretty abysmal unit testing. So one rule we've set for ourselves is to put unit tests in wherever we find ourselves enhancing the code. Coverage is still woeful, but it is less woeful than it was a month ago. Another thing is to make sure no commit has commented-out code.

Code doesn't get to a messy state by a single act, nor can it be fixed in one. The complex mess that is the average codebase (at least average from my experience) is the way it is because of a long path to that state. Making things better is a similarly long path, but thankfully one that be addressed as part of enhancing a system. Fragility is quantitative, not qualitative.