This is the first article in a series that I am writing about how to create and maintain better quality unit tests. Over the last few years I have been involved in several different development projects where we have made a valiant effort to unit test our code. All of the team members saw the benefits of writing automated unit tests and were motivated to make the unit testing framework a successful. However, there was always a constant struggle to keep the unit tests relevant and beneficial. This was mostly because the tests would not stay current with the code and was quickly outdated. It didn’t provide the expected benefits to help refactor and grow the code base.
In many instances our teams have seen the unit tests even become a hindrance during the development or refactoring process and make things harder to maintain (rather than easier as they should) for the team. I have even seen source control commits with comments like ”fix stupid test.” Why does this happen? Is there anything we can do to have our unit tests evolve successfully with the production code and provide a solid foundation for additions or refactorings in the future? Is there any way we can make them conducive to change?
There are several reasons why well intentioned unit testing frameworks can actually become a hindrance to a project:
- The design of the code does not take into consideration the fact that it will be tested.
The single most important artifact of testing is not the automated tests themselves. That’s right, the tests are not the most important aspect of unit testing – they are only a side benefit. The most immediate (and obvious) benefit people usually see for automated unit testing is that the code is now better tested and therefore less likely to have bugs. Although this is a very important benefit to unit testing it is not the most important aspect.
The most important benefit of unit testing (with well written tests) is that it strongly encourages good design. If your code is written badly, it is almost guaranteed that it will be difficult to test. If you find a unit of code that is proving difficult to test, you should step back and take a look at the design of the code in question. Making code that is easy to test requires using sound software design patterns and SOLID principles. The act of writing the tests (especially in a TDD manner) will have a positive impact on the quality of code under development.
- Tests are not written first.
TDD encourages developers to write tests before writing the code under test. This forces the developer to think about the code being written from the consumer’s perspective. Because of this, it encourages good design. It also insures that the code under test is well covered by the testing framework. When tests are written after the production code, the unit tests are usually more difficult to write. This is also a big opportunity for gaps to be introduced into test coverage.
- Gaps in the unit tests.
There are some parts of the application that may prove difficult to test. In these cases it is always tempting to forgo unit testing on these portions of the code. However, these gaps in the tests make it more likely that future development will avoid even larger portions of the code and can have a negative impact on the confidence level in the testing framework.
- Badly written tests.
Tests, like any other code artifact, when written badly are very difficult to maintain. Badly written tests are difficult to understand and refactor. When the tests are difficult to maintain then they get in the way of developers trying to add features or refactor code.
Instead of fixing these tests, a common practice is to set them to be ignored or to lower the success criteria. Once these sorts of Band-Aid test fixes begin to be applied, the usefulness of the test framework quickly degrades. In fact it may even become detrimental to the project. If you don’t trust the tests and you still have to hack them to update them then they are really not providing any value but instead are just getting in the way.
Another issue is that the testing framework may inspire confidence where none is warranted. A developer making a change may feel safe that their change didn’t break any functionality because the tests all passed. However, in reality the test coverage and quality is so minimal that it provides little to no guarantee in the correctness or impact to the code base.
The common theme for all of these issues which can cause problems for a unit testing framework is that it is very important to write high quality test code. Test code should be first class artifact of your project, just like the production code. This may seem like an extra amount of work at first, but in the end it will greatly benefit the project.
According to Bob Martin in his book Clean Code (2009): “Test code is just as important as production code. It is not a second-class citizen. It requires thought, design, and care. It must be kept as clean as production code.” (p.124) Obviously, I completely agree with this assessment. It is critical to the success of a unit testing framework that they maintain the same code quality standards for understandability and maintainability as the rest of the code base.
One of the myths I often hear about unit testing is that it’s acceptable to write tests with less concern for quality than for production code. I’ve frequently heard comments like: “It’s just a test, it doesn’t matter if it follows our standards or good software development practices. All that matters is it verifies the code and passes.” I’ve even proceeded to write tests with this as a guiding principle. Unfortunately, in all of these cases the testing framework did not provide the benefits I had hoped for and quickly became a detriment to the project.
I have even heard very knowledgeable programmers joke that you want your production code DRY, but it’s ok to have your testing code be a little bit wet. The implication here is that it is acceptable for test code to violate good software design principles and practices. I believe that this approach to unit testing is ineffective and is a leading contributor to the issues that many of us have encountered as our testing frameworks evolved.
Ok, now that we know why unit test frameworks sometimes struggle and why it is important to write good tests, how should we proceed? I have found several good practices, patterns, and tools to greatly improve the quality of my test frameworks. I plan to follow with future articles in this series that will outline these by taking an example of a badly written test class and gradually improve upon it.
Next in the series: Example of a badly written test class