Many developers aim for 100% code coverage, but that is not always a good thing.
When 100% code coverage becomes the goal, it’s easy to lose sight of what you’re doing and why you’re doing it.
Should you aim for 100% code coverage? No. 100% code coverage sounds like a good idea on paper, but it’s difficult to achieve and can be very expensive. Developers spend too much time on unit tests that don’t provide any value, and 100% code coverage can cause more issues in the code than it prevents.
Increasing code coverage to 100% is also detrimental to development cycles.
To avoid falling into this trap, all developers should have unit testing as an integral part of their coding process. Unit testing ensures that your applications are stable and you can patch bugs before they become big problems. But don’t focus only on code coverage as a single metric for code quality.
What is code coverage, and why does it matter?
Code coverage is a unit testing term that defines the amount of code for which unit tests have been written. In other words, code coverage measures how much of the code in your application has been executed when you run unit tests.
For example, 100% code coverage means having 100% of your code tested with the unit tests.
Code often has bugs, so you write unit tests to find those mistakes and fix them before customers find them. For example, if a unit test runs on 10% of the code it should run, 90% of the untested code can potentially have bugs.
Why is code coverage important?
Code coverage is necessary because it lets developers know what parts of their code are covered with tests. Coverage reports come in handy when the developer uses code coverage to estimate which parts still need to be unit-tested. Another benefit is that you have more confidence in changing the code that has full code coverage.
What methods of code coverage exist?
There are several different methods of code coverage:
- Function coverage – The function coverage is the percentage of functions the test suite has executed.
- Statement coverage – The statement coverage is the percentage of executable statements that the test suite has executed.
- Branch coverage – The branch coverage is a metric that measures the percentage of conditional branches (for example, branches in the if-else statement) in the source code that is executed during the test.
- Condition coverage – a technique used to test and check variables in the conditional statement.
Why can 100% code coverage be bad for your application?
Code coverage measures how effectively automated tests are testing code. There is no set rule on how much unit test code needs to cover production code. Some experts recommend up to 100%. The problem with this approach is that unit tests can be very time-consuming once you pass a certain threshold, say 70%.
You get smaller returns on your effort once you get past a certain percentage on the road to 100% code coverage. Not only that, but you might hurt your code quality.
What do I mean by that? To further increase code coverage and write more tests, you need to change the production code and introduce additional levels of abstraction.
We can solve any problem by introducing and extra level of indirection.
– Andrew Koenig
…except for the problem of too many levels of indirection.
Introducing additional levels of abstraction can introduce complexity. We all love adding interfaces to our code. But that new complexity can slow you down in the future. In addition, to make a single change, you often need to modify several files. That kind of modification is also known as shotgun surgery.
The worst thing that can happen to a developer is to get a direct order from a manager to maintain 100% code coverage all the time. It’s a recipe for disaster! Developers then start to write unit tests just for the sake of maintaining the desired code coverage. That results in unit tests that are unhelpful and unmaintainable.
The new unit tests contain too many or redundant assertions that don’t provide much insight into the unit-tested code.
What is the ideal code coverage?
The ideal code coverage is the desired percentage of unit tests for your source code. This can depend on how much unit testing has been done and what unit testing needs to be done. A good rule of thumb for unit test coverage is to determine what level of coverage should be achieved before determining any other factors.
Ideal code coverage: is it 100%?
No, it’s less. How much exactly, it’s hard to estimate. The ideal code coverage depends on the type of project you have and how easy it is to write unit tests.
As a general rule, aim to have code coverage between 50-80%. This article has some tips on how to improve code coverage.
However, don’t focus only on the number. Code coverage metrics shouldn’t be the guide whether or not you are producing quality code.
Instead, track the number of bugs in your system and write unit tests for the features that break more than other system parts.
What to do instead of tracking code coverage
The first step you can take is to write good tests. When a test case fails, it should be easy to know which code caused the failure. Unit tests should be as small as possible without jeopardizing their utility. In other words, unit tests should have a single purpose and focus on a single unit-tested block of code. For example, unit tests for function Foo() should only deal with function Foo() and not deal with any other code that might be required.
The second thing that you might do is add another type of test automation: integration tests and UI tests. They are slower than unit tests, but they are also more stable. In addition, integration and UI tests can help you identify bugs that unit tests might miss.
The problem with unit tests is that they are very limited in their scope.
Unit tests only test one unit of code at a time, but the production code interacts with many other coding units. For example, you might have page controllers, model layers, views, configurations, and more in your application. Unit tests are often not sufficiently good at catching bugs or errors in these other areas. Because of this limitation in focus and execution, unit testing does not always give you the complete picture of how your code will work in a deployed application.
You should combine unit testing with other forms of automated and manual testing to ensure that bugs won’t slip through into production environments.
Also, focus on having a clean code. Make sure you refactor all the time. Clean code is unit testable.
Conclusion
Automated testing is an integral part of the coding process, and unit testing ensures that applications are stable and you can patch bugs before they become big problems.
Unit test coverage is essential in larger applications with lots of code where bugs hide in unknown areas. But 100% code coverage is not the best goal to have.
As I explained at the beginning of the article, high code coverage is a by-product of a clean, maintained, and tested code. As such, the percentage varies depending on the complexity or requirements of each project.