March 3, 2016
Imagine the following situation: a team has developed for 6 months a great product that immediately sells. Users show their passion for the product by asking new features. If the team does not deliver the new features fast enough, their happiness will decrease. They might even switch to a competing application. The team must deliver quickly.
Unfortunately, it takes weeks or months to validate the product completely, without accounting the time for bug-fixing. The team cannot deliver in time if the product is tested completely. What can they do?
The most common strategies are:
- test only new features, hoping that the old ones have not modified
- analyze the impact of changes and test only the affected features
- users test the application during a stabilization period.
All these strategies translate into temporarily decreasing the care for the product in order to obtain faster delivery. Unfortunately, the technical teams in the scenarios above are taking business risks. Their major risk is missing the impact on certain application areas and delivering bugs. This can turn into:
- Decreased user happiness and creating detractors. In business and life winning a person’s trust is hard, but very easy to loose.
- Increased support cost. Every bug reported by users means time spent to understand, solve, test and deploy of the new version. Expenses accumulate in: call center, development, testing, operational etc.
- Opportunity cost: while the team solves bugs, the competition can produce new features that get the users. Solving bugs is equivalent from the business point of view with running in place.
The team could validate the whole application in hours and not in weeks or months?
What if each programmer could learn after each code change, in minutes, that nothing else was affected (with 80+% probability)?
The article on Software Craftsmanship mentions the main idea of the movement: a software craftsman can deliver quality under pressure. Given the circumstances above, a software craftsman should deliver new features in the given time with as few bugs as possible. How few? Less than 10 per release. This is a conservative number. Agile methods (including Scrum) demand 0 known bugs at the end of each 2-3 weeks sprint, after a validation by testers.
But any software application has bugs!
The name “bug” is a euphemism for mistake. Mistakes are normal in any human activity, including software development.
How can we reduce the mistakes and their impact? Unit testing is a tool that can help, but it is not the only one. Others are: code review, pair programming, reducing the number of lines of code, design by contract.
Unit testing means writing code, named testing code, that validates the production code. Testing most of the application becomes automatic.
History: programmers often believe that unit testing is a new practice. The reality is that unit testing was used from the times of mainframe computers and punch cards. Back then, debugging was very difficult because it implied reading tens of meters of sheets of paper printed with the program results and execution information. Automated tests that ran at the same time as the program gave much more information about the source of the mistakes.
What about testers? They are often afraid that they’ll lose their work once automated testing is introduced. The truth is that testers become much more important because they’re the only ones who can discover hidden problems, difficult or impossible to find with automated tests. They help increase the probability of correctness from 80+% to almost 100%.
Unit tests have a few important properties:
- each test validates one small behavior of the application
- run really fast, under a few minutes
- are short and easy to read
- run on the push of a button, without additional configurations
To be fast, unit tests often use so called “test doubles”. Similar with plane pilots that learn in a simulator before driving a plane, unit tests use pieces of code that look like the production code but are only used for testing. Stubs and mocks are the most common test doubles; others exist but are less used.
A stub is a test double that returns values. A stub is similar with a very simple simulation: when a button is pressed, a value appears. A manually written stub looks like this:
A mock is a test double that validates collaboration between objects. Mocks validate method calls with certain parameters, for a certain number of times. A mock can be used to validate method calls for methods that don’t return values.
Test doubles can be created using special frameworks, like mockito for Java (ports exist for other languages) or moq for .NET.
Test doubles were initially used only in places where controlling the system was very hard or where tests were slow due to calls to external systems. In time, test doubles became more used in all unit tests, resulting in the “mockist” way of testing. The article Mocks aren’t stubs by Martin Fowler has much more details.
Unit tests are written by a programmer, during the implementation of a feature.
Unfortunately, the most common way of writing tests is sometime after the implementation was finished. The usual result is that the tests are not written or they are written based on the existing implementation and not on the correct application behavior.
Test First Programming is a way to write tests that has the following steps:
- design the solution
- write the minimum code (compilable, if the programming language is compiled) based on the design
- write one or more tests that show what the implementation should do; the tests will fail
- write the code to make tests pass.
By applying Test First Programming, the developers make sure that they write unit tests and they test the required behavior.
Test Driven Development (TDD) can be considered a third way of writing tests.
Actually, TDD is a way of making incremental design.
Writing tests takes a long time!
Case studies and personal experiences show that the time spent just to develop a feature is indeed increasing when adopting unit testing. Same studies show the time spent to maintain the product decreases considerably, showing that unit testing can be a net optimization in the total development time.
However, no study can change the programmer’s feeling that he or she writes more code. Often, programmers assume that the project is slower because of automated testing.
How do I start?
The best way to adopt unit testing is carefully and incrementally, following a few important points:
Option 1. Clarify unit testing concepts
Programmers should handle fearlessly concepts like: stubs, mocks, state tests, collaboration tests, contract tests, dependency injection etc. Programmers should also understand what cases are worth testing.
Mastering this concept is possible in a few ways:
- specialized unit testing workshops (Mozaic Works offers such a workshop that constantly had feedback over 9.25/10 from the attendees)
- pair programming between a tester and a programmer
- pair programming between an experienced unit tester and a beginner. The experienced unit tester can be an external technical coach
- reading books (see the recommended books at the end), Internet resources or attending community events
- attending conferences where concepts related to unit testing are discussed
Option 2. A technical coach or trainer can work with programmers, helping them transfer theoretical information in their everyday practice such that productivity is least affected
Option 3. Write unit tests for the most important parts of the application and then of the features with the highest risk of mistake
Option 4. Use the “Test pyramid” testing strategy to eliminate as many mistakes as possible
Option 5. If lots of code without tests exists (legacy code), learning additional techniques to write tests on existing code is recommended. More details in a future article.
A few common mistakes related to unit testing are:
- Writing too many integration tests3 (that exercise more classes and modules) that are slow and fragile instead of unit tests that are small, fast, easy to maintain
- Not using test doubles, or using them for other purposes than they were created for. Test doubles help writing short, fast tests.
- The test names don’t express the tested behavior. The test name can provide a lot of information when the test fails.
- Extensive usage of debuggers when tests fail. Well written tests immediately tell where the problem is when they fail. Debugging is still useful in exotic situations.
- Not caring for the testing code. The test code is at least as important as the production code.
Unit testing is one of the tools a programmer can use to significantly reduce the number of mistakes he or she makes when writing code. When correctly used, unit testing can significantly reduce the time spent with bug fixing, reducing the load on colleagues that take care of support and testing and allowing the introduction of new features faster, resulting in higher competitiveness.
Care must be taken when adopting unit testing by following industry practices (the pyramid of tests, using test doubles etc). External help (training and technical coaching) can make the difference between a successful and a challenged adoption. Technical coaching is a fast way of adopting a new practice, with hands-on, on the job and on production code help of a person specialized in the technique and in teaching it.
A software craftsman masters unit testing and uses it to protect from his or his colleagues’ mistakes. He/she is certain can deliver bug-free software even under time pressure. Of course, the condition is to know unit testing so well that it is the default way of work even when under pressure.
„The Art of Unit Testing”, Roy Osherove
„xUnit Test Patterns”, Gerard Meszaros
„Growing Object Oriented Software Guided by Tests”, Steve Freeman, Nat Pryce
The most well known case study on unit testing took place at Microsoft. More details here.