At the time of this post, I have taught unit testing for 5+ years, to a few hundreds of developers, both in classroom and in their own environment, on their own code, be it Java, C#, C++, PHP, Python, Javascript, Objective-C or ActionScript. No matter the technology or the environment, the problems they encountered were very similar. I decided to write some of them today, hoping you will benefit from this list.
1. Writing the wrong type of tests
You know you have this problem when your tests are long, they fail often for strange reasons and you still find lots of bugs in production.
This is probably the oldest and most documented problem related to automated testing in general. People tend to write too few unit tests and too many system tests. The usual reason they give is “this is how we can be sure everything works”.
This argument seems very logical. Unfortunately, things are not that simple.
Any system test involves a lot of moving parts. Such tests are slow, brittle, very difficult to understand and to debug. If it wasn’t enough, JB Rainsberger explained very clearly that millions of tests are required in order to cover a software application through system tests. The number increases combinatorically, because they must check combinations between behaviors of various parts of the system.
From my experience, this problem is due to the lack of knowledge and misunderstanding of test doubles and of writing simple tests.
2. Misunderstanding test doubles
You know you have this problems if your mocks are more complicated than the production code. Actually, you have this problem when your grandmother can’t understand your mocks.
Test doubles are an excellent mechanism for simplifying tests, although a strange one. Another complicating factor is that the same type of test double is named differently depending on the book.
I defined a glossary for my courses so that everyone understands the same thing. The simplest form you need to know is:
- Stub – a test double that returns values. It is used for isolating state tests.
- Mock – a test double that verifies method calls. It is used for implementing collaboration tests.
Stubs and mocks are very easy to write manually. They are slightly easier to write with a mocking framework. My favourite mocking frameworks are: moq for .NET, GMock for C++, mockito for Java, Python, PHP, Javascript.
Test doubles are so powerful that they are almost scary. You might need to write tests for a while using them before getting comfortable to use them in real scenarios. Once you do, there’s no stopping to your testing anymore.
3. Test names don’t communicate intent
You know you have this problem if you look at the list of tests and you don’t understand what the application is doing.
Naming things in software is notoriously difficult. Naming tests might be one of the most important things you do as a programmer.
I often see tests named like Test1, Test2, Test12312321. Or CreateUser (when?). Or ThrowsNullReferenceException (again, when? And why should I care?)
I’ve used three strategies for naming tests:
- Phrase: UserIsCreatedWithUsernameAndPassword, UserIsCreatedWithFacebookCredentials, CannotCreateUserWithEmptyUserName etc.
- Given / When / Then replaced by underscores: UsernameAndPassword_CreateButtonIsClicked_UserIsCreated
- BDD framework style where you name the feature and the behavior.
No matter which one you use, the name you give is important. Here are some hints:
- Never give misleading names to tests.
- Describe what happens and when. If you only describe what happens, you won’t know enough about the test, especially in a large application.
- Avoid technical details. Eg. “CannotCreateUserWithEmptyUserName” instead of “InvalidParameterExceptionThrownOnEmptyUserName”
4. Debugging tests often
You know you have this problem when, if a test fails you jump directly in the debugger.
Tests should be simple and focused. When a test fails, you should find out in less than two minutes what the problem is. You can do it if the test name describes the behavior, the test code is very short and easy to read and the assert message describes the error.
I only resort to debugger about once or twice a year, when really strange things happen in tests. Otherwise, I either write more tests or simplify the existing ones until I can understand the problem.
Debugging is hard and requires a lot of brain power. If the computer can do more work to give you more data so that you can solve the problem, then let it do it.
5. Allowing test code to rot
You know you have this problem when you clearly see that the tests are hard to understand but don’t change them.
When I don’t understand code, I refactor it until I do (and write tests for it as well, but that’s out of scope now). I do this even more with tests.
If I can’t put enough energy into the code that checks my code, then I’m wasting my time writing it. Either write tests or don’t. If you write them, make sure they are good. Test code quality should be as high as production code quality; it’s equally important (Robert C. Martin says test code might be even more important than production code). Treat it with care and love.
So, these were my 5 tips for unit testing. I hope they help.
Would you like to have better unit testing? We can help with:
- Attending the “Unit Testing Core Practices” workshop, also available in-company
- Attending the “Test Driven Development (TDD) ” workshop, also available in-company
- Attending the “Acceptance Test Driven Development” workshop with Markus Gärtner
If the situation is more complex, we would be happy to create a customized package for your needs.
