What is effective testing? Most professional developers know that a certain amount of test coverage is required, but many don't know how to write tests that are effective and helpful. Here are a few tips to help you write better tests.
Write tests first
Often, when a developer is asked what are tests for, the answer is, “they verify that my code works.” From that perspective, it seems to make sense that many people write their tests after the actual code is complete to make sure that it runs, but there are some problems with this approach.
First, when a developer writes the test after writing the code, he/she has never had the chance to see the test fail and then subsequently pass (once the code is written). Never seeing a test fail may mean that it is just working by coincidence. The developer sees the green bar and assumes everything is good, but that green bar is actually masking a bug.
Second, it’s not a rare case that the code is untestable. For example, if it has a lot of dependencies, it can be difficult to write a good test case for it. If somebody wants to test it, he/she will have to spend a lot of time breaking these dependencies (but quite often, people just give up and decide to live with the code being completely untested).
Third, this approach leads to trouble when a code base grows beyond a certain size. Developers often forget to update some test cases or they fix failing tests in an inappropriate way just to make them pass. This leads to rot in the test code base, which is a huge problem. After a while, the amount of effort to keep a rotten test code base in sync with the actual system will be more than the effort required to update the actual code and it can eventually slow down the whole process so much that a team has to abandon their tests completely. Now, they can’t be sure that their changes will not break something, so no refactoring can be done and they have to do manual regression testing after each change.
Last but not least, by writing tests after the code, you usually test just methods, not behaviour. This is not effective, because tests should focus on a valuable behaviour that the class under tests provides. So one needs to test the features of the code to make sure that it works as expected and can be used efficiently in collaboration with its clients instead of just covering all methods with tests one by one - quite often you need to call several methods of your class in a row to collaborate with other objects.
On the other hand, if you write tests first, following the test-driven development (TDD) principles, you get a whole bunch of benefits. First, your tests become a runnable specification that describes what your code does and not how it does it. As a result, you start thinking in terms of features, not methods, which helps maintain a necessary level of abstraction.
Writing tests first also makes them more readable and maintainable; you think about a certain scenario and this allows you to keep your test clean from unnecessary details. It also forces you to think about the dependencies. Because you have to pass all the dependencies to the object under test, you will get a clear indicator whether your design is good or bad. If your test becomes too large and complicated, the class under test probably has too many responsibilities and you need to decompose it.
Make sure tests are readable
Another key factor in creating effective tests is readability. A test that is easy to read is easy to maintain. You should always apply the same code standards to your test code as you apply for production code. Follow the common clean code rules: no magic numbers, good and explanatory variable naming, no nested ifs, etc. “Clean Code: A Handbook of Agile Software Craftsmanship” by Robert C. Martin is a great guide on how to write clean and readable code.
Test methods should be small -- ideally 10 to 15 lines -- and contain just a few asserts (ideally one). It should be very clear what the test does and what atomic scenario is being executed. If your test is long and has lots of assertions, you will have to spend a lot of time just figuring out what it actually does. Your assertions should always be self-explanatory, so that you can easily understand the reason of failure.
assertEquals(“Price discount hasn't been applied”, expectedPrice, actualPrice);
Also, the test method should have a very clear and descriptive name that states what feature is being tested:
public void testCheckout1() // BAD
public void testCheckoutWithOneCartItemAndDiscountAppliedGivesFreeShipping() // GOOD
And don’t be afraid to use long method names. You are not going to call these methods anywhere, but you will be able to understand what kind of scenario is being tested at a glance.
Use test builders
Don't duplicate creation of test objects inside your tests. Instead, use test object builders to help you easily create and maintain families of such objects. By using the builders you will greatly increase readability and remove duplication from your tests. An example can look like following:
List<Movie> movies = Arrays.asList(
MovieBuilder.movie().withTitle("Blade Runner") // <- here's the builder being used
.withAddedActor("Harrison Ford")
.withAddedActor("Rutger Hauer")
.build(),
MovieBuilder.movie().withTitle("Star Wars") // <- ... and also here
.withAddedActor("Carrie Fisher")
.withAddedActor("Harrison Ford")
.build());
There’s an Eclipse plugin called Fluent Builders that can generate builders automatically from any class, saving you tons of time.
(For more on test builders, see “Growing Object-Oriented Software, Guided by Tests” by Steve Freeman and Nat Pryce.)
Use a consistent structure
All of your test should have the same canonical structure:
Setup (prepare context and environment)
Execution (trigger the tested behaviour)
Verification (check that results are what we expect)
Teardown (clean up everything that can influence other tests and release all resources)
Note that last point. This means that all the tests should be completely isolated from each other. Remember that a chain is as strong as its weakest link. If you create a chain of tests and one test in the chain fails, all the remaining tests will automatically fail. Would the other tests have passed if that one failure had not occurred? You won't know. Better to keep your tests completely decoupled and independent.
Also, do not load data from hardcoded location in the file system, this will ensure that your test suite can be run in different environments under different operation systems and this will increase the maintainability of the test suite.
/home/ynovikov/workspace/project/src/test/resources/config.properties // NO. Absolute path is used
../test/resources/config.properties // YES. Relative path is used
Assertions should be precise - assert only those results that are triggered by test scenario and are not covered by other tests. This will make your test bas more robust.
Try to use mocks only for something that you can’t change, don’t use them as stubs. There’s a good article by M. Fowler about that.
Make sure that the code under test satisfies SOLID principles. If you do that together with writing your test first, you are very likely to product code that is easy to test, change and maintain.
Very important: your tests (not only unit, but acceptance and integration tests as well) should be fully automated so that you can make running your test as a part of continuous integration process. This will ensure that your build is always in a good and working state.
One more thing to remember – effective unit tests should be fast. This will allow you to execute them very often, maybe after every single change you make to the code. This will provide you fast feedback if your changes broke something so that you can very quickly locate and fix the newly introduced bug.
If you start developing a user story, write acceptance test(s) first and don’t forget to follow TDD when you start implementing the feature itself. Each acceptance test suite should include a happy path scenario (when everything works as expected) and tests for cases when something goes wrong or for alternative behaviour.
Write integration tests to ensure that your code works correctly with some third-party libraries or services that you can’t change. All integration points with such things like payment gateways, geoip providers, persistent mechanisms etc. should be covered by integration tests.
If you’ve just started a project from scratch, the first thing you should do is to build a “walking skeleton” – the minimal possible configuration of your product that can be built, deployed and tested in a continuous integration environment against all kind of tests (unit, acceptance, integration). For example, if you develop a web application, it can just show you a page that reads a couple of records from database and displays them (and you create all kinds of tests for this functionality). This will ensure that your test environment and test frameworks are up and running, so that you can be sure that they are working correctly and ready for serious work.