Performance testing is a task often left to the last stages of software development, after the code is checked in, passed QA for functional testing, and no longer fresh in the minds of developers who wrote it. With JUnitPerf, you can leverage your existing suite of JUnit tests to get performance measurements while you're developing. The framework provides easy to use decorator classes for creating:
- Timed tests to measure execution time
- Load tests for measuring how your code runs with multiple requests
Performance tuning should always be done with a performance goal in mind, and JUnitPerf shouldn't be the end all measurement tool for the performance your code. Instead it is more useful for
- Getting a basic idea of where your execution bottlenecks may lay
- Evaluating the performance difference between multiple technical solutions
- Uncovering immediate concurrency issues
JUnitPerf uses decorator classes to wrap existing JUnit tests. This makes it possible to add performance measurements even to HttpUnit or DbUnit tests.
In our examples we will use the following unit test
public class SimpleUnitTest extends TestCase {
...
public void test1() {
... //some operations
}
...
A TimedTest can be used to provide a time limit for your running tests. Tests will fail if they take longer than the given time limit to execute. This is useful if you know you don't want to exceed a certain time constraint. Here we are running all tests within the SimpleUnitTest class as a suite with a time limit of 1 second.
public class SimplePerfTest {
public static void main(String[] args) {
int timeLimit = 1000;
Test timedTest = new TimedTest(new TestSuite(SimpleUnitTest.class), timeLimit);
junit.textui.TestRunner.run(timedTest);
}
}
You can use LoadTest to create a load test with a specified number of threads(JVM will constrain the upper thread limit) running your test case either simultaneously or at randomly dispersed start times. We will do the simplest example of running a test suite with 100 threads concurrently.
public class SimplePerfTest {
public static void main(String[] args) {
int load = 100;
Test loadTest = new LoadTest(new TestSuite(SimpleUnitTest.class), load);
junit.textui.TestRunner.run(loadTest);
}
}
In a more complex example of using LoadTest, we can simulate 100 threads running our test code, with a random delay before the addition of each concurrent thread. In addition to the RandomTimer class to specify a delay, there is also the ConstantTimer to specify a constant delay between thread addition.
public class SimplePerfTest {
public static void main(String[] args) {
int threads = 100;
int delay = 10;
int randomVariation = 20;
Timer randomTimer = new RandomTimer(delay, randomVariation);
Test loadTest = new LoadTest(new TestSuite(SimpleUnitTest.class), threads, randomTimer);
junit.textui.TestRunner.run(loadTest);
}
}
Lastly, we can combine both TimedTest, LoadTest, and RepeatedTest (part of JUnit) to string together a rather sophisticated concurrent performance test. Here's a test that concurrently runs your unit test in 100 threads, each repeating 10 times before ending, with a time limit of 2 seconds for the test to complete. Threads are being added after every 10ms by using the ConstantTimer.
public class SimplePerfTest {
public static void main(String[] args) {
int timeLimit = 2000;
int threads = 100;
int delay = 10;
int repetitions = 10;
Timer randomTimer = new ConstantTimer(delay);
Test timedTest = new TimedTest(new TestSuite(SimpleUnitTest.class), timeLimit);
Test repeatedTest = new RepeatedTest(timedTest, repetitions);
Test loadTest = new LoadTest(repeatedTest, threads, randomTimer);
junit.textui.TestRunner.run(loadTest);
}
}
JUnitPerf tests are great for quickly spotting performance problems early in the development cycle, requiring very little extra work if your regular coding practice already includes JUnit, and extensions of JUnit such as HttpUnit and DbUnit. With multithreaded LoadTests, it can also be helpful in uncovering concurrency issues normally difficult to discover with code inspection or other forms of testing. There is unquestionable value in performing performance testing on a fully functional and integrated system, but you can cut down both cost and risk to your project by validating the performance of your designs early, especially architectural decisions that may be hard to change later on (though being Agilists, we always try to make change easy. But that's a topic for another day).
Resources:
http://clarkware.com/software/JUnitPerf.html
Home of JUnitPerf
Home of JUnit
Home of DbUnit
http://httpunit.sourceforge.net/
Home of HttpUnit
http://www.ibm.com/developerworks/java/library/j-cq11296.html
Performance testing with JUnitPerf
