A while back I was trying to do some initial tries at Ruby as a scripting language with JRuby. That means that I had a Java system, but needed to do some util scripting for the system, and wanted to avoid those pesky bash scripts. Because I had a Java system and Maven infrastructure in place I really wanted it all to work nicely with that. But the whole Ruby and gem ecosystem threw me a bit off. Long story short: I never quite figured out how to do that and had to move on getting things done.
Madaspeak has a nice post showing how you can run Cucumber in Maven as tests for your Java classes. Cucumber is an interesting BDD framework from Aslak Hellesøy, and in the process you can also learn how to enable running of JRuby in Maven.
This is probably fairly well known, but I couldn’t find any doc on it when I searched so I’ll put it up here.
First off: Don’t load your Spring Context in your tests unless you absolutely have to. Some integration tests should load it, but keep it to a minimum. Loading the context is expensive, especially if you load up Hibernate and maybe H2 in that context.
Avoid loading the context is important both for speed and design. I have seen way too many tests where the context is loaded just because someone don’t want to do mocking. Besides, if you have to do a lot of mocking your design is usually too coupled, and should be changed.
So how can you speed up your tests in Maven?
Spring will cache the context and make sure it is only loaded once. But due to details I have not studied; in Maven, this only works for the tests that are in the same suite. It should work with fork=once on the surefire-plugin, but for some reason we need the suites too. Test which one works for you.
Drawbacks? If there is state in your context (database, stateful services) you will have created a dependency between your tests. That’s why, when it comes to database-testing you should use the AbstractTransactionalSpringContextTests which will automatically roll back your changes at the end of your tests. If state is modified in a way that has nothing to do with your database use setDirty() to signal to Spring that the context should be recreated for the next test. Of course then you get the time penalty of recreation.
I read this post about Clover and using it to minimise the number of tests run. A nice idea, so I decided to have a go at it.
What it does is use the test-coverage that it was originally written to do, to figure out which tests exercise which classes. So when you change Class1 and Class2 it knows which that Test1, Test2, Test3 and Test4 need to be rerun to check if you have broken anything.
I did an initial test on our build which takes almost 10 minutes. I’m of the slightly paranoid type so I like to run the full build to verify my changes, at least before I check in. With changes in one class Clover figured it should rerun about 20 tests, and ran in about 4 minutes. That’s 6 minutes saved many times a day for each developer. We’re not using it at our build server just yet, but trying to save time for each developer before commit.
Any downsides? Of course. The initial time (mvn clean removes all info) to build is almost doubled for my project. You won’t catch all errors, and updates to dependencies won’t be caught. And there is a problem handling deleted classes. For some reason it creates a optimized-src directory which is a copy of all your sources and compiles from there. If you delete a class in your src folder it won’t be deleted in optimized-src and you could get compilation errors. After some initial tests it seems these problems are bigger in theory than they are in real day situations.
Using Clover should be no excuse bad tests though. I know all too well how I can really mess up my own tests. The long test times often stem from our inability to focus on testing the logic separately from infrastructure as databases or external services. And of course loading the Spring context is done way too much. But that’s stuff for another post.
But even if you have good unit tests there will be time to save. And everything that can keep me from getting distracted when developing is a good thing.
I’ll give this a good run until the 30 day trial licence expires, and maybe invest. Maybe we will see something similar in Cobertura too…
Update: It looks like it is also activated when doing mvn eclipse:eclipse . Because it redefines the source folders to target/clover/src-optimized this is also what is written in you Eclipse project. So be sure to have an easy way to disable running if you’re going to use this.