I read this post about Clover and using it to minimise the number of tests run. A nice idea, so I decided to have a go at it.
What it does is use the test-coverage that it was originally written to do, to figure out which tests exercise which classes. So when you change Class1 and Class2 it knows which that Test1, Test2, Test3 and Test4 need to be rerun to check if you have broken anything.
I did an initial test on our build which takes almost 10 minutes. I’m of the slightly paranoid type so I like to run the full build to verify my changes, at least before I check in. With changes in one class Clover figured it should rerun about 20 tests, and ran in about 4 minutes. That’s 6 minutes saved many times a day for each developer. We’re not using it at our build server just yet, but trying to save time for each developer before commit.
Any downsides? Of course. The initial time (mvn clean removes all info) to build is almost doubled for my project. You won’t catch all errors, and updates to dependencies won’t be caught. And there is a problem handling deleted classes. For some reason it creates a optimized-src directory which is a copy of all your sources and compiles from there. If you delete a class in your src folder it won’t be deleted in optimized-src and you could get compilation errors. After some initial tests it seems these problems are bigger in theory than they are in real day situations.
Using Clover should be no excuse bad tests though. I know all too well how I can really mess up my own tests. The long test times often stem from our inability to focus on testing the logic separately from infrastructure as databases or external services. And of course loading the Spring context is done way too much. But that’s stuff for another post.
But even if you have good unit tests there will be time to save. And everything that can keep me from getting distracted when developing is a good thing.
I’ll give this a good run until the 30 day trial licence expires, and maybe invest. Maybe we will see something similar in Cobertura too…
Update: It looks like it is also activated when doing mvn eclipse:eclipse . Because it redefines the source folders to target/clover/src-optimized this is also what is written in you Eclipse project. So be sure to have an easy way to disable running if you’re going to use this.