One of the main objective for Sonar in 2011 is to go a step further in the support of Continuous Inspection. Indeed, prior to version 2.5, Sonar could already take a snapshot of the overall quality of an application and view the evolution of quality measures across snapshots with the TimeMachine service. But this was not sufficient to provide at quick answers to some very valuable questions such as :
what changed in my application over last 30 days ?
did quality improve during version 2.7 software increment ?
which violations have been created since 1st of January and by who ?
how much is new code covered by unit tests ?
which projects have increased their technical debt during last 3 months ?
You probably know already that JaCoCo is the most performant code coverage engine. But you might not know that you can now combine it with Sonar to assess the code coverage by integration tests. This was the most voted Sonar issue (SONAR-613) and the latest version of the Sonar JaCoCo Plugin solves it. I am now going to explain how.
By default, Sonar embarks two tools to calculate code coverage by unit tests on java projects : Cobertura and Clover. But last week, we also released plugins for two other coverage tools : Emma and JaCoCo. Both plugins are available in the Sonar Plugin Library. So I thought it would be a good time to compare all of them and share the results with the community.
A debate occurred within the team about a year ago to decide whether Sonar should enable to report on error details of failing unit tests. One year later, it sounds bizarre that this debate could take place, given the comprehensive solution provided by Sonar to review and improve unit tests and code coverage.
Here is a one minute video to summarize this one year of effort :
Of course, when you start developing a new application, the Sonar web server is not the place to start from to do Test Driven Development; Sonar is not an IDE, Sonar will not create unit tests for you and Sonar won’t replace your precious Agile developers. But once the first bunch of code has been commited, you should certainly use it to monitor and review the code health and more particularly the unit tests and code coverage.
At a glance, you get it all on the Sonar project dashboard : total number of tests, tests in error, tests in failure, skipped tests, code coverage and total duration of unit tests. On any of those measures, you not only get an indication of the short term trend through the usual green and red arrows but also alerts if any threshold has been reached.
Then, to get the bigger picture, you can access the TimeMachine to see evolution of the latest versions/snapshots. In the following example, number of unit tests and code coverage are increasing whereas unit tests duration decreases, which is pretty good :
Obviously, when you have seen all the numbers, you will want to understand how it breaks down. So on every measure, you can click to drill down.
In this example, you see that a unit test fails in FindbugsRulesRepositoryTest.java but what is wrong ?
And what’s true for unit tests in error is also true for code coverage: drill down and then display code coverage detail:
In fact, Sonar offers multiple entry points (aka hunting tools) to improve code coverage. That really depends of what you are looking for.
1. To make sure coverage is homogeneous across packages, simply use the treemap
2. To understand why a module has low coverage, drill down on the percentage
3. For quick wins, check the classes cloud: any big red class is a target
4. To reduce risk on the project, the best entry point is the “Most complex & less tested files” in the Hotspot where files that have the highest uncovered cyclomatic complexity are shown
To get all that, you just have to :
1. Download Sonar
2. Unzip Sonar
3. Launch the Sonar web server (sonar.xx start)
4. Launch the Sonar Maven plugin (mvn sonar:sonar)
