There’s an update to this article with some corrections and an updated batch script here.

In my last post I wrote about some of the nuances of using SonarQube with a Unity project. It works mostly well, but code coverage was missing.

Code coverage is a useful tool: it tells you how much of your code is covered by tests, so basically, how much of your code is reliable. It implies the presence of tests, which is sadly not that common in the game industry – this is reflected by Unity’s tools, which are adequate, but not spectacular.

Code coverage, for example, isn’t even supported until 2019.3, which at the time I’m writing this, is in beta. Code coverage is served as a package, which is a preview version. Still, we can work with this, if you’re willing to risk the instability of beta versions.

For this to work, you have to install Unity 2019.3 (at least), then open the package manager, check ‘Show Preview Packages’, and get the Code Coverage package.

So here’s what we’ll do: we start the analysis process (SonarScanner.MSBuild.exe /begin), build the solution, run some tests with code coverage enabled, and process the results.

Yes, you will need tests for this to work. Familiarize yourself with the Unity Test Framework (it is based on NUnit if you’re from a .NET background).

Unity will generate an xml file (it can also generate HTML reports too, but we’ll ignore that), which is in OpenCover format. Fortunately this is supported by SonarQube out of the box (I’m using version 8.0). To process the file, we pass /d:sonar.cs.opencover.reportsPaths="TestResults.xml" when calling begin. Note that we’re telling the scanner where to look for the file before the file even exists.

The result file will be created at projectPath\CodeCoverage\codecoverage-opencov\EditMode\TestCoverageResults_0000.xml. You will need to figure out where your Unity installation is; if you use Unity Hub (please do!), it is something like this: c:\Program Files\Unity\2019.3.0f1\Editor\Unity.exe. Replace your version accordingly. I have yet to find a (non-hacky) way to start the “appropriate Unity version”.

Next, we’ll modify the original batch file to run unit tests. Unity has a quite a few command line arguments; what we need is -runTests, -batchmode and of course the code coverage ones. But enough talk, here’s the .bat file:

@echo off

SET "scanner=c:\progs\sonar\scanner\SonarScanner.MSBuild.exe"
SET "msbuild=c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\MSBuild.exe"
SET "unity=c:\Program Files\Unity\2019.3.0f1\Editor\Unity.exe"

SET "unity_project=C:\unity_other\codecoverage"
SET "sonar_project=codecoverage"
SET "solution=codecoverage.sln"

SET "coverage=\CodeCoverage\codecoverage-opencov\EditMode\TestCoverageResults_0000.xml"

echo Start pre-processing project
"%scanner%" begin /k:"%sonar_project%" /d:sonar.cs.opencover.reportsPaths="%unity_project%%coverage%"

echo Rebuilding solution
"%msbuild%" %solution% /t:Rebuild

echo Running tests and code coverage
"%unity%" -runTests -batchmode -projectPath "%unity_project%" -testResults "%unity_project%\testResults.xml" -testPlatform EditMode -forgetProjectPath -enableCodeCoverage -coverageOptions assemblyFilters:-*unity*

echo Pushing results to Sonar Server
"%scanner%" end

Note how I set environment variables; pay attention to the apostrophes. Filenames with spaces are tricky. unity_project is the path for your Unity project. I assume you run this from within the project folder, but I use absolute paths anyway (it shouldn’t be too hard to modify this script to handle command line parameters).

My SonarQube project is also called “codecoverage”. The Unity project and the SonarQube project can have different names if you want.

Having to define both unity_project and solution feels a bit awkward, but it’s valid – I know it’s not used 90% of the time, but I wouldn’t omit it either.

coverage is a constant for now, but as stated earlier, the code coverage package is just a preview, so that may change any time.

And now, if you run your test, your SonarQube project will have accurate code coverage data. Well, it would tell you how much of your code is covered by tests, which will in turn say something about your tests.

For me, personally – it tells me to get my shit together. But, if you’re like me, you’ve already turned off coverage requirements in the SonarQube quality gate, so… ;)