Azure DevOps Code Coverage and Automated Builds Explained

Azure DevOps Code Coverage and Automated Builds are two powerful tools that can help you ensure your code is reliable and efficient. Code coverage measures the percentage of your code that's being executed during testing, while automated builds run your tests and build processes automatically.

Code coverage is a crucial aspect of software development, as it helps you identify which parts of your code need more attention. With Azure DevOps, you can set up code coverage metrics to track the percentage of code executed during testing. This can be done using tools like SonarQube, which integrates seamlessly with Azure DevOps.

Automated builds, on the other hand, save you time and effort by running your tests and build processes without manual intervention. In Azure DevOps, you can configure automated builds to run on every check-in or at specific intervals, ensuring that your code is always up-to-date and reliable.

Code coverage is a measure of how much of your code is executed during automated testing, typically expressed as a percentage.

It helps identify which parts of the code are missing tests, allowing you to prioritize testing efforts.

The goal of code coverage is to ensure that at least 80% of the code is executed during testing, as recommended by the SonarQube plugin.

This percentage is not a hard and fast rule, but it provides a good starting point for improving code quality.

Code coverage can be measured at different levels, including line, branch, and method coverage.

Line coverage, for example, measures how many lines of code are executed during testing.

Branch coverage, on the other hand, measures how many different paths of execution are tested.

Method coverage measures how many methods are executed during testing.

Different programming languages have different code coverage tools, such as OpenCover for .NET and Emma for Java.

These tools provide detailed reports on code coverage, helping you identify areas that need more testing.

Code coverage is an essential aspect of ensuring the quality and reliability of your code.

Configuring code coverage is a straightforward process that can be done in a few easy steps. To change the default settings of the code coverage experience for pull requests, you must include a configuration YAML file named azurepipelines-coverage.yml at the root of your repo.

The settings that can be changed in this file are the status, target, and comments. The status setting determines whether code coverage status checks should be posted on pull requests, the target setting sets the target threshold value for diff coverage, and the comments setting determines whether a comment containing coverage details for each code file should be posted in the pull request.

Here are the possible values for each setting:

Enabling code coverage in an automated build is a crucial step in ensuring your code is thoroughly tested. You can run unit tests and collect code coverage from your YAML Pipelines by editing your YAML to include the "dotnet test" command with the argument –collect:"Code Coverage".

To run these tests and collect code coverage, you'll need to edit your YAML pipeline. This is a straightforward process that requires minimal technical expertise.

The "dotnet test" command is used to run unit tests, and the –collect:"Code Coverage" argument is used to capture code coverage information. This information will then be displayed in the build summary in Azure DevOps.

In Azure DevOps, you'll be able to view the test results and the latest code coverage percentage in the build summary. This provides a clear picture of your code's test coverage and quality.

To change the default code coverage settings for pull requests, you must include a configuration YAML file named azurepipelines-coverage.yml at the root of your repo.

The settings that can be changed are listed below:

To fix the default pull request code coverage settings, which are turned on but have the setting for displaying results set to off, add a file named azurepipelines-coverage.yml to the root of your Git Repository and set the comments value to on.

Configuring code coverage involves specifying the necessary inputs to collect and publish accurate results. This includes identifying the code coverage tool and its associated summary file.

The code coverage tool can be either Cobertura or JaCoCo, and it's specified using the codeCoverageTool input. This input is required and must be set to one of the allowed values, either Cobertura or JaCoCo, with JaCoCo being the default.

The summary file location is another required input, specified using the summaryFileLocation input. This input should point to the path of the summary file containing code coverage statistics, and it can contain minimatch patterns.

To specify the path to source files, you can use the pathToSources input. This input is optional but required when coverage XML reports don't contain an absolute path to source files. For example, JaCoCo reports don't use absolute paths, so you'll need to specify a pattern like $(System.DefaultWorkingDirectory)/MyApp/src/main/java/.

To add code coverage to your Azure DevOps pipeline, you need to add the --collect argument to the test command, which accepts a data collector like Code Coverage or XPlat Code Coverage.

You can exclude test projects from code coverage by creating a .runsettings file and specifying the test assemblies to include or exclude. This can be done by creating a file called coverage.runsettings in the root of your repository with the necessary content.

After running the pipeline, you can view the code coverage details on the overview page, including the percentage of code coverage. You can also download the coverage file and open it in Visual Studio to see which code is covered and not covered by tests.

Code coverage status is posted from a pipeline, following the naming convention {name-of-your-pipeline/codecoverage}. This status check evaluates the diff coverage value for all the code files in the pull request.

The default threshold for code coverage is at least 70% of changed lines being covered by tests, but you can change this to a value of your choice. The diff coverage threshold target can be configured in the settings.

You can view the % diff coverage value for each of the files by turning on details in the configuration section. This posts details as a comment in the pull request.

Lines that are changed are annotated with coverage indicators to show whether those lines are covered. This makes it easier to see which lines of code need more attention.

The code coverage for pull requests feature is currently available only for Azure Repos.

Xplat Code Coverage has some distinct differences compared to other data collectors. It automatically excludes test projects from code coverage, eliminating the need for a .runsettings file.

One notable difference is that Xplat Code Coverage doesn't display results within the Azure DevOps pipeline. Additional steps are required to publish code coverage results, using the PublishCodeCoverageResults task.

Xplat Code Coverage generates a coverage.cobertura.xml file, which can't be opened or downloaded in Visual Studio. A third-party tool like Coverlet is necessary to view the results.

Code coverage requirements can be tricky to set up, but it's a great way to ensure your code is solid. You can change the default code coverage check in a pull request to make it required by adding a Status Check to your Git Repository's branch policies.

To do this, you'll need to click on Project settings, then Repositories, and finally your repository name. This will take you to the Settings page for your repository. From there, click on the Policies tab and then the Main branch to bring up the policy editor.

You'll see a section called Status Checks, where you can add a new status check for code coverage. Click on the + button and select the status policy named "{build-pipeline-definition-name}/codecoverage". This will ensure that code coverage is a required step in your pull request process.

If you want to strictly enforce a minimum code coverage percentage, you can use the Build Quality Checks extension. This extension allows you to set a target percentage and fails the pipeline if it's not met.