Azure DevOps Automated Testing Simplified

Automating testing in Azure DevOps can be a game-changer for your development team.

With Azure DevOps, you can create and run automated tests for your code, catching bugs and issues early on. This saves you time and reduces the risk of errors making it into production.

Automated testing also helps to ensure that your code is working as expected, giving you confidence in your product's quality. You can run these tests on a schedule, or even when code changes are made.

This allows you to catch issues before they become major problems, and makes it easier to maintain a high level of quality in your product.

To set up Azure DevOps, start by creating a new project, which will house both execution pipelines and a code repository for storing scripts. I created a new Azure DevOps project named ContosoOutdoors for demonstration purposes.

For the repository, follow the documentation at https://docs.microsoft.com/en-us/azure/devops/repos/git/create-new-repo?view=azure-devops to create a new repository. Be sure to choose the Node option in the .gitignore dropdown to ensure only the correct items in your code get committed.

In creating the repository, I had two repositories: ContosoOutdoors and ContosoOutdoorsAutomation. ContosoOutdoors is the default repository created, and ContosoOutdoorsAutomation will house the test automation scripts.

To set up a demo project, start by opening your console or terminal. Create folders for your Main and Test Projects, and then create a Test Project in the 'test' folder and build the project.

The resulting structure will have a similar layout to the example, with the main difference being the project name. You can create an analogue for your project by replacing 'TechFabric' with your project's name.

Add a simple function to your project, which you can cover and test with unit-tests. Create an additional class for the Main 'TechFabricSln' Project, such as the 'Bought' class.

Now, let's create a quick check for a declared variable in the Test Project. Write down the code that verifies something you'd like to test, like the 'isBoughtBy' method in the 'Bought' class.

As a final step, create a GIT repository and push your code. This will allow you to track changes in a source control tool, rather than on your local machine.

To run your unit tests, select your Project's repository and run Selenium Test locally. If you see green lines, you can be pretty sure you've made the first step in protecting your project from uninvited guests.

Before you can start setting up Azure DevOps, you need to prepare the environment. This involves ensuring your testing framework and test projects are properly set up.

You'll want to choose a testing framework compatible with Azure Pipelines. For .NET projects, options include NUnit, xUnit, and MSTest.

Make sure your test projects are organized and ready to be executed. This will save you time and effort in the long run.

The specific testing frameworks you'll need will depend on the type of projects you're working with. For example, if you're working with Java projects, you'll need JUnit, while Python projects will require PyTest.

Creating work items in Azure through testRigor is a straightforward process. You can directly create work items for errors seen through testRigor.

To start, go to the Errors section in testRigor and click Create Work Item in Azure for the error you want to create a work item for. This will open a form that you need to fill out.

Select the relevant Work Item Type and then click Create at the bottom. The work item will then be created in Azure.

Switch to your Azure DevOps account, go to Dashboards -> Work Items, and you'll see this newly created work item at the top of the list.

Automated testing in Azure DevOps is a game-changer for software quality and readiness.

The first step is to set up test projects that contain the test scripts and are configured to use a testing framework compatible with Azure DevOps, such as xUnit, NUnit, or MSTest for .NET applications.

You'll want to use a testing framework that integrates well with Azure DevOps to ensure smooth execution of automated tests.

To run automated tests, you need to create a build pipeline in Azure Pipelines that's triggered automatically whenever code is committed to the repository.

This pipeline configuration includes tasks to build the application and run the automated tests, providing instant feedback on the software's quality and readiness.

Automated tests can be integrated into release pipelines to validate applications in different environments, such as staging and production.

To get started with automated testing in Azure DevOps, you'll want to familiarize yourself with the following:

By following these basics, you'll be well on your way to leveraging automated testing in Azure DevOps to improve software quality and reduce testing time.

You can create a build pipeline in Azure DevOps using a YAML file, which defines the pipeline's steps. This file can be created in the repository, and it includes steps to install dependencies, build the solution, and run tests.

A build pipeline compiles code, runs tests, and produces artifacts. To create a build pipeline, you can define build steps, such as installing dependencies, building your solution, and running your tests. For example, a .NET Core project can include test execution using the dotnet test command.

In Azure DevOps, you can create a build pipeline by selecting a source repository, choosing a build template, and adding tasks to the pipeline. For example, you can add an npm install task to install dependencies, a Copy Files task to copy project files, and a Publish Build Artifacts task to publish the build artifacts.

To create a release pipeline, you can start with an empty template and add tasks to execute your tests. For example, you can add an NPM task to run your Selenium tests, and a Publish Test Results task to publish the test results. The release pipeline will act as your Selenium test runner, executing your tests and publishing the results.

Here are the basic steps to create a build and release pipeline in Azure DevOps:

To create a build and run on continuous integration, you'll need to login to Azure DevOps and go to 'Pipelines' -> 'Builds'. From there, press 'New pipeline' to start the process.

In Azure DevOps, you'll read a configuration file and execute the steps as described. This will help you create a build procedure that includes your test runs and makes a good basic fundament for further quality assurance processes.

Press 'Save and Run' button with 'commit directly to the master branch'. After the build procedure is done, you can see the builds results in Logs.

The 'DotNetCoreCLI' command is used in the DotNetCoreCLI task, which is nothing more than the 'dotnet test' command. Detailed information about your Test Run for the build can be found in 'Tests' block of the build.

You can also check detailed report about all your test runs in Test Plan -> Runs. Now, your Build Pipeline contains at least one Unit test that works every time you run the Build, verifying if there are any unexpected changes or not.

Here are the steps to create a build pipeline:

1. Create a YAML file for the pipeline (e.g., azure-pipelines.yml).

2. Define build steps to install dependencies, build your solution, and run your tests.

3. Use the PublishTestResults task to make test results visible in Azure DevOps.

Note: You can adjust the script commands based on project requirements and the programming language used.

To import your Azure DevOps test cases into testRigor, you'll need to create a test suite in testRigor and link it to your Azure Test Plan and Azure Test Suite.

Make sure your testRigor test suite is correctly linked to the right Azure Test Plan and Azure Test Suite, as this will allow you to see an import button in testRigor.

Click the Import test cases from Azure DevOps button, and choose the test cases with test steps you want to import through the popup.

Selenium is an open source test framework for web applications that supports any popular browser and can be run on almost every operating system.

To implement UI tests using Selenium, you'll need to add Selenium references, driver, and test to your existing project. This involves opening your solution in an IDE, going to the .Test Project, and managing NuGet packages.

You'll need to add packages such as Selenium.WebDriver.ChromeDriver, Selenium.Support, Selenium.WebDriver, and Microsoft.TestPlatform.TestHost to your .Test Project.

Here are the packages you'll need to add:

These packages will enable you to run UI tests using Selenium.

Selenium is a powerful tool for automating web browsers, and it's often used in conjunction with Azure Pipelines to create a continuous delivery pipeline. Selenium supports any popular browser and can be run on almost every operating system.

To integrate Selenium with Azure Pipelines, you need to create a new pipeline. This involves opening the Releases page in the Azure Pipelines section, clicking on the 'New pipeline' button in the Releases block, and then connecting to Team Services/Azure.

You can add Selenium to an existing test project by opening the solution in an IDE, going to the .Test Project, and managing NuGet packages. You'll need to add packages such as Selenium.WebDriver.ChromeDriver, Selenium.Support, Selenium.WebDriver, and Microsoft.TestPlatform.TestHost.

To write a Selenium test, you'll need to create a new class for Selenium tests, such as 'SeleniumTest.cs'. In this class, you can add a test that finds a specific page and verifies its content. For example, you could write a test that finds the Microsoft page and verifies that it contains the 'Windows' menu.

Here are the steps to add Selenium to a test project:

After adding Selenium to your test project, you'll need to publish the chromedriver for your test project into the .csproj file. This will allow you to get the chromedriver into the artefacts after the project is published.

To save and export your test script, you'll need to choose Export from the ellipses next to your test scenario.

Make sure to select JavaScript Mocha as the language and uncheck all the boxes on the modal that appears.

Save your file as test.js within your project.

You'll also need to update your package.json file to change the test value to save all output to a file called result.xml.

This will allow Mocha to run your test and export it to common test result formats.

Once you've made these changes, you can commit and push to Azure DevOps.

Using AI-based tools can make a huge difference in automated testing.

One of the biggest challenges with tools like Azure DevOps is that you need to be proficient in coding to write test scripts, which isn't a skill manual testers and business stakeholders typically have.

You need a common platform for team members like developers, testers, product managers, and business analysts to collaborate on, especially for end-to-end test cases. This platform should be user-friendly, not just for coders.

Training teams to understand the intricacies of Azure can be tough, especially for those who don't work with code daily.

Selenium is often used, but it may not be enough to handle modern website verification scenarios.

Continuous Integration is a game-changer for Azure DevOps automated testing. It allows you to run tests automatically whenever code is committed or according to a scheduled trigger. This helps catch unexpected problems before pushing the build to test or customer environments.

To get started, log in to Azure DevOps and go to Pipelines > Builds. Press the New pipeline button to create a build procedure that includes your test runs. Azure DevOps will read the configuration file and execute the steps as described.

The build procedure is done when you press the Save and Run button, committing directly to the master branch. You can see the build results in Logs, especially the 'DotNetCoreCLI' command, which is nothing more than the 'dotnet test' command.

The detailed information about your test run for the build can be found in the 'Tests' block of the build. You can also check the detailed report about all your test runs in Test Plan > Runs.

Here's a step-by-step guide to creating a build pipeline:

By following these steps, you can create a build pipeline that includes your test runs and sets the foundation for further quality assurance processes.

To create a release pipeline in Azure DevOps, you'll need to add tasks to execute your tests. Start by pointing to your artifact from your build pipeline, selecting "Build" as the source type, and configuring the settings to point to your build pipeline.

You can add a new stage to your pipeline, like "DEV Test", which will be used for running tests on demand. This is useful for management and moderation purposes.

In your release pipeline, you'll want to add tasks like the VSTest Platform Installer, VSTest Task, and Publish Test Results. These tasks will help you run your tests and collect data for statistical analysis.

To run your UI Selenium tests, you'll need to select the "Run with options" menu in the "Execute" block of your Test Plans. Choose the build and stage where you want to run your tests, and select the "Dev Tests" stage.

Here are the tasks you'll need to add to your release pipeline:

These tasks will help you run your tests and collect data for statistical analysis.

In your release pipeline, you can also use the "Continue on error" option to prevent the pipeline from stopping if one or more tests fail.

To execute your Selenium tests, you'll need to use the npm test command. This command will run your tests and save the results in an XUnit format.

Here's a summary of the tasks you'll need to add to your release pipeline:

By following these steps, you'll be able to create a release pipeline that executes your UI Selenium tests and collects data for statistical analysis.