Azure Pipelines Environment Variables Best Practices

Setting up Azure Pipelines environment variables can be a bit tricky, but don't worry, we've got you covered. It's essential to follow best practices to ensure your pipelines run smoothly and efficiently.

Use a consistent naming convention for your environment variables. This makes it easier to understand and manage your variables, especially in large-scale pipelines. For example, using a prefix like "BUILD_" or "ENV_" can help you quickly identify the purpose of each variable.

Avoid hardcoding sensitive values like API keys or database credentials directly into your pipeline. Instead, use Azure Key Vault or a similar secure storage solution to store and retrieve these values. This way, you can keep your pipeline configuration separate from sensitive data.

Azure Pipelines Environment Variables are a powerful tool for managing variables across pipelines. You can reuse variables across pipelines by setting them once and using them multiple times.

Pipeline variables are commonly used when building executable code with the MSBuild task. They can be used to set variables like configuration, platform, and more. For example, you can set a variable called "configuration" to "debug" and use it with the MSBuild task.

Azure Pipelines runtime parameters offer more control over what values can be passed to a pipeline at runtime. You can supply different values to scripts and tasks at runtime, control parameter types, ranges allowed, and defaults.

Here are some scenarios where runtime parameters come in handy:

Pipeline variables can be specified in Azure DevOps in the pipeline UI when you create a pipeline from the YML file. You can specify defaults and/or mark the variables as "secrets." This is useful if you plan on triggering the pipeline manually and want to set the value of a variable at queue time.

You can use default variables in two ways: as parameters to tasks in a release pipeline or within your scripts. For example, you can use a default variable directly as an input to a task, like $(Release.Artifacts.{Artifact alias}.DefinitionName).

Managing environment variables in Azure Pipelines is a crucial aspect of automating your deployment processes. You can define custom variables at various scopes, including variable groups, release pipeline variables, and stage variables.

Variable groups are useful for sharing values across all definitions in a project, while release pipeline variables share values across all stages within a release pipeline. Stage variables, on the other hand, share values within a specific stage of a release pipeline.

You can use the Variables tab in the release pipeline to define and manage these variables. In the Pipeline Variables page, set the Scope drop-down list to the appropriate environment when adding a variable.

To view the current values of all variables, select Pipelines > Releases, and then select your release pipeline. Open the summary view for your release and select the stage you're interested in. In the list of steps, choose Initialize job, and then scroll down to see the values used by the agent for this job.

Here are some key facts to keep in mind:

System environment variables are a crucial aspect of managing environment variables in Azure Pipelines. You can use system variables in your scripts or tasks to call Azure Pipelines REST APIs.

The URL of the service connection in Azure Pipelines is stored in the System.TeamFoundationServerUri variable. This variable is used to call Azure Pipelines REST APIs. For example, the URL might be https://fabrikam.vsrm.visualstudio.com/.

You can also use the System.TeamFoundationCollectionUri variable to call REST APIs on other services such as Build and Version control. This variable stores the URL of the Team Foundation collection or Azure Pipelines.

To run a release in debug mode, you can set the System.Debug variable to true. This will display additional information during the release execution, making it easier to diagnose and resolve issues or failures.

Here are some key system variables:

You can use default variables in two ways: as parameters to tasks in a release pipeline or within your scripts. To use a default variable in your script, you must first replace the . in the default variable names with _.

Custom variables can be defined at various scopes, including variable groups, release pipeline variables, and stage variables. These variables can be used to avoid duplicating values and make it easier to update all occurrences with a single change.

Variable names are transformed to uppercase, with "." and " " replaced with "_", when accessed from a script. For example, Agent.WorkFolder becomes AGENT_WORKFOLDER.

To set a secret variable in a variable group, you can add secrets to the group or link secrets from an existing Azure Key Vault.

Pipeline variables are useful when you plan on triggering the pipeline manually and want to set the value of a variable at queue time. You can specify defaults and/or mark the variables as “secrets” in the pipeline UI.

If you specify a variable in the YML variables section, you cannot create a pipeline variable with the same name. If you plan on using pipeline variables, you must not specify them in the “variables” section!

Here's a list of variable scopes and their uses:

You can use a default variable directly as an input to a task, or within your scripts. To use a default variable in your script, you must first replace the . in the default variable names with _.

Dynamic logging is a powerful feature in Azure DevOps that allows you to create and update variables at runtime. This can be done using logging commands, which are essentially commands that print output to a log file.

To create a dynamic variable, you can use the logging command to retrieve a value, such as the connection string to a storage account, and then assign it to a variable. For example, you can use the "az cli" command to retrieve the connection string and then assign it to a variable called "currentUser".

You can create a variable with the value of a command output by wrapping the command in double quotes and using the $(var) syntax, but be careful not to confuse this with a bash or PowerShell variable. Instead, use the env command to create environment variables, which can be more readable and easier to manage.

If you need to make the variable a secret, simply add "issecret=true" to the logging command. This will ensure that the variable is not displayed in the pipeline logs.

One special case of dynamic logging is the "build.updatebuildnumber" logging command, which allows you to calculate and update the build number at runtime. This is particularly useful for multi-stage pipelines, where you may need to update the build number based on the output of a previous stage.

You can set secret variables in Azure Pipelines to keep sensitive information secure.

Secret variables are encrypted at rest with a 2048-bit RSA key. They are available on the agent for tasks and scripts to use, but be careful about who has access to alter your pipeline.

To set a secret variable, follow these steps: Go to the Pipelines page, select the appropriate pipeline, and then select Edit. Locate the Variables for this pipeline, add or update the variable, and select the option to Keep this value secret to store the variable in an encrypted manner. Save the pipeline.

Secret variables are not automatically decrypted into environment variables for scripts. You need to explicitly map secret variables as environment variables to reference them in YAML pipelines.

Secret variables should not contain structured data, as this can make the logs unreadable. For example, if "{ "foo": "bar" }" is set as a secret, "bar" isn't masked from the logs.

Some operating systems log command line arguments, so never pass secrets on the command line. Instead, map your secrets into environment variables.

Azure Pipelines makes an effort to mask secrets when emitting data to pipeline logs, but you still need to take precautions. Never echo secrets as output.

Here's a summary of the recommended ways to work with secret variables: