Azure ML Client for Python Development and Deployment

The Azure ML Client for Python is a powerful tool for machine learning development and deployment. It allows you to easily work with Azure Machine Learning services from your Python code.

With the Azure ML Client, you can create and manage machine learning models, datasets, and experiments with just a few lines of code. This makes it a great choice for data scientists and developers who want to quickly prototype and deploy machine learning models.

By using the Azure ML Client, you can take advantage of Azure's scalable and secure infrastructure to train and deploy your models. This means you can focus on building your models without worrying about the underlying infrastructure.

You can define an environment in Azure Machine Learning in several ways, including using curated environments, Docker images, build contexts, and conda specifications.

Curated environments are predefined environments containing popular ML frameworks and tooling, and are maintained by Azure Machine Learning for bug fixes and security patches. You can use them out of the box for training or deployment by referencing a specific version or the latest version of the environment.

To create an environment from a Docker image, provide the image URI of the image hosted in a registry such as Docker Hub or Azure Container Registry. Azure Machine Learning also provides a set of CPU and GPU Ubuntu Linux-based base images with common system dependencies that you can use for your environments.

The Python Package Library is a crucial part of working with Azure Machine Learning environments. You can install the Azure ML client library for Python using pip.

To install the package, simply run the following command in your terminal: pip install azure-ml. The current version of the package is 1.17.1.

The package has been tested with Python versions 3.8, 3.9, 3.10, 3.11, and 3.12. This ensures that you can use the package with your preferred version of Python.

To get started with the package, you can import the required libraries in your Python script. The libraries you'll need are from azure.ai.ml, from azure.ai.ml.entities, from azure.ai.ml.constants, and from azure.identity.

Here are the specific libraries you'll need to import:

These libraries will provide you with the functionality you need to work with Azure Machine Learning environments.

Telemetry is a feature that collects usage and failure data about the Azure ML Python SDK when used in a Jupyter Notebook.

The SDK team uses this data to understand how the SDK is used and improve it, which helps resolve problems and fix bugs.

Telemetry data is only collected when the SDK is used in a Jupyter Notebook, and it's not collected for any other use of the Python SDK.

The telemetry feature is enabled by default for Jupyter Notebook usage and can't be enabled for non-Jupyter scenarios.

To opt out of telemetry in a Jupyter scenario, you need to pass in enable_telemetry=False when constructing your MLClient object.

Registration is a crucial step in working with Azure Machine Learning. You can store and version your models in your workspace in the Azure cloud by using the model registry.

The model registry helps you organize and keep track of your trained models. You can register models as assets in Azure Machine Learning by using the Azure CLI, the Python SDK, or the Machine Learning studio UI.

To register a model, you can use the Azure CLI or the Python SDK. These tools support two types of model registration: custom and MLflow. Custom models are trained with a custom standard that Azure Machine Learning doesn't currently support, while MLflow models are trained with MLflow and stored in a folder with specific files.

Here are the key differences between custom and MLflow model types:

Once you've registered your model, you can create a registered model from a model that's located on your local computer, an Azure Machine Learning datastore, or the output from an Azure Machine Learning job.

To install the Azure ML client library for Python, you'll need to use pip, specifically version 1.17.1.

The Azure ML client library for Python can be found on PyPI and Conda, making it easy to access and install.

You can test the package with Python versions 3.8, 3.9, 3.10, 3.11, and 3.12.

The package has been thoroughly tested with these Python versions, ensuring compatibility and reliability.

In the latest updates to the Azure ML Client, several key features have been added or improved.

The ability to update the CMK encryption key for a workspace was enabled in version 1.2.0, released on December 5, 2022. This feature is a significant enhancement for users who need to manage their encryption keys.

Other notable updates include the addition of support for Fairfax and MoonCake clouds for registry discovery baseurls, as well as the addition of OS Patching Parameters to Compute Instance.

Here are some specific changes made in version 1.2.0:

Additionally, the Registry list operation now accepts a scope value to allow subscription-only based requests, as introduced in version 1.1.0 on November 7, 2022.

In the 1.2.0 release, updating the CMK encryption key for a workspace is now possible. This feature allows for more flexibility in managing encryption keys.

The JobService class and services param to command() have been marked as experimental, indicating that they're still in development but available for testing.

A replication_count value has been added to the schema of SystemCreatedStorageAccount in Registry, which is useful for tracking storage account information.

Fairfax and MoonCake clouds are now supported for registry discovery baseurl. This expansion of cloud support makes it easier to use these services.

Variable args are now supported as pipeline input in DSL Pipeline, enabling more flexible pipeline creation.

OS Patching Parameters have been added to Compute Instance, making it easier to manage patching for compute instances.

The upper bound dependencies version for tqdm, strictyaml, colorama, and opencensus-ext-azure have been updated.

Missing "properties" have been added to batch deployment, ensuring that all necessary information is included.

The cases for system job services (Tracking and Studio) are now retained, preventing errors due to case sensitivity.

The registry begin_delete method return type has been updated.

The sweep job optional input cannot be empty, and a fix has been implemented to prevent this issue.

The bool test for output in download operation has been fixed, resolving an issue that was causing problems.

Compute Instance schedules are now being created correctly, thanks to a fix for the schedule not being created issue.

The experimental warning has been removed from Compute Schedules, making it easier to use this feature.

The input type for idle_time_before_shutdown has been changed to int, and the name has been updated to idle_time_before_shutdown_minutes.

The 1.1.0 release on November 7, 2022, brought some exciting changes to the registry list operation, which now accepts a scope value to allow subscription-only based requests.

This change is a game-changer for users who need to manage their registries on a subscription level. It's a more efficient way to work with registries, and it's great to see this feature being added.

Most configuration classes from the entity package now implement the standard mapping protocol, which means they can be used more easily and consistently.

This change makes it easier to work with configuration classes, as they now follow a standard protocol. It's a small but significant improvement that will make a big difference in the long run.

The registry delete operation has been added, and it now returns an LROPoller. The operation's name has also been changed to begin_delete.

This change makes it easier to delete registries, as the operation now returns a progress indicator. It's a great feature for users who need to manage their registries regularly.

The values of JobService.job_service_type are now using snake case, such as jupyter_lab, ssh, tensor_board, and vs_code.

This change is a minor but nice improvement, as it makes the code more readable and consistent. It's a small detail, but it's a good example of how attention to detail can make a big difference.

The Command function now accepts services param of type Dict[str, JobService] instead of dict.

This change makes it easier to work with the Command function, as it now accepts a more specific and consistent type of input. It's a small but significant improvement that will make a big difference in the long run.

Here's a summary of the changes:

You'll notice that the 0.1.0b8 release on October 7, 2022, brought some exciting changes to the Azure AI ML platform.

One of the key updates was the support for passing JobService as an argument to Command(). This opens up new possibilities for customizing your workflows.

The release also introduced a show_progress parameter to MLClient, which allows you to enable or disable progress bars for long-running operations. This can be a huge time-saver when working with complex tasks.

Another notable feature is the support for month_days in RecurrencePattern when using RecurrenceSchedule. This makes it easier to schedule tasks with recurring intervals.

Here are some of the other key updates in the 0.1.0b8 release:

These updates demonstrate the platform's commitment to making machine learning workflows more efficient and customizable.

The 0.1.0b7 release brought some exciting changes to the platform. Spark job submission is now supported.

Entity load and dump have become more flexible, as they can now accept a file pointer as input. This is a great feature for users who need to work with large datasets.

The input names for load and dump have been changed from "path" to "source" and "dest", respectively. This change is likely to cause some confusion for users who are used to the old names, but it's a good thing the old names still work, albeit with a warning.

Managed Identity Support for Compute Instance is now experimental, which means it's still in the testing phase but could become a stable feature soon.

Idle Shutdown support for Compute Instances is now available, allowing instances to shut down after a period of inactivity. This feature can help save resources and reduce costs.

Online Deployment Data Collection will soon support eventhub and data storage. This feature will make it easier to collect and analyze data from online deployments.

Syntax validation for scoring scripts in Batch Deployment and Online Deployment will prevent users from submitting bad deployments. This is a great feature for users who are new to the platform or need to troubleshoot their scripts.

Here's a list of all the changes in the 0.1.0b7 release:

In the 0.1.0b6 release, AutoML Component support was added, allowing for more automated machine learning capabilities.

This release also introduced a new feature, skip_validation, which can be used when creating or updating Jobs and Components.

The update fixed a bug that caused mismatch errors when updating scale settings for KubernetesOnlineDeployment.

A minor tweak was made to the az CLI command, removing the unnecessary print statement that appeared when deleting OnlineEndpoints.

Here's a quick rundown of the key changes: