Azure Endpoint Setup and Management Guide

To set up an Azure endpoint, you'll need to create a new endpoint in the Azure portal. This involves specifying the name and protocol for your endpoint.

The Azure portal provides a user-friendly interface for creating and managing endpoints. You can access this interface by navigating to the Azure dashboard and clicking on the "Endpoints" tab.

To create a new endpoint, you'll need to provide a unique name and specify the protocol as either HTTP or HTTPS. This information will be used to identify and access your endpoint.

Azure endpoints can be used to expose APIs and other services to the public internet, making it easier to access and integrate with your cloud-based applications.

You can create an endpoint in the Azure classic portal or using Azure PowerShell cmdlets. To create an endpoint, sign in to the Azure classic portal, click Virtual Machines, and then click the name of the virtual machine that you want to configure.

Common endpoints like Remote Desktop, Windows PowerShell Remoting, and Secure Shell (SSH) are typically created for you automatically, depending on the operating system you choose. However, you can configure additional endpoints while creating the virtual machine or afterwards as needed.

Each endpoint has a public port and a private port: the public port is used by the Azure load balancer to listen for incoming traffic from the Internet, while the private port is used by the virtual machine to listen for incoming traffic.

To create a new endpoint, follow these steps: If you haven't already done so, sign in to the Azure classic portal. Click Virtual Machines, and then click the name of the virtual machine that you want to configure. Click Endpoints, and then click Add.

The public port is used by the Azure load balancer to listen for incoming traffic from the Internet, while the private port is used by the virtual machine to listen for incoming traffic. You can configure additional endpoints while creating the virtual machine or afterwards as needed.

Here's a summary of the endpoint configuration options:

After creating an endpoint, you can use an access control list (ACL) to define rules that permit or deny the incoming traffic to the public port of the endpoint based on its source IP address. However, if the virtual machine is in an Azure virtual network, you should use network security groups instead.

Managing Azure Endpoint is a crucial step in ensuring secure and controlled access to your Azure resources. You can manage the Access Control List (ACL) on an endpoint to restrict traffic based on source IP address.

To add, modify, or remove an ACL on an endpoint, sign in to the Azure classic portal, click Virtual Machines, and then click the name of the virtual machine that you want to configure. Click Endpoints, select the appropriate endpoint, and then click Manage ACL to open the Specify ACL details dialog box.

The rules in the ACL are evaluated in order, starting with the first rule and ending with the last rule. This means that rules should be ordered from least restrictive to most restrictive. For example, you can use rules to allow only traffic from specific computers corresponding to your computers on the Internet or to deny traffic from specific, known address ranges.

To define the set of computers that can send traffic, use rows in the list to add, delete, or edit rules for an ACL and change their order. The Remote Subnet value is an IP address range for incoming traffic from the Internet that the Azure load balancer uses to permit or deny the traffic based on its source IP address. Be sure to specify the IP address range in CIDR format, such as 131.107.0.0/16.

If you're managing Azure endpoints, you'll want to manage the ACL (Access Control List) on an endpoint to restrict traffic based on source IP address. To do this, sign in to the Azure classic portal and click Virtual Machines.

Click the name of the virtual machine you want to configure and then click Endpoints. From the list, select the appropriate endpoint.

To add, modify, or remove an ACL on an endpoint, click Manage ACL in the taskbar to open the Specify ACL details dialog box. You can use rows in the list to add, delete, or edit rules for an ACL and change their order.

The Remote Subnet value is an IP address range for incoming traffic from the Internet that the Azure load balancer uses to permit or deny the traffic based on its source IP address. Be sure to specify the IP address range in CIDR format, also known as address prefix format. An example is 131.107.0.0/16.

Rules are evaluated in order, starting with the first rule and ending with the last rule. This means that rules should be ordered from least restrictive to most restrictive.

Here's a quick rundown of how to order your rules:

You can connect to a private-link service using an alias, which is a unique moniker generated when a service owner creates the service behind a standard load balancer.

To use the alias, create a private endpoint with the manual connection approval method. This involves setting the manual request parameter to True during the private-endpoint create flow, as described in the New-AzPrivateEndpoint command and the az network private-endpoint create command.

Consumers can request a connection to the private-link service by using either the resource URI or the alias. They can share the alias offline with the service owner.

Azure Endpoint Deployment is a powerful tool for hosting machine learning models. You can deploy a model by providing model files, scoring script code, an environment to run your model, and settings to specify the instance type and scaling capacity.

To deploy a model, you'll need to create an online endpoint, which can contain multiple deployments with different configurations. Each deployment can have a specific configuration, such as using VMs with a CPU or GPU SKU, and running a specific version of a model.

A deployment can reference a model and container image defined in Environment at any time, for example when the deployment instances undergo security patches or other recovery operations. However, if you use a registered model or container image in Azure Container Registry for deployment and later remove the model or the container image, the deployments that rely on these assets can fail when reimaging occurs.

Here are the key aspects of online deployment options:

Azure Machine Learning supports model deployment to online endpoints for coders and noncoders by providing options for no-code deployment, low-code deployment, and Bring Your Own Container (BYOC) deployment.

A deployment is a set of resources and computes required to host the model that does the inferencing. To deploy a model, you must have model files, a scoring script code, an environment to run your model, and settings to specify the instance type and scaling capacity.

The key attributes of a deployment include the name of the deployment, the endpoint name, the model to use, the code path, the scoring script, the environment, instance type, and instance count. You can use a reference to an existing versioned model in the workspace or an inline model specification.

You can reference the model and container image defined in Environment at any time, for example when the deployment instances undergo security patches or other recovery operations. However, if you use a registered model or container image in Azure Container Registry for deployment and later remove the model or the container image, the deployments that rely on these assets can fail when reimaging occurs.

Azure Machine Learning supports model deployment to online endpoints for coders and noncoders by providing options for no-code deployment, low-code deployment, and Bring Your Own Container (BYOC) deployment. No-code deployment uses out-of-box inferencing for common frameworks like scikit-learn, TensorFlow, PyTorch, and Open Neural Network Exchange (ONNX) via MLflow and Triton.

The following table highlights the key aspects of the online deployment options:

A single online endpoint can have multiple deployments. As the endpoint receives incoming traffic requests, it can route percentages of traffic to each deployment, as in the native blue/green deployment strategy. The endpoint can also mirror or copy traffic from one deployment to another, called traffic mirroring or shadowing.

Debugging is a crucial step in ensuring your Azure endpoint is working as expected. You can test-run your endpoint locally to validate and debug your code and configuration before deploying to Azure.

Azure Machine Learning provides several ways to debug online endpoints locally, including using the Azure Machine Learning inference HTTP server, local endpoint, and container logs. Local debugging with Azure Machine Learning inference HTTP server allows you to test your endpoint locally and validate your code and configuration.

To deploy locally, you need the Docker Engine installed and running. Azure Machine Learning then creates a local Docker image to mimic the online image. This process builds and runs deployments for you locally and caches the image for rapid iterations.

Local debugging typically involves checking that the local deployment succeeded, invoking the local endpoint for inferencing, and reviewing the output logs for the invoke operation. Local endpoints have limitations, including no support for traffic rules, authentication, or probe settings, and support for only one deployment per endpoint.

For more advanced debugging, you can use client-side tools such as Docker Desktop to debug what happens in the container. Local debugging with local endpoint and Visual Studio Code (preview) is also available, but this feature is currently in public preview and should not be used for production workloads.

Here are the steps for local debugging with local endpoint:

Traffic Management is a crucial aspect of Azure Endpoint. You can route percentages of traffic to each deployment, as in the native blue/green deployment strategy.

A single online endpoint can handle multiple deployments, allowing you to mirror or copy traffic from one deployment to another, called traffic mirroring or shadowing.

Traffic mirroring is useful for testing a new deployment with production traffic without impacting customer results. This is achieved by mirroring 10% of traffic to a green deployment, while the rest is routed to the blue deployment.

You can also use traffic mirroring for shadow testing, where the results of the mirrored traffic are not returned to clients, but metrics and logs are recorded.

To implement load balancing, the endpoint can allocate certain percentages of traffic to each deployment, adding up to 100%. This allows you to roll out a new green deployment to a small subset of users or requests before rolling it out completely.

A request can bypass traffic load balancing by including an HTTP header with the name of the deployment you want it to route to. This header is set to azureml-model-deployment.

Managed network isolation is a crucial aspect of securing your Azure endpoint. You can configure security for inbound scoring requests and outbound communications separately.

Inbound communications use the private endpoint of the Azure Machine Learning workspace, while outbound communications use private endpoints created for the workspace's managed virtual network. For more information, see Network isolation with managed online endpoints.

Private endpoints provide a privately accessible IP address for the Azure service, but do not necessarily restrict public network access to it. All other Azure services require additional access controls, however.

These controls provide an extra network security layer to your resources, providing protection that helps prevent access to the Azure service associated with the private-link resource. Private endpoints support network policies, including Network Security Groups (NSG), User Defined Routes (UDR), and Application Security Groups (ASG).

To enable network policies for a private endpoint, see Manage network policies for private endpoints. To use an ASG with a private endpoint, see Configure an application security group (ASG) with a private endpoint.

Here are some key considerations for network policies:

A private endpoint is a unique name within a resource group that specifies the properties to connect to a private-link resource. This connection is established between the customers from the same network.

The private endpoint must be deployed in the same region and subscription as the virtual network, but the private-link resource can be deployed in a different region.

Private endpoints provide a privately accessible IP address for the Azure service, but do not necessarily restrict public network access to it. The platform validates network connections, allowing only those that reach the specified private-link resource.

Here are the properties of a private endpoint:

Network configuration is crucial for a secure and reliable Azure endpoint. You can secure communication with the online endpoint by using private endpoints.

Inbound communications use the private endpoint of the Azure Machine Learning workspace, while outbound communications use private endpoints created for the workspace's managed virtual network. For this reason, it's essential to configure security for inbound scoring requests and outbound communications separately.

To connect to a private-link resource, you need to configure your DNS settings correctly. The DNS settings must resolve to the private IP address of the private endpoint.

The network interface associated with the private endpoint contains the information required to configure your DNS. This includes the fully qualified domain name (FQDN) and private IP address for a private-link resource.