Azure Open AI Custom Model Tutorial and Configuration

To create an Azure Open AI custom model, you'll first need to prepare your data. This involves collecting and labeling relevant data, which can be a time-consuming process but is essential for training an accurate model.

A good starting point is to understand the different types of data you can use, such as text, image, or audio. You can then use tools like Azure Blob Storage to store and manage your data.

Once you have your data ready, you can start training your model using Azure Machine Learning. This involves defining your model architecture and configuring hyperparameters to optimize its performance.

The configuration process can be complex, but Azure provides a range of tools and resources to help you get started, including the Azure Machine Learning SDK and the Azure Open AI documentation.

Suggestion: Create an Ai Model in My Azure Tenant

To get started with creating an Azure Open AI custom model, you'll need to meet some prerequisites. First, you'll need an Azure subscription, which you can create for free.

To fine-tune your Azure Open AI model, you'll need access to Cognitive Services OpenAI Contributor. If you don't already have this access, you may need additional permissions to view quota and deploy models in Azure AI Studio.

Here are the specific prerequisites you'll need to meet:

You'll also need to use a specific API version, which is 2024-08-01-preview or later. This will ensure that you have the necessary functionality to create and fine-tune your custom model.

To work with Azure Open AI, you'll also need to have Python 3.8 or later installed, as well as certain Python libraries, such as json, requests, and openai.

Here's an interesting read: Azure Azure-common Python Module

Creating a Custom Model is a straightforward process, and to start, you need to create your training and validation datasets. The more training examples you have, the better, and it's best practice to provide hundreds, if not thousands, of training examples to be successful.

Broaden your view: Azure Ai Training

You can use the Create custom model wizard in Azure AI Studio to create and train a fine-tuned model for your Azure resource. To do this, open Azure AI Studio, sign in with credentials that have access to your Azure OpenAI resource, and browse to the Tools > Fine-tuning pane.

The first step in creating a custom model is to choose a base model. You can select from several available base models, including babbage-002, davinci-002, gpt-35-turbo (0613), and gpt-35-turbo (1106).

To select the base model, open the Base model pane and choose a base model from the Base model type dropdown. You can then select Next to continue.

Here are the available base models you can choose from:

Once you've chosen your base model, the next step is to either choose existing prepared training data or upload new prepared training data to use when customizing your model.

Curious to learn more? Check out: Model Catalog Azure

You can configure your parameters for a fine-tuning job in Azure AI Studio.

The batch size is a crucial parameter that determines the number of training examples used to train a single forward and backward pass. Larger batch sizes tend to work better for larger datasets.

You can set the batch size to a specific value, or you can use the default value, which is determined algorithmically based on your training data.

The learning rate multiplier is another important parameter that affects the performance of your fine-tuned model. A larger learning rate tends to perform better with larger batch sizes, but may lead to overfitting.

The recommended range for the learning rate multiplier is between 0.02 and 0.2.

You can also specify the number of epochs to train your model for. An epoch refers to one full cycle through the training dataset.

If you set the number of epochs to -1, the number of epochs is determined dynamically based on the input data.

The seed parameter controls the reproducibility of the job. If a seed isn't specified, one will be generated for you.

Here's a summary of the parameters you can configure:

To deploy your custom model in Azure Open AI, you can use the Azure CLI, which requires specifying a name for the deployment. You can also use the Playgrounds in Azure AI Studio to experiment with your new deployment.

You can deploy a model with Azure CLI by running a command that requires replacing placeholders with corresponding values for your customized model. The placeholders include your Azure subscription, resource group, resource name, deployment name, and fine-tuned model ID.

To deploy a fine-tuned model, you'll need to use the REST API, which requires separate authorization and a different API path. You can also use Azure OpenAI Studio or Azure CLI for deployment. You'll need to generate an authorization token, which can be done by launching the Cloud Shell from the Azure portal and running the 'az account get-access-token' command.

Discover more: Azure Ai Api

Cross region deployment allows you to fine-tune a model in one region and deploy it to a different region. This means you can train your model in one part of the world and use it in another, which is super useful if you have users in multiple locations.

The only limitation is that the new region must support fine-tuning, so make sure to check that before deploying.

You can deploy cross subscription/region via Python or REST, which is great because it gives you flexibility in how you set up your deployment.

If this caught your attention, see: Azure Shared Responsibility Model

After your model is deployed, you can use it like any other deployed model in Azure AI Studio. You can experiment with your new deployment in the Playgrounds, using the same parameters as with other deployed models, such as temperature and max_tokens.

For fine-tuned models, you'll use the Completions playground and the Completions API for models like babbage-002 and davinci-002, or the Chat playground and the Chat completion API for models like gpt-35-turbo-0613.

You can also use the REST API to call your fine-tuned model from your own application. This allows you to use the new fine-tuned model in your prompt flow to build your generative AI application.

To use a deployed fine-tuned model, you'll need to ensure that the system message used to guide the model is the same as the one used for training. If you use a different system message, the model might not perform as expected.

See what others are reading: Azure Ai Studio Api

You can deploy a model with Azure CLI, specifying a name for the deployment of your customized model. You'll need to replace placeholders in the Azure CLI command with the corresponding values for your customized model.

Here's a summary of the variables you'll need to specify when deploying a fine-tuned model using the REST API:

After you deploy a customized model, it will be inactive if no completions or chat completions calls are made to it for 15 days. If the deployment remains inactive for greater than 15 days, it will be deleted.

Model Management is a crucial aspect of working with Azure Open AI custom models. You can generate checkpoints during training, which are fully functional versions of your model that can be deployed and used for fine-tuning jobs.

Checkpoints can be particularly useful in preventing overfitting, as they provide a snapshot of your model before overfitting occurs. This means you'll have the three most recent versions of your model available to deploy, including the final epoch as your fine-tuned model.

You can manage your custom models by deleting deployments, models, and training files when you're done with them. This is a good practice to keep your workspace organized and avoid unnecessary costs.

Checkpoints are generated after each training epoch completes, providing a snapshot of your model prior to overfitting.

Each checkpoint is a fully functional version of your model, deployable and usable as a target model for subsequent fine-tuning jobs.

Checkpoints can be particularly useful for preventing overfitting, allowing you to revert to a previous version of your model if needed.

The three most recent versions of your model will be available to deploy after a fine-tuning job completes, including the final epoch as your fine-tuned model.

The previous two epochs will be available as checkpoints, giving you a history of your model's development.

You've created a customized model, but now you're wondering how to clean up your deployments and files. You can delete the deployment and model once you're done with them.

To delete your deployment, you can use Azure AI Studio or the Azure CLI. If you're using the Azure CLI, you'll need to replace placeholders with the corresponding values for your customized model.

You can also delete training and validation files you uploaded to the service, if needed. These files must be formatted as a JSON Lines (JSONL) document.

Before deleting your custom model, make sure to delete any existing model deployment. You can't delete a custom model if it has an existing deployment.

Here are the methods you can use to delete your customized model:

You can also delete training and validation files using various methods, including Azure AI Studio, the REST APIs, and the Python SDK.

Here's an example of how to delete training, validation, and result files using the Python SDK:

You can also use the REST API to generate a list of models for a given Azure OpenAI resource. This can help you find the ID of your existing fine-tuned model.

You might like: Azure Openai Python

Model Evaluation is an essential step in fine-tuning a custom model on Azure Open AI. Evaluations are conducted in dedicated, customer-specific, private workspaces.

The evaluation process simulates a conversation with your fine-tuned model to assess the potential to output harmful content using specified categories such as violence, sexual, hate, and fairness, self-harm.

If a model is found to generate output containing content detected as harmful at above an acceptable rate, you'll be informed that your model isn't available for deployment, with information about the specific categories of harm detected.

The model evaluation process is automatic and is conducted within your fine-tuning job as part of providing the fine-tuning capability. Only the resulting assessment (deployable or not deployable) is logged by the service.

You can also view the evaluation results in Azure AI Studio, where you can see charts showing the loss and mean token accuracy of your model over time.

Here are some key points to keep in mind when it comes to model evaluation:

To fine-tune an Azure OpenAI model, you'll need to follow these steps: select a base model to fine-tune, which influences both performance and cost. You can choose from different versions of the base model, like gpt-35-turbo, and select the version you'd like to fine-tune.

To fine-tune a model, you'll need to prepare your training and validation data, which should be formatted as a JSON Lines (JSONL) document with each line representing a single prompt-completion pair. The minimum number of training examples is 10, but having at least 50 high-quality training examples is recommended.

You can prepare your training data using the OpenAI command-line interface (CLI), which includes a data preparation tool that validates, gives suggestions, and reformats your training data into a JSONL file ready for fine-tuning. The more training examples you have, the better, and doubling the dataset size can lead to a linear increase in model quality.

Broaden your view: Azure Openai Chat Completions Api

To start the fine-tuning job, you'll need to upload your training data to the service. You can upload your files to the service using the Azure OpenAI Studio, or you can use the REST API to create a new fine-tuning job. The REST API generates a name for the deployment of your customized model.

Here are the key variables you'll need to deploy your fine-tuned model: token, subscription, resource_group, resource_name, model_deployment_name, and fine_tuned_model. You can retrieve these values from your fine-tuning job results or Azure OpenAI resource.

After you deploy a customized model, if it remains inactive for more than 15 days, the deployment is deleted. However, the underlying customized model is not deleted and can be redeployed at any time.

On a similar theme: Azure Kubernetes Service vs Azure Container Apps

API and Authentication is a crucial part of working with Azure Open AI Custom Model. You can configure the openai package to use Azure OpenAI using environment variables, such as setting the OPENAI_API_TYPE to azure_ad.

Worth a look: Azure Openai Function Calling

To authenticate with Azure OpenAI, you have two options: API Key or Azure Active Directory (AAD). Using the API key is the easiest way to get started, and you can find it in the Azure portal under your Azure OpenAI resource.

You can also use Azure Active Directory, which is more suitable for complex security requirements. To use AAD in Python with LangChain, you'll need to install the azure-identity package and set OPENAI_API_TYPE to azure_ad.

Here are the two ways to authenticate with Azure OpenAI:

Note that using AAD requires granting a role assignment Cognitive Services OpenAI User scoped to your Azure OpenAI resource, which can be done for a user, group, service principal, or managed identity.

API Configuration is a crucial step in using Azure OpenAI. You can configure the openai package to use Azure OpenAI using environment variables.

To do this in bash, you'll need to set the following environment variables: AZURE_OPENAI_API_KEY, AZURE_OPENAI_API_ENDPOINT, and AZURE_OPENAI_API_SECRET. This can be done by adding the following lines to your bash configuration file:

```bash

export AZURE_OPENAI_API_KEY="your_api_key"

export AZURE_OPENAI_API_ENDPOINT="your_api_endpoint"

export AZURE_OPENAI_API_SECRET="your_api_secret"

```

Alternatively, you can configure the API right within your running Python environment by using the openai package's configuration options. This can be done by importing the package and setting the configuration options directly.

If you're developing locally, you'll need to have the Azure CLI installed and be logged in. You can install the Azure CLI here and then run az login to log in.

For more insights, see: Azure Powershell vs Cli

Active Directory Authentication is a great way to add an extra layer of security to your Azure OpenAI interactions. You can authenticate using Azure Active Directory (AAD) instead of an API key.

There are two ways to authenticate with Azure OpenAI: API Key and Azure Active Directory (AAD). Using AAD is a good option if you have complex security requirements. You can find more information on how to use AAD with Azure OpenAI here.

To use AAD, you need to add a role assignment to your Azure OpenAI resource. You can grant this role assignment to a user, group, service principal, or managed identity. For more information about Azure OpenAI RBAC roles, see here.

If this caught your attention, see: Azure Create Custom Role

The easiest way to get started with AAD authentication is to use the DefaultAzureCredential class. This class is easy to use and can be customized if necessary. You can try Managed Identity first and then fall back to the Azure CLI if necessary.

To use AAD in Python with LangChain, you need to install the azure-identity package. Then, set OPENAI_API_TYPE to azure_ad. Finally, use the DefaultAzureCredential class to get a token from AAD by calling get_token.