What is Azure Anomaly Detection and How Does it Work

Azure Anomaly Detection is a cloud-based service that helps you identify unusual patterns in your data. It uses machine learning algorithms to detect anomalies in real-time.

This service is designed to work with various types of data, including time series data, which is data that changes over time. Azure Anomaly Detection can be used to monitor website traffic, server performance, and other metrics.

The service uses a technique called one-class classification to identify anomalies. One-class classification is a type of machine learning algorithm that can identify data points that don't fit a normal pattern. This is useful for detecting unusual spikes in data that could indicate a problem.

Azure Anomaly Detection can be integrated with other Azure services, such as Azure Monitor and Azure Log Analytics, to provide a comprehensive view of your data.

There are four types of cost anomalies to watch out for in Azure.

Anomalous spike in Azure Service costs refers to sudden spikes in costs for specific services like VM, App Service, and Storage compared to normal usage.

Anomalous spikes in Cost per Usage indicate cost fluctuations per unit of usage, which can signal a shift towards more expensive services or a change in pricing plans.

Anomalous spikes in business unit cost allow engineers to take ownership of their costs by identifying spikes in specific business units like teams or environments.

Anomalous drop on unit economics signifies a drop in revenue compared to cost, indicating inefficiencies in resource usage.

Here are the four types of anomalies in a concise list:

Cost anomalies can be categorized into four types based on their granularity. Each type reveals a different aspect of cost behavior and requires a unique approach to identify and address the issue.

Anomalous spikes in Azure Service costs refer to sudden and significant increases in the cost of a particular Azure service, such as VM, App Service, or Storage. These spikes can be a sign of inefficient resource allocation.

Anomalous spikes in Cost per Usage are a type of anomaly that determines the cost fluctuation per unit of usage. For instance, if there's a spike in cost per hour of compute, it may indicate a shift towards expensive services.

Anomalous spikes in business unit cost allow engineers to take ownership of their costs by identifying cost spikes in specific business units, such as cost per team or environment. This approach supports the principle of FinOps, "Everyone takes ownership for their cloud usage and optimize spending."

Anomalous drops on unit economics signify a decrease in the ratio between revenue and cost, indicating a loss of efficiency in resource usage. This type of anomaly supports the principle of FinOps, "Decisions are driven by business value of cloud."

Here are the four types of anomalies based on granularity:

Multivariate anomaly detection is a powerful tool that evaluates multiple signals and their correlations to find sudden changes in data patterns.

This approach helps identify anomalies that might affect your business, allowing you to take proactive measures to mitigate their impact.

Multivariate anomalies can be particularly challenging to detect because they involve multiple variables and their complex relationships.

However, with the right techniques and tools, you can uncover these anomalies and make informed decisions to protect your business.

Using multivariate anomaly detection, you can evaluate multiple signals and their correlations to find sudden changes in data patterns before they affect your business.

The detection process is where the magic happens with Azure Anomaly Detector. It's a set of APIs that allows you to monitor and detect anomalies in time-series data.

You can use the univariate API, which comes with a pre-trained model, or the multivariate API, which requires training before use. Both APIs use unsupervised learning techniques to learn patterns and behaviors in the data.

The univariate API can detect spikes, dips, and deviations from cyclic patterns, while the multivariate API can handle complex scenarios with multiple variables and underlying correlations.

You can customize the service to detect any level of anomaly, and deploy it in the cloud or at the intelligent edge.

Azure offers several tooling options for cost anomaly detection, making it easier for businesses to identify irregularities and optimize spending.

Native tools are sufficient for companies with minimal spending and a few subscriptions.

For complex environments, advanced tooling is required to go beyond traditional methods.

Turbo360's Azure Cost Management tool is a SaaS-based solution that introduces automated anomaly detection and alert mechanisms.

It leverages historical cost data to autonomously identify and notify stakeholders of unexpected cost changes.

Turbo360 dynamically assesses average cost differences daily and monthly, unlike conventional approaches that require users to set thresholds manually.

Anomaly detection can be detected at any desired business unit level, such as per product or team.

The Anomaly Detector can be easily embedded into apps to quickly identify problems as they occur.

No machine-learning background is required to use the Anomaly Detector, which ingests time-series data of all types and selects the best-fitting anomaly detection model for high accuracy.

To implement Azure Anomaly Detection, you'll need to install the @azure/ai-anomaly-detector package using npm.

You can then easily embed anomaly detection capabilities into your apps, no machine-learning background required.

To use the Anomaly Detector API, you'll need to prepare your data by formatting it into a JSON request object, specifying the granularity of your data and including the actual data in a series array.

To install the Azure Anomaly Detector client library for JavaScript, you'll need to use npm. Specifically, you'll need to install the @azure/ai-anomaly-detector package. This will give you access to the AnomalyDetectorClient, which provides methods for anomaly detection.

The AnomalyDetectorClient has three main methods: detectEntireSeries, detectLastPoint, and detectChangePoint. These methods allow you to detect anomalies in different parts of your data.

To get started with anomaly detection, you'll first need to prepare your data. This involves formatting your time series data into a JSON request object. The object should include a "granularity" field, which specifies the rate at which your data is sampled. It can be set to a value like "yearly", "daily", or "secondly", and can also be customized if needed.

Here's a quick rundown of the data constraints:

With your data prepared, you can then use the AnomalyDetectorClient to detect anomalies. This involves creating an Anomaly Detector resource in the Azure Portal and obtaining the resource endpoint and API key. With these values, you can create a client and start using the AnomalyDetectorClient to detect anomalies in your data.

To create an API key, you can find it in the Azure Portal by clicking Keys and Endpoint under Resource Management, or use the Azure CLI snippet.

Note that the API key is also referred to as a "subscription key" or "subscription API key", so don't worry if you see it referred to by a different name.

You can use the Azure Portal to browse to your Anomaly Detector resource and retrieve an API key, or use the Azure CLI snippet to do it programmatically.

Once you have your API key, you can use the AzureKeyCredential class to authenticate the client.

Note that you'll need both the API key and the endpoint of your Anomaly Detector resource to create a client object.

You can find the endpoint for your Anomaly Detector resource in the Azure Portal, or use the Azure CLI snippet to get it.

With both the API key and endpoint in hand, you're ready to start using the Anomaly Detector API.