Azure Blob Storage Retention Policy Made Easy with Lifecycle Management

Azure Blob Storage provides a feature called Lifecycle Management that makes it easy to implement a retention policy. This feature allows you to specify rules for when to transition or delete blobs.

You can create up to 10 lifecycle rules per container, and each rule can specify a different action. For example, you can create a rule to transition blobs from hot to cool storage after 30 days.

Lifecycle Management also allows you to specify a retention period, which determines how long blobs are retained before being deleted. This period can be set in days or years, depending on your needs.

By using Lifecycle Management, you can simplify your retention policy and reduce costs associated with storing data.

Related reading: Azure Storage Account Lifecycle Management

Retention policies in Azure Blob Storage are used to manage the lifetime of your data. They ensure that your data is retained for a specified period, providing a safeguard against accidental deletion or modification.

Recommended read: What Is the Data Storage in Azure Called

To set up a retention policy, you need to enable blob versioning for your storage account. This can be done by going to the Storage accounts page on the Azure portal, navigating to Data management and selecting Data protection, and then opting to enable versioning for blobs.

Blob versioning allows you to retain all versions of your blobs or delete them after a specific timeframe. You can choose to retain all versions, which is useful for auditing and compliance purposes.

Here are some common scenarios where lifecycle policies are used:

To establish a default time-based retention policy, you can follow these steps:

Note that a default time-based retention policy applies to all new blob versions, but it doesn’t affect existing versions. If you’ve migrated a container to support version-level immutability, the prior container-level policy becomes the default for the container.

Curious to learn more? Check out: Azure Storage Container

To configure a retention policy for your Azure blob storage, you can set up a default time-based retention policy, which applies to all new blob versions. This policy doesn't affect existing versions.

You can enable blob versioning for your storage account by going to the Storage accounts page on the Azure portal, navigating to Data management and selecting Data protection, and opting to enable versioning for blobs. You can then decide whether to retain all versions or delete them after a specific timeframe.

To modify an unlocked retention policy, you can shorten or lengthen the retention interval and decide whether to allow additional writes to append blobs in the container. You can also delete an unlocked policy.

To modify an unlocked policy, you can call the Get-AzRmStorageContainerImmutabilityPolicy command to retrieve the policy, and then call the Set-AzRmStorageContainerImmutabilityPolicy command to update the policy. Alternatively, you can use the az storage container immutability-policy extend command in the Azure CLI to update the policy.

Curious to learn more? Check out: Which Azure Storage Service Supports Big Data Analytics

Rule actions are the backbone of a lifecycle policy, determining what happens to your data based on specific conditions.

Tiering is supported for block blobs, allowing you to move them to the cool, cold, or archive tier. However, it's not yet supported in premium block blob storage accounts.

Worth a look: Linode Block Storage

Here's a list of supported actions for each type of blob:

The delete action has some specific rules, such as not deleting the current version of a blob until any previous versions or snapshots associated with that blob have been deleted.

Lifecycle policies are a powerful tool for managing data storage, allowing you to automate the process of moving data to different storage tiers based on its age and usage patterns.

Lifecycle policies can be used to transition block blobs to cooler storage tiers, such as cool or archive storage, based on their age. For example, you can move blobs that haven't been modified in over 30 days to cool storage, and blobs not modified in 90 days to the archive tier.

To configure lifecycle policies, you need to enable blob versioning for your storage account, which allows you to track and manage different versions of your data.

You might like: Azure Archive Storage

Lifecycle policies support tiering and deletion of current versions, previous versions, and blob snapshots. You can define at least one action for each rule, such as tiering to cool or archive storage, or deleting the blob.

The following table summarizes the supported actions for each tier:

Lifecycle policies can also be used to optimize costs by moving data to the most cost-effective storage tier based on its age and usage patterns. For example, you can move hot data to cool storage after a certain period of time, and then move it to archive storage after a longer period of time.

To modify an unlocked retention policy, you can update the policy to shorten or lengthen the retention interval, and to allow additional writes to append blobs in the container.

Lifecycle policies can also be used to move data based on last accessed time, which can help to reduce storage costs and improve data management. To enable last access time tracking, you need to explicitly enable access time tracking, and then you can use the LastAccessTime property to determine whether the run condition daysAfterLastAccessTimeGreaterThan is met.

By using lifecycle policies, you can automate the process of managing your data storage, and ensure that your data is stored in the most cost-effective and efficient way possible.

Worth a look: Azure Storage Cost

Expiration and Deletion is a crucial aspect of Azure Blob Storage retention policy. You can configure a lifecycle management policy to expire data by deletion based on data age, which can be set to delete all block blobs that haven't been modified in the last 365 days.

Data can also be expired if explicitly marked for deletion. This is done by configuring a lifecycle management policy to expire data that are tagged with blob index key/value attributes, such as "Project = Contoso".

The LifecyclePolicyCompleted event is generated when the actions defined by a lifecycle management policy are performed. A summary section appears for each action that is included in the policy definition.

Here are the possible actions included in the policy definition and their corresponding summary sections:

The results summary of each action, such as the deleteSummary, tierToCoolSummary, tierToColdSummary, and tierToArchiveSummary, provide valuable information about the lifecycle policy completion.

Managing previous versions of your data is crucial for compliance and auditing purposes. You can enable blob storage versioning to automatically maintain previous versions of an object.

This feature is particularly useful for data that's modified and accessed regularly throughout its lifetime. You can create a policy to tier or delete previous versions based on their age.

The version age is determined by evaluating the version creation time. A policy rule can move previous versions within a container to the cool tier if they're 90 days or older after version creation, and delete previous versions that are 365 days or older.

Here's a summary of the versioning policy:

By enabling versioning and setting up a policy, you can ensure that your data is properly managed and retained for as long as needed.

When managing containers and blobs in Azure Blob Storage, you have several options for organizing and expiring data.

You can move data based on age, such as the date it was last created, modified, or last accessed. To track last accessed time, you must explicitly enable access time tracking.

You can also move data by blob index tag or by prefix match. This makes it easy to categorize and manage your data.

If you have blob storage versioning enabled, you can move previous versions of an object. This is useful for maintaining a record of changes.

Data can be expired by deleting it based on the age rules mentioned earlier. This helps keep your storage clean and organized.

See what others are reading: Azure Blob Storage Move Files between Containers C#

There is no cost for Lifecycle Management policies, but you'll be billed for standard operation costs associated with moving a blob to a different tier.

You'll incur write operation charges of the destination tier when moving to a cooler tier, and read operation charges of the source tier when moving to a warmer tier.

The minimum number of required days a blob must remain in a tier varies: 30 days for Cool, 90 days for Cold, and 180 days for Archive.

Deleting or moving your blob before the required number of days pass will result in an early deletion penalty, which is calculated based on the number of days you moved it before the minimum.

If you start by loading your data into the appropriate tier, you'll avoid the cost of moving blobs between tiers.

You can specify the tier your blob should be assigned to when uploading the blob, or set the default access tier for a storage account to a cooler tier.

There are also charges for "other operations" if you have last access time enabled, which updates every 24 hours.

Here's a quick summary of the minimum days required for each tier: