Azure Cloud Storage for Scalable and Secure Data

Azure Cloud Storage is designed to handle massive amounts of data, scaling to meet the needs of your application.

With Azure Cloud Storage, you can store up to 5 TB of data in a single container, making it ideal for large-scale applications.

Your data is secure with Azure Cloud Storage, thanks to its robust security features, including encryption at rest and in transit.

Azure Cloud Storage also supports object-level encryption, which allows you to encrypt individual files and folders for added security.

Additional reading: Azure Storage Account Encryption

Azure Blob Storage helps you create data lakes for your analytics needs, and provides storage to build powerful cloud-native and mobile apps.

You can store and access unstructured data at scale with Azure Blob Storage, which optimizes costs with tiered storage for your long-term data.

Azure Blob Storage offers multiple storage tiers, including hot, cool, cold, and archive, allowing you to store massive amounts of infrequently or rarely accessed data in a cost-efficient way.

If this caught your attention, see: What Is Azure Storage

To access Azure Blob Storage, you can use various authorization methods, including Microsoft Entra integration, identity-based authentication over SMB, Shared Key, shared access signatures (SAS), and Active Directory Domain Services with Azure NetApp Files.

Here are some of the authorization methods supported by Azure Blob Storage:

Azure Blob Storage is a powerful tool for creating data lakes for analytics needs, and provides storage to build powerful cloud-native and mobile apps. It optimizes costs with tiered storage for long-term data, and flexibly scales up for high-performance computing and machine learning workloads.

Azure Blob Storage helps you store petabytes of data cost-effectively, with multiple storage tiers and automated lifecycle management. You can replace your tape archives with Blob storage and never worry about migrating across hardware generations.

Azure Storage supports the following authorization methods: Microsoft Entra integration, identity-based authentication over SMB, Shared Key, shared access signatures (SAS), and Active Directory Domain Services with Azure NetApp Files. Each service is accessed through a storage account with a unique address.

To get started, see Create a storage account. Additionally, Azure provides specialized storage, including Azure NetApp Files, which makes it easy for enterprise line-of-business and storage professionals to migrate and run complex, file-based applications with no code change.

Azure Storage offers several data services, including Azure Blobs, Azure Files, Azure Elastic SAN, Azure Queues, Azure Tables, Azure managed Disks, and Azure Container Storage. Each service is accessed through a storage account with a unique address.

Here are the data services offered by Azure Storage:

You can also use shared access signatures (SAS) to temporarily allow third parties access to a single container or put credentials into an untrusted environment such as a CI build server. A SAS URL can be obtained from the Azure portal or the Azure Storage Explorer.

Readers also liked: Give Onedrive Access to Storage

Public access is an important consideration when working with Azure Blob storage. It determines who can access your container and its contents.

Expand your knowledge: Give Access to Azure Blob Storage

The public access level of a container can be set to either "blob" or "container". This option is configurable through the "public_access" setting.

You can also set the public access level using an environment variable called "RCLONE_AZUREBLOB_PUBLIC_ACCESS". This variable is a string that specifies the access level.

Here are the possible values for the public access level:

The public access level is not required, so you can choose not to set it if you prefer.

Curious to learn more? Check out: Azure Storage Not Displaying

Cloud Storage Manager is a user-friendly tool that provides insightful data and management features to optimize your storage usage and reduce Azure Storage spending. It gives you complete control over your Azure cloud storage.

You can see your Azure Storage Accounts, Containers, and Blobs Information in different tabs, detailing information such as name, consumption, resource group, Azure Datacentre location, and the size of the item. This makes it easy to identify the biggest Storage Accounts, Containers, or Blobs.

Each tab allows you to drill down into each item for further information, and right-clicking on an item provides options like jumping directly to the portal.

Cloud Storage Manager also enables you to Search all your Azure Storage Accounts, which is particularly useful when you need to know where a particular File or Blob resides. It will search your entire Azure Tenancy, through each and every Storage Account for that particular Blob.

If you're looking to scan your Azure Storage Accounts, Cloud Storage Manager can help you quickly and easily scan all your Azure Blob Storage Accounts and see where all your Azure Storage is being consumed and most importantly the areas where you can save money.

To set up Cloud Storage Manager, you'll need to configure the Azure Storage Account Name, which can be done by setting the "--azureblob-account" parameter. This parameter can be set to the Azure Storage Account Name in use, or left blank to use SAS URL or Emulator.

Azure Container Storage integrates with Kubernetes and utilizes existing Azure Storage offerings for actual data storage, offering a volume orchestration and management solution purposely built for containers. This provides substantial benefits, including Rapid scale out of stateful pods, Improved performance for stateful workloads, and Kubernetes-native volume orchestration.

Azure Storage provides robust security features to protect your data. Microsoft Entra integration is recommended for superior security and ease of use, and supports authentication and authorization with Azure role-based access control (Azure RBAC).

Azure Storage supports multiple authorization methods, including Microsoft Entra ID, Identity-based authentication over SMB, Authorization with Shared Key, and Authorization using shared access signatures (SAS).

Here are the different authorization methods supported by Azure Storage:

Azure Storage also provides encryption at rest, which automatically encrypts all data prior to persisting to the storage account and decrypts it prior to retrieval.

Azure Storage offers scalable solutions to meet your data needs. Sixteen nines of designed durability ensure that your data is protected.

With geo-replication, you can rest assured that your data is safe. This feature allows you to replicate your data across different regions, providing an extra layer of protection.

To achieve this level of durability, Azure Storage stores multiple copies of your data. You can select a redundancy option when setting up your storage account, and for more information, see Azure Storage redundancy and Azure Files data redundancy.

Azure NetApp Files provides locally redundant storage with 99.99% availability. This means that your data is not only durable, but also highly available.

Azure Storage takes security seriously, with a robust set of features to protect your data. Microsoft Entra ID, formerly Azure Active Directory, provides authentication and role-based access control, making it a recommended choice for secure access to storage accounts.

Authentication with Microsoft Entra ID and role-based access control is just the beginning. Encryption at rest is also a key feature, automatically encrypting all data before it's stored and decrypting it before it's retrieved. This ensures that your data is protected, even if it's accessed by unauthorized parties.

Azure Storage also supports identity-based authentication over SMB for Azure Files, using on-premises Active Directory Domain Services, Microsoft Entra Domain Services, or Microsoft Entra Kerberos. This provides an additional layer of security for your data.

Encryption at rest is not the only way Azure Storage secures your data. Azure NetApp Files data traffic is inherently secure by design, staying within customer-owned VNet and not providing a public endpoint. However, data-in-flight encryption can be optionally enabled for NFSv4.1 and SMB3 data.

Additional reading: Sell Cloud Computing Services

Here's a summary of the security features available in Azure Storage:

By using these security features, you can be confident that your data is protected and secure in Azure Storage.

You can authenticate with a service principal and client secret in rclone by setting the tenant, client ID, and client secret variables. The credentials can also be placed in a file using the service_principal_file configuration option.

To use a service principal with a client secret, you'll need to know the tenant ID, client ID, and client secret of the service principal. The tenant ID is also known as the service principal's directory ID.

Here are the variables you'll need to set:

Alternatively, you can place the credentials directly into the rclone config file under the client_id, tenant, and client_secret keys.

If you prefer to store the credentials in a file, you can use the service_principal_file configuration option. This option is optional, but it can be useful if you want to keep your credentials separate from your rclone configuration.

The service_principal_file option requires a path to a file containing the credentials. You can find more information on creating an Azure service principal and assigning an Azure role for access to blob data on the relevant pages.

Here are the details of the service_principal_file option:

Managed Service Identity is a powerful tool for securing your Azure Storage accounts. It allows you to use managed service identity credentials, which only work when running in an Azure service.

To use managed service identity credentials, you need to set the `use_msi` parameter to true and unset the `env_auth` parameter. This authentication method is only available when running in an Azure service.

If you have multiple user identities to choose from, you must explicitly specify one of the `msi_object_id`, `msi_client_id`, or `msi_mi_res_id` parameters. If none of these parameters are set, this is equivalent to using `env_auth`.

A unique perspective: How to Keep Your Onedrive from Running Out of Storage

Here are some details about the parameters you can use with managed service identity credentials:

These parameters can be configured using either the command-line interface or environment variables. If you're using the command-line interface, you can specify the parameters using the `--azureblob-msi-object-id`, `--azureblob-msi-client-id`, or `--azureblob-msi-mi-res-id` flags.

Intriguing read: How to Upload File to Google Cloud Storage Using Reactjs

Archive tier blobs cannot be updated, which means if you try to update one, rclone will produce an error.

To avoid this, you can use the "--azureblob-archive-tier-delete" flag, which deletes the existing archive tier blob before uploading its replacement. This has the potential for data loss if the upload fails, and may also cost more since deleting archive tier blobs early may be chargeable.

Here are some details about this flag:

Modification times are stored as metadata on the object with the mtime key, using RFC3339 Format time with nanosecond precision.

This means you can easily access and view modification times without any performance overhead. The metadata is supplied during directory listings, making it convenient to keep track of when files were last modified.

You might enjoy: Azure Blob Storage Add Metadata

If you want to use the Azure standard LastModified time stored on the object as the modified time, you can use the --use-server-modtime flag. Note that rclone can't set LastModified, so using the --update flag when syncing is recommended if using --use-server-modtime.

MD5 hashes are stored with blobs, providing an additional layer of security and verification. Blobs that were uploaded in chunks only have an MD5 if the source remote was capable of MD5 hashes, such as the local disk.