Azure Vnet Flow Logs Enable Network Visibility and Control

Azure VNet flow logs provide a detailed view of network traffic flowing through your virtual network. This visibility is crucial for troubleshooting and security purposes.

By enabling flow logs, you can collect data on network traffic, including source and destination IP addresses, ports, and protocols. This information can be used to identify potential security threats or performance issues.

Flow logs can be stored in a storage account or a log analytics workspace, making it easy to monitor and analyze network traffic. Azure provides a user-friendly interface to manage and configure flow logs, making it a straightforward process.

Suggestion: Prompt Flow Azure

Network Security Groups (NSGs) are a crucial part of Azure VNet flow logs, allowing you to manage the security of your virtual networks and subnets. An NSG contains security rules that allow or deny network traffic to or from Azure resources connected to it.

NSGs can be associated with a subnet or a network interface of a virtual machine (VM), and all traffic flows in your network are evaluated through the rules in the applicable NSG. This evaluation result is what generates NSG flow logs.

Additional reading: Azure Dataflow

You can manage the security of your virtual networks and subnets by using network security groups, which contain security rules that allow or deny network traffic to or from the Azure resources that the network security group is connected to. NSG flow logs are collected through the Azure platform and don't require any change to your Azure resources.

Here are the key types of network security group rules: Terminating rules: These rules are evaluated when the flow is terminated, and NSG flow logs are written to storage accounts.Non-terminating rules: These rules are evaluated when the flow is not terminated, and NSG flow logs are not written to storage accounts.

Check this out: Azure Log Analytics Storage Cost

NSG flow logs are a powerful tool for monitoring and analyzing network traffic. They operate at Layer 4 of the OSI model and record all IP flows going in and out of a network security group.

You can identify unknown or undesired traffic, monitor traffic levels and bandwidth consumption, and filter flow logs by IP and port to understand application behavior.

NSG flow logs are collected at 1-minute intervals through the Azure platform and don't affect your Azure resources or network performance in any way. They are written in JSON format and show outbound and inbound flows per network security group rule.

Each log record contains the network interface (NIC) that the flow applies to, 5-tuple information, the traffic decision, and (for version 2 only) throughput information.

NSG flow logs have a retention feature that allows deleting the logs automatically up to a year after their creation. This is available only if you use general-purpose v2 storage accounts.

There are two types of network security group rules: terminating and non-terminating. Each has different logging behaviors.

To manage NSG flow logs, you can use various tools and methods, including the Azure portal, PowerShell, Azure CLI, REST API, and Azure Resource Manager.

Here are some key properties of NSG flow logs:

Inbound TCP rules can be tricky to manage, especially when it comes to non-default security rules. These rules are implemented in a stateless way due to platform limitations.

You might notice that flows affected by non-default inbound rules become non-terminating, which can make it difficult to track byte and packet counts. This means the numbers reported in NSG flow logs and Network Watcher traffic analytics might not be accurate.

To resolve this issue, you can set the FlowTimeoutInMinutes property on the associated virtual networks to a non-null value. This will help you achieve default stateful behavior.

Setting FlowTimeoutInMinutes to 4 minutes can provide default stateful behavior, while setting it to up to 30 minutes can ensure long-running connections don't get disconnected.

Virtual network flow logs simplify the scope of traffic monitoring because you can enable logging at virtual networks. Traffic through all supported workloads within a virtual network is recorded.

Virtual network flow logs avoid the need to enable multiple-level flow logging, such as in network security group flow logs. Network security groups are configured at both the subnet and the network interface (NIC) in network security group flow logs.

Readers also liked: Azure Function Log

Virtual network flow logs support identification of traffic that Azure Virtual Network Manager security admin rules allow or deny. This is not supported in network security group flow logs.

We recommend disabling network security group flow logs before enabling virtual network flow logs on the same underlying workloads to avoid duplicate traffic recording and additional costs.

To share access to your NSG, you'll need an access key for Azure Blob Storage. This key will give you the necessary permissions to share your NSG flow logs.

You'll also need a storage token associated with the resources you want to share. This token will help you authenticate and authorize access to the shared logs.

To store the shared logs, you'll need to specify the name of the container where they'll be stored. This is where you'll be able to retrieve each PT1h.json blob, which contains the flow logs.

Here's a list of the necessary information to share access to your NSG:

Traffic Analysis and Monitoring is a crucial aspect of Azure VNet flow logs. You can identify top talkers in your network, combine with GeoIP data to identify cross-region traffic, and understand traffic growth for capacity forecasting.

To gain deeper insights, you can use flow data to verify network isolation and compliance with enterprise access rules. This is especially useful for meeting compliance requirements.

Here are some key benefits of traffic analysis and monitoring:

Identifying top talkers in your network is a crucial step in usage monitoring and optimization. By analyzing flow logs, you can easily identify the top talkers and understand their communication patterns.

Flow logs can be combined with GeoIP data to identify cross-region traffic, which is essential for capacity forecasting and optimizing network performance. This combination helps you understand where traffic is coming from and where it's going.

Understanding traffic growth is vital for capacity forecasting, and flow logs provide valuable insights into this aspect. By analyzing traffic growth, you can predict future network demands and make informed decisions about capacity planning.

Overly restrictive traffic rules can hinder network performance, and flow logs can help you identify and remove them. By analyzing flow logs, you can understand which rules are causing issues and make adjustments accordingly.

Here's a summary of the key benefits of usage monitoring and optimization:

By leveraging these benefits, you can optimize your network performance, improve user experience, and reduce costs.

Inbound internet traffic to VMs can be a concern, especially if you're not aware of the default SNAT settings in Azure.

VMs without public IPs will use default SNAT, which assigns an IP address to facilitate outbound connectivity.

You might see flow log entries for flows from internet IP addresses if the flow is destined to a port in the range of ports assigned for SNAT.

Azure doesn't allow these flows to the VM, but they are logged in Network Watcher NSG flow logs by design.

To prevent unwanted inbound internet traffic, it's recommended to explicitly block it with a network security group.

Load Balancer Traffic Distribution is a crucial aspect of traffic analysis and monitoring.

You can leverage two enrichment fields in the NTANetAnalytics table, SrcLoadBalancer and DestLoadBalancer, to examine traffic distribution.

Looking at the source IP, traffic going from the VM to the load balancer, can reveal interesting insights.

For instance, a specific IP address, 10.1.1.70, may not be getting much traffic.

Examining the time distribution can provide more context, perhaps the machine hasn't been active for a long time.

Alternatively, looking at the Destination IP can show a different picture.

In this case, traffic to certain VMs may indicate an issue with load distribution.

Analyzing these aspects can help identify potential problems with traffic distribution.

When working with Azure VNet Flow Logs, it's essential to understand how they interact with Private Endpoints and ExpressRoute.

Azure VNet Flow Logs can be used to monitor traffic from on-premises networks to Azure and vice versa, especially when using ExpressRoute.

With ExpressRoute, you can leverage fields like SrcExpressRouteCircuit and DestExpressRouteCircuit to track traffic.

Private Endpoints enable secure and private access to Azure services, which can be beneficial for organizations with strict security requirements.

Logging flows on an ExpressRoute gateway subnet is not recommended because traffic can bypass the gateway, such as with FastPath. This can lead to missing outbound flows to virtual machines.

You might think that enabling NSG flow logs on an ExpressRoute gateway subnet is a good idea, but it's not. If you do enable it, you won't capture outbound flows to virtual machines.

Outbound flows to virtual machines must be captured at the subnet or NIC of the VM, not at the ExpressRoute gateway subnet. This is because traffic can bypass the gateway, making it impossible to log flows at this point.

Intriguing read: Azure Vnet Gateway

ExpressRoute traffic is not tied to NSGs, just like Azure Firewall is not tied to NSGs. This means we can't use NSGs to filter ExpressRoute traffic.

We can, however, use VNet Flow Logs to monitor ExpressRoute traffic. Specifically, we can use the fields SrcExpressRouteCircuit and DestExpressRouteCircuit to track traffic between on-prem and Azure.

By leveraging prefix aggregation, we can show traffic from on-prem to Azure and Azure to on-prem using VNet Flow Logs.

Traffic can't be recorded at the private endpoint itself. You can capture traffic to a private endpoint at the source VM.

The traffic is recorded with the source IP address of the VM and destination IP address of the private endpoint. This allows you to track traffic flowing to a private endpoint.

You can use the PrivateEndpointResourceId field to identify traffic flowing to a private endpoint. This field provides a unique identifier for the private endpoint, making it easier to track and analyze traffic.

For more detailed information on how to use the Traffic analytics schema, see the relevant documentation.

Check this out: How to Use Windows Azure

To configure NSG flow logs, you can use various tools such as the Azure portal, PowerShell, Azure CLI, REST API, or Azure Resource Manager. These tools allow you to create, change, disable, or delete NSG flow logs.

NSG flow logs are collected at 1-minute intervals through the Azure platform and are written in JSON format, showing outbound and inbound flows per network security group rule. Each log record contains the network interface (NIC) that the flow applies to, 5-tuple information, the traffic decision, and (for version 2 only) throughput information.

To manage NSG flow logs effectively, consider enabling them on critical subnets and all network security groups attached to a resource. This will ensure that all traffic is recorded and can be used for auditing and security purposes.

Managing NSG flow logs is a crucial part of network security. You can create, change, disable, or delete NSG flow logs using various tools such as the Azure portal, PowerShell, Azure CLI, REST API, and Azure Resource Manager.

To manage NSG flow logs, you can navigate to the Network Watcher service and select NSG flow logs under LOGS. From the list of NSGs, select your VM(s) and under Flow logs settings, select On to enable the NSG flow logs.

NSG flow logs are collected at 1-minute intervals through the Azure platform, and they don't affect your Azure resources or network performance in any way. They are written in JSON format and show outbound and inbound flows per network security group rule.

You can export, process, analyze, and visualize NSG flow logs using tools like Network Watcher traffic analytics, Splunk, Grafana, and Stealthwatch. NSG flow logs include the following properties: network interface (NIC) that the flow applies to, 5-tuple information, the traffic decision, and (for version 2 only) throughput information.

To ensure all traffic is recorded, it's recommended to enable NSG flow logs on all network security groups attached to a resource. This is especially important in scenarios where you use multiple network security groups.

Here are some best practices for managing NSG flow logs:

By following these best practices, you can ensure that your NSG flow logs are properly managed and provide valuable insights into your network traffic.

To create a flow log, you can follow these steps.

First, navigate to the Network Watcher section in the Azure portal.

Next, select Flow logs under Logs.

Then, click on the + Create or Create flow log blue button.

On the Basics tab, enter or select the following values:

To enable traffic analytics, select the Next: Analytics button, or select the Analytics tab.

On the Analytics tab, enter or select the following values:

After entering all the required values, select Review + create. Review the settings, and then select Create.