Azure Serial Console Safe Mode for Custom Windows Server Images

If you're deploying custom Windows Server images to Azure, you might be wondering how to troubleshoot issues that arise during boot.

Troubleshooting can be a challenge, especially when the boot process is non-standard.

The Azure Serial Console Safe Mode is a feature that allows you to access the serial console of your virtual machine (VM) and troubleshoot boot issues.

This feature is available for custom Windows Server images, which are a common use case for Azure.

To access the serial console, you need to enable Serial Console functionality for Windows Server. Newer Windows Server images on Azure have Special Administration Console (SAC) enabled by default.

SAC is supported on server versions of Windows, but not on client versions like Windows 10, Windows 8, or Windows 7.

To enable Serial Console functionality for Windows Server, you need to make sure boot diagnostics is enabled on your VM or virtual machine scale set. If you're using Azure, this is a crucial step to get your serial console up and running.

Newer Windows Server images on Azure have Special Administration Console (SAC) enabled by default, which is supported on server versions of Windows but not on client versions like Windows 10, Windows 8, or Windows 7.

Custom Image Enablement is a crucial step in ensuring your Windows Server images are fully functional.

For older Windows Server images created before February 2018, you can automatically enable the serial console through the Azure portal's run command feature. Select Run command, then choose the command named EnableEMS from the list.

To manually enable the serial console, connect to your Windows virtual machine by using Remote Desktop, then from an administrative command prompt, run the following commands:

In some cases, you may need to enable the SAC console offline. To do this, attach the windows disk for which you want SAC configured as a data disk to the existing VM, then from an administrative command prompt, run the following commands:

NMI calls are a powerful tool for debugging and troubleshooting Windows Server systems. They can be used to monitor for hardware issues that require specific response times.

A non-maskable interrupt (NMI) is designed to create a signal that software on a virtual machine won't ignore. Historically, NMIs have been used to monitor for hardware issues.

Programmers and system administrators often use NMI as a mechanism to debug or troubleshoot systems that are not responding. This can be a game-changer for resolving complex technical issues.

The serial console can be used to send an NMI to an Azure virtual machine by using the keyboard icon in the command bar. After the NMI is delivered, the virtual machine configuration will control how the system responds.

Windows can be configured to crash and create a memory dump file when receiving an NMI. This can provide valuable insights into the system's behavior.

If you're using an older Windows Server image, you can enable the serial console through the Azure portal's run command feature by selecting the command named EnableEMS.

To manually enable the serial console for Windows VMs or virtual machine scale sets created before February 2018, you'll need to connect to your Windows virtual machine using Remote Desktop.

From an administrative command prompt, run the following commands: bcdedit /set {default} bootdebug on and bcdedit /set {default} debugtype sac.

Reboot the system for the SAC console to be enabled.

Alternatively, you can enable the SAC offline by attaching the Windows disk to the existing VM and running the same commands from an administrative command prompt.

You can use the serial console to troubleshoot various issues with your Azure Linux VM. For example, if your FSTAB file is broken, you'll need to press the Enter key to continue and use a text editor to fix it, possibly in single user mode.

Pressing the Enter key will allow you to access the file and make the necessary corrections. If your firewall rules are causing issues, such as blocking SSH connectivity, you can use the serial console to interact with your VM without needing SSH.

This can be especially helpful if you're having trouble accessing your VM through SSH. To do this, you can use the iptables man page for more information. Similarly, if your firewalld is blocking SSH access, you can access the VM through serial console and reconfigure firewalld.

You can find more details on firewalld in the firewalld documentation. If your filesystem is corrupted, you can use the serial console to troubleshoot the issue, as seen in the serial console section of Azure Linux VM cannot start because of file system errors.

This can be a lifesaver if you're experiencing issues with your VM's filesystem. To interact with the bootloader, you can restart your VM from within the serial console blade to access GRUB on your Linux VM.

For more details and distro-specific information, see Use serial console to access GRUB and single user mode.

Azure Serial Console doesn't review, inspect, or store any data transmitted in and out of the virtual machine serial port.

There's no data to encrypt at rest because of this.

Host-based encryption is enabled by default for all Azure Serial Console connections, ensuring in-memory data paged to disks by virtual machines is encrypted.

Custom Boot Diagnostics Storage is a feature that allows you to store boot diagnostic data on a separate partition, making it easier to troubleshoot issues.

This feature is particularly useful for systems with limited storage space, as it helps to free up space on the primary drive.

By storing boot diagnostics on a separate partition, you can also improve system performance and reduce the risk of data corruption.

This is because boot diagnostics can take up a significant amount of space, and storing them on a separate partition helps to keep your primary drive organized.

You can configure Custom Boot Diagnostics Storage during the installation process, or later through the system settings.

To do so, you'll need to create a separate partition on your hard drive, and then configure the system to store boot diagnostics on that partition.

The benefit of Custom Boot Diagnostics Storage is that it allows you to troubleshoot issues more efficiently, and also helps to prevent data loss in the event of a system failure.

This is especially important for businesses that rely on their systems for critical operations, as it can help to minimize downtime and reduce the risk of data loss.

Azure Serial Console doesn't review, inspect, or store any data that's transmitted in and out of the virtual machine serial port, so there's no data to encrypt at rest.

In other words, your data is not being collected or stored by Azure Serial Console, which is a big plus for security.

To ensure that any in-memory data that's paged to disks by virtual machines running Azure Serial Console is encrypted, use the host-based encryption.

Host-based encryption is enabled by default for all Azure Serial Console connections, so you don't need to do anything extra to secure your data.

Accessing the serial console can be a lifesaver when dealing with system configuration issues. You can use it to interact with the bootloader, even if your SSH configuration is wonky.

To access GRUB on your Linux VM, simply restart your VM from within the serial console blade. This will give you a chance to troubleshoot any GRUB-related issues.

If your firewall rules are blocking SSH connectivity, you can use serial console to interact with your VM without needing SSH. This is especially useful if you've configured iptables to block SSH access.

You can also use serial console to reconfigure firewalld if it's blocking SSH access. More details on this can be found in the firewalld documentation.

In some cases, you might need to be in single user mode to fix issues with the FSTAB file. Press the Enter key to continue and use a text editor to fix the FSTAB file if necessary.

Here's a quick rundown of common scenarios where serial console can be particularly helpful:

Most Azure Linux distributions have the serial console configured by default. This means you can access the serial console without any extra setup.

Red Hat Enterprise Linux, CentOS, Debian, Ubuntu, and CoreOS all have serial console access enabled by default. This is a big plus for troubleshooting issues in the Azure serial console.

If you're using SUSE, the SLES images available on Azure have serial console access enabled by default.

The serial console is a powerful tool for troubleshooting and debugging issues in your Azure VM.