Introduction

Native client support is a fairly new feature in Azure Bastion, which allows users to use native SSH and RDP programs to connect to Bastion instead of using the Azure Bastion web interface. In this article, we explore how Azure Bastion Native Client support works, and how an adversary could abuse this feature to perform attacks against Azure VMs over private IP addresses, without having direct network connectivity to the VM.

What is a Bastion host?

Bastion hosts provide access to internal resources from an external network. They are generally used as an entry point into some zone in a private network. By funneling all traffic through these servers, administrators can limit network attack surface to a system that is hardened and heavily monitored.

Bastion hosts are generally single purpose systems that only listen on one port. No other software runs on these systems other than the service running at the port, such as ssh. This again reduces the attack surface of the system.

What is Azure Bastion

Azure Bastion is a managed Bastion host running in a customer’s Virtual Network. Microsoft markets it as a secure way to access internal virtual machines without exposing public IP addresses directly on those systems.

Since Azure manages the host for the customer, the customer does not need to worry about patching or management, and relies on Microsoft to ensure no vulnerabilities exist on the host or on the services running there. Additionally, users are required to authenticate to bastion using their Azure AD credentials, in addition to protocol specific authentication once a session is established with an internal VM.

Bastion supports RDP and SSH, and provides users with access to a browser based session for these protocols through the Azure Portal, based on Apache Guacamole. However, a new feature is available that allows users to connect via their native SSH or RDP client instead of the web interface, which is what this article is about.

Azure Bastion Native Client Support

Instead of logging in through the Azure Portal, Azure Bastion now allows users to connect using their native RDP or SSH clients. So how does this work under the hood?

Native Client Setup

According to Microsoft’s documentation, the user must use the Azure CLI to establish a connection using their native clients. There are two options for connecting to a VM over RDP, for example, through bastion. These two options look like the following:

az network bastion rdp --name "<BastionName>" --resource-group "<ResourceGroupName>" --target-resource-id "<VMResourceId>"

az network bastion tunnel --name "<BastionName>" --resource-group "<ResourceGroupName>" --target-resource-id "<VMResourceId or VMSSInstanceResourceId>" --resource-port "<TargetVMPort>" --port "<LocalMachinePort>"

Note that the second command specifies a local port and a “resource port”. In practice, both of these commands are the same, but the second command allows the user to define the connection in more detail.

By running the second command, the Azure CLI creates a tunnel and listens on a local port. By connecting to that local port with the native rdp client, the user can access the internal VM over rdp.

Reverse Engineering Native Client Support

TLDR; skip this section if you just want to know how the native client connections work. This section explains the process of reverse engineering the service.

The RDP session is established with the local system on the port specified in the command. Knowing this, we can infer that the Azure CLI is listening on our local socket, accepting connections, and forwarding all data received on those sockets towards the Bastion service, somewhere.

The easiest way to identify where this data is being sent is to look into this code in the Azure CLI, since it is open source.

After some digging, the code for this command in the Azure CLI can be found at the following file:

https://github.com/Azure/azure-cli/blob/main/src/azure-cli/azure/cli/command_modules/network/tunnel.py

def _listen(self):
        self.sock.setblocking(True)
        self.sock.listen(100)
        index = 0
        while True:
            self.client, _address = self.sock.accept()

            auth_token = self._get_auth_token()
            host = 'wss://{}/webtunnel/{}?X-Node-Id={}'.format(self.bastion.dns_name, auth_token, self.node_id)
            verify_mode = ssl.CERT_NONE if should_disable_connection_verify() else ssl.CERT_REQUIRED
            self.ws = create_connection(host,
                                        sockopt=((socket.IPPROTO_TCP, socket.TCP_NODELAY, 1),),
                                        sslopt={'cert_reqs': verify_mode},
                                        enable_multithread=True)
            logger.info('Websocket, connected status: %s', self.ws.connected)
            index = index + 1
            logger.info('Got debugger connection... index: %s', index)
            debugger_thread = Thread(target=self._listen_to_client, args=(self.client, self.ws, index))
            web_socket_thread = Thread(target=self._listen_to_web_socket, args=(self.client, self.ws, index))
            debugger_thread.start()
            web_socket_thread.start()
            logger.info('Both debugger and websocket threads started...')
            logger.info('Successfully connected to local server..')
            debugger_thread.join()
            web_socket_thread.join()
            self.cleanup()
            logger.info('Both debugger and websocket threads stopped...')
            logger.info('Stopped local server..')

In the above code, we can see that the Azure CLI is listening on a local port, accepting new connections, establishing a websockets session with a Bastion URL, and spinning off a thread to handle that communication between the new client and the websockets section.

This answers our initial question: Azure CLI is forwarding all tcp data through websockets towards the Bastion service. The next question is, how is that websockets tunnel established?

This question is quickly answered by forwarding all HTTP traffic from Azure CLI through a proxy (Burp Suite, in this case). In Burp, the following screenshot shows the series of requests that are issued, where the last request upgrades the user to a websocket connection:

In summary, the websockets session is established via the following process:

1. Use Azure Bearer token issued for Azure CLI to authenticate to abcdefg, and fetch a new Bastion token, specifying the Azure VM and remote port to connect to on the other side of the tunnel
2. Use Azure Bastion token to create a new Azure Bastion session by calling defghijk
3. Upgrade the HTTP session to a websockets session using the Azure Bastion token

End result: Application data can be sent through the websockets tunnel to the port specified on the internal Azure VM.

TLDR; how native client support works

The following diagram depicts the flow of traffic during an RDP session to Azure Bastion using the native client feature:

Upon receiving an API command to initialize a new session, Azure Bastion performs the following tasks:

1. Creates a TCP connection from the managed Bastion VMs towards the internal customer owned VM, on the port specified in the API request
2. Creates a session key associated with this port
3. Accepts a new websockets connection using this session key, and forwards all data from that websockets connection to the TCP connection on the internal VM

On the client side, the Azure CLI does the following:

1. Listens on a local port
2. Accepts new connections on that local port, and for each connection, calls the Bastion API to initialize the new session as described above
3. Creates a new thread that forwards all data received on the new connection towards the Bastion websockets tunnel

The Dangers of Azure Bastion Native Client Support

At this point, it is clear that Azure Bastion Native Client support feature is a bit misleading to administrators. By enabling this configuration, end users are able to communicate directly with the internal VM via a websockets connection, where data is sent bit-for-bit to the backend VM. This is completely different than a traditional Bastion host, which only speaks the protocol of the service used for remote access (rdp or ssh, in this case).

Traditionally, a client communicates directly to the hardened Bastion host on the remote access protocol. By passing all data through a websockets tunnel directly to the internal Azure VMs, we lost the security properties of the hardened external host, and expose potentially insecure internal resources directly to untrusted input from the remote host.

More concerning yet is the ability to choose the port on the internal VM that websockets data will be forwarded to. The implications of this design are the following:

1. Internal VMs can be port-scanned through Azure Bastion
2. Open ports with vulnerable services running on them can be accessed and exploited by anyone that can request a Bastion session for a specific VM.

To illustrate this point, the Azure CLI can be used to create a Bastion session against an internal Azure Windows VM running IIS:

The Burp websockets history shows the plaintext HTTP request that is tunneled through websockets to the remote system:

It is not hard to imagine a scenario where an attacker could abuse this functionality to perform web application attacks, for example, over the websockets tunnel. Similarly, vulnerabilities on other protocols could easily be exploited over this tunnel.

Attacker requirements for abusing Azure Bastion Native Client support

An attacker must have the following requirements satisfied in order to establish a websockets tunnel to an internal Azure VM:

1. Azure RBAC reader role assigned to the VM
2. Azure RBAC reader role assigned to the NIC associated with the private IP address
3. Azure RBAC reader role assigned to Azure Bastion service
4. The network must contain routes that allow the Azure Bastion subnet to communicate with the VM, either on the same VNet or across a VNet peering.

Architectural recommendations

1. Use Azure Policy to disable native client support for all Bastion services. An Azure Policy definition that audits or blocks this feature located here that may be applied to the root management group using the following commands:

az deployment mg create --location WestEurope --name deployBastionTunnelingPolicyDefinition --template-file bastionCheckNativeClientSupportDefinition.bicep -m <managementGroupId>

This creates an Azure policy definition, which will need to be assigned to the root management group in audit or deny mode.

If for some reason native client support cannot be disabled, the following mitigations will reduce the risk of this issue:

1. Ensure that the subnet Azure Bastion is deployed to has an NSG applied to it, which limits outbound connectivity to ports 3389 and 22
2. Limit the number of users that have RBAC reader access scoped to a high level resource in the Azure resource heirarchy, such as the root management group.
3. Include an NSG rule on the Bastion subnet that limits connectivity from Bastion to specific IP addresses or a small set of subnets.

Future research

This blog post is meant to bring visibility to the security challenges related to the native client support configuration on Azure Bastion. However, there is plenty of research and development to be done to follow up this abuse technique. The following list includes some of my ideas:

• Port scanning over Azure Bastion - Traditional port scanning does not work because the TCP connection terminates on the local system. To identify if a remote port is open, an attacker would need to perform service discovery scans over each websockets connection to fingerprint or identify an open service. This is possible, but it is not fast, and poses some challenges as an attacker. Further research into clever port scanning techniques over Azure Bastion would make this technique more effective.

• Protocol-specific tools for communicating directly over the websockets connection. While tunneling traffic through a local port and the Azure CLI works, it is a cumbersome process and slows down testing.