By Keith J. Jones, Corelight Sr. Security Researcher
Introduction and Background
Many modern VPN providers use the OpenVPN protocol in their clients and servers. Threat actors are also known to use OpenVPN. Zeek is unable to natively detect and parse the OpenVPN protocol but we can give it that functionality by writing a plugin that implements a protocol analyzer. This blog will discuss how to detect and analyze the OpenVPN protocol using Zeek and this plugin. The finished plugin and installation documentation is available here:
https://github.com/corelight/zeek-openvpn
The OpenVPN protocol analyzer plugin was written with the aid of Binpac (https://github.com/zeek/binpac) and we will assume that you are familiar with it. In addition, if you are not familiar with these Zeek resources they are recommended:
- https://zeek.org/2020/04/16/writing-my-first-protocol-analyzer/
- https://old.zeek.org/development/howtos/dpd.html
Resources specific to the OpenVPN protocol include:
- https://build.openvpn.net/doxygen/network_protocol.html
- https://openvpn.net/community-resources/openvpn-protocol/
- https://wiki.wireshark.org/OpenVPN
Understanding OpenVPN
Now that we have the foundation out of the way we will discuss the OpenVPN protocol so that we have enough information to detect and parse it. OpenVPN can run in two modes: TLS and shared secret. The protocol looks different on the network depending on which mode the server and client are running. In TLS mode, the OpenVPN is capable of supporting multiple users with different keys while the shared secret requires all clients to have the same pre-shared secret key. In most commercial applications, the OpenVPN server will be in TLS mode to support multiple users and that is what we will discuss here. In addition, the user can add additional security to the TLS authentication that translates into additional HMAC information included in OpenVPN messages.
Many of Zeek’s protocol analyzers only need one component. The OpenVPN client will try to connect to an OpenVPN server using either the TCP or UDP protocol, so in reality we will be writing four protocol analyzer components: UDP without HMAC, UDP with HMAC, TCP without HMAC, and TCP with HMAC. There are only a few different fields between TCP, UDP, HMAC, and no HMAC protocol data units (PDUs) that we must keep separate. We must keep these protocol analyzers separate because we do not have a field in the header to tell us which mode we are parsing while we are parsing. It is a chicken and egg problem for Binpac so we have to make four components to cover each situation.
All TCP OpenVPN messages begin with a “packet_length” field that is two bytes. Next, all OpenVPN messages will have a “MessageType” byte that consists of an opcode in the upper 5 bits and a key id in the lower 3 bits. The opcode will tell us what type of message this is and how to parse the rest of the message. The available opcodes are:
- P_CONTROL_HARD_RESET_CLIENT_V1 = 0x01,
- P_CONTROL_HARD_RESET_SERVER_V1 = 0x02,
- P_CONTROL_SOFT_RESET_V1 = 0x03,
- P_CONTROL_V1 = 0x04,
- P_ACK_V1 = 0x05,
- P_DATA_V1 = 0x06,
- P_CONTROL_HARD_RESET_CLIENT_V2 = 0x07,
- P_CONTROL_HARD_RESET_SERVER_V2 = 0x08,
- P_DATA_V2 = 0x09
There are three basic layouts of the rest of the message based on this opcode: CONTROL, ACK, and DATA messages. Control messages contain:
- Session ID
- Optional HMAC information
- Ack Packet ID Array Length
- Ack Packet ID Array
- Remote Session ID if “Ack Packet ID Array Length” > 0
- Packet ID
- Data
Similar to TCP, OpenVPN ACK messages contain the same information as a control message, but do not have a data field. Data messages come in two flavors: version 1 and version 2. Version 1 includes just the payload while v2 includes a peer ID before the payload. All of these structures are defined in the following file:
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpn-defs.pac
When an OpenVPN connection occurs, the client sends a control hard reset client opcode to the server. The server replies with a control hard reset server, and then both sides exchange TLS information via the P_CONTROL_V1 message type. After the TLS information has been exchanged, the VPN begins and the data is transmitted via OpenVPN data messages (either v1 or v2). It is this OpenVPN handshake that we will look for with our protocol analyzer. You can look at an example of a UDP OpenVPN session (without HMAC) in the following PCAP:
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/tests/Traces/openvpn.pcap
After associating Wireshark’s OpenVPN analyzer with UDP port 1198 you will see the following traffic:
You can use Wireshark to dissect the OpenVPN messages to verify the fields we will parse with Zeek:
Notice that the same type of message in UDP with HMAC includes additional fields in https://github.com/corelight/zeek-openvpn/blob/v0.0.2/tests/Traces/openvpn_udp_tls-auth.pcapng:
When HMAC is included we have three fields: the actual HMAC value, the packet-ID (which is different from the message packet-ID below it), and the net time. TCP mirrors UDP except there is a packet_length up front in https://github.com/corelight/zeek-openvpn/blob/v0.0.2/tests/Traces/openvpn_tcp_nontlsauth.pcap:
Note that the data inside the “P_CONTROL_V1” messages will contain SSL information we can send on to Zeek’s SSL protocol analyzer. We will also raise events in Zeek every time we see a CONTROL, ACK, or DATA message so that the user can do further processing on that information if they wish. Next, the zeek-openvpn plugin layout is discussed.
The zeek-openvpn Plugin Layout
The OpenVPN analyzer will have four components representing UDP/TCP and HMAC/No HMAC in our Plugin.cc:
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/Plugin.cc
We then define each of the components in their .h and .cc files:
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPN.h
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPN.cc
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPNHMAC.h
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPNHMAC.cc
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPNTCP.h
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPNTCP.cc
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPNTCPHMAC.h
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPNTCPHMAC.cc
Each of these components needs their binpac files defining their parsers:
- https://github.com/corelight/zeek-openvpn/blob/master/src/openvpn.pac
https://github.com/corelight/zeek-openvpn/blob/master/src/openvpn-protocol.pac
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpnhmac.pac
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpnhmac-protocol.pac
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpntcp.pac
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpntcp-protocol.pac
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpntcphmac.pac
https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpntcphmac-protocol.pac
Those binpac files refer to common binpac files across all OpenVPN modes:
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpn-defs.pac
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpn-analyzer.pac
Our plugin will need Zeek scripts and signature files. One such signature file is https://github.com/corelight/zeek-openvpn/blob/v0.0.2/scripts/zeek/openvpn/dpd.sig, which are the signatures used to activate this plugin. In dpd.sig, you can see that the different OpenVPN components are activated depending on the hard reset handshake we discussed previously. If one of these signatures match, the associated binpac component is activated.
Binpac keeps track of the OpenVPN handshake through two boolean variables: https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpn-analyzer.pac#L4. The hard resets are expected via the dpd.sig, then in this Binpac file P_CONTROL_V1 messages are expected in both directions before a P_DATA_* message confirms the connection as OpenVPN. It is only then that this binpac file calls “ProtocolConfirmed” and OpenVPN is officially added to the “service” field in the conn.log.
The new OpenVPN events can be found in: https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/events.bif. The record types can be found in: https://github.com/corelight/zeek-openvpn/blob/v0.0.2/scripts/types.zeek.
OpenVPN TLS Data
If we are careful, we are able to extract the TLS information in the OpenVPN P_CONTROL_V1 messages. We have to do this carefully because with UDP the messages may arrive out of order. Out of order UDP OpenVPN messages would be difficult to reconstruct in this plugin, so a sequence number is added to each component’s header file called “orig_seq” and “resp_seq” as so: https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPN.h#L36. If we receive messages that are out of order, these fields will help us detect that error. If the sequences are not in order, then we choose not to send any more data on to the SSL module for TLS processing. OpenVPN TCP connections should be ordered and this would not be a problem in theory, but we left the sequence check logic just to be sure.
Once a P_CONTROL_V1 packet has been detected, the data is forwarded via a forward SSL function:
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpn-defs.pac#L71
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpn-defs.pac#L89
In each component, this is wired up through similar lines as such:
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/openvpn-protocol.pac#L9
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPN.h#L27
- https://github.com/corelight/zeek-openvpn/blob/v0.0.2/src/OpenVPN.cc#L45
In this plugin, SSL’s “DeliverStream” function is used to pass the data on to that plugin for further TLS processing. Note that UDP is not supported in Zeek’s SSL module before v4.1, however TCP is.
Examples
There are PCAP files in the this directory that you can use for these examples: https://github.com/corelight/zeek-openvpn/tree/master/tests/Traces
In the first example we will print the events
$ cat test.zeek
@load zeek/openvpn
event OpenVPN::control_message(c: connection, is_orig: bool, msg: OpenVPN::ControlMsg) { print cat(msg); }
event OpenVPN::ack_message(c: connection, is_orig: bool, msg: OpenVPN::AckMsg) { print cat(msg); }
event OpenVPN::data_message(c: connection, is_orig: bool, msg: OpenVPN::DataMsg) { print cat(msg); }
$ zeek -Cr openvpn.pcap test.zeek
[opcode=7, key_id=0, session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=0, data_len=0, msg_type=7]
[opcode=8, key_id=0, session_id=\xe7\xa7~\xba|\x1cQ\xb5, packet_id_ack_array=[0], remote_session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id=0, data_len=0, msg_type=8]
[opcode=5, key_id=0, session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id_ack_array=[0], remote_session_id=\xe7\xa7~\xba|\x1cQ\xb5, msg_type=5]
[opcode=4, key_id=0, session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=1, data_len=277, msg_type=4]
[opcode=4, key_id=0, session_id=\xe7\xa7~\xba|\x1cQ\xb5, packet_id_ack_array=[1], remote_session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id=1, data_len=1174, msg_type=4]
[opcode=4, key_id=0, session_id=\xe7\xa7~\xba|\x1cQ\xb5, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=2, data_len=1174, msg_type=4]
[opcode=4, key_id=0, session_id=\xe7\xa7~\xba|\x1cQ\xb5, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=3, data_len=1108, msg_type=4]
[opcode=5, key_id=0, session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id_ack_array=[1], remote_session_id=\xe7\xa7~\xba|\x1cQ\xb5, msg_type=5]
[opcode=5, key_id=0, session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id_ack_array=[2], remote_session_id=\xe7\xa7~\xba|\x1cQ\xb5, msg_type=5]
[opcode=4, key_id=0, session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id_ack_array=[3], remote_session_id=\xe7\xa7~\xba|\x1cQ\xb5, packet_id=2, data_len=498, msg_type=4]
[opcode=4, key_id=0, session_id=\xe7\xa7~\xba|\x1cQ\xb5, packet_id_ack_array=[2], remote_session_id=\x9a\xa6\xb1\xe44\x88\x9a\xd3, packet_id=4, data_len=158, msg_type=4]
[opcode=4, key_id=0, session_id=\xe7\xa7~\xba|\x1cQ\xb5, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=5, data_len=228, msg_type=4]
…
$ grep openvpn conn.log
1613755368.960989 Cdgj2c1INQ1A2Hcflk 192.168.88.3 50568 46.246.122.61 1198 udp openvpn 44.271572 5825 8524 SF - - 0 Dd 57 7421 48 9868 -
In the case of TCP, we can pull TLS information from the SSL log (and hashes if we have JA3 loaded):
$ zeek -Cr openvpn_tcp_nontlsauth.pcap test.zeek
[opcode=7, key_id=0, session_id=Pz(\xa7\x82ux\x8d, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=0, data_len=0, msg_type=7]
[opcode=8, key_id=0, session_id=\x9a\xd7G\xbe\xb2M\x8a\x1b, packet_id_ack_array=[0], remote_session_id=Pz(\xa7\x82ux\x8d, packet_id=0, data_len=0, msg_type=8]
[opcode=5, key_id=0, session_id=Pz(\xa7\x82ux\x8d, packet_id_ack_array=[0], remote_session_id=\x9a\xd7G\xbe\xb2M\x8a\x1b, msg_type=5]
[opcode=4, key_id=0, session_id=Pz(\xa7\x82ux\x8d, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=1, data_len=100, msg_type=4]
[opcode=4, key_id=0, session_id=Pz(\xa7\x82ux\x8d, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=2, data_len=100, msg_type=4]
[opcode=4, key_id=0, session_id=Pz(\xa7\x82ux\x8d, packet_id_ack_array=[], remote_session_id=<uninitialized>, packet_id=3, data_len=26, msg_type=4]
[opcode=5, key_id=0, session_id=\x9a\xd7G\xbe\xb2M\x8a\x1b, packet_id_ack_array=[1], remote_session_id=Pz(\xa7\x82ux\x8d, msg_type=5]
[opcode=5, key_id=0, session_id=\x9a\xd7G\xbe\xb2M\x8a\x1b, packet_id_ack_array=[2], remote_session_id=Pz(\xa7\x82ux\x8d, msg_type=5]
...
[opcode=6, key_id=0, data_len=124, peer_id=<uninitialized>, msg_type=6]
[opcode=6, key_id=0, data_len=124, peer_id=<uninitialized>, msg_type=6]
[opcode=6, key_id=0, data_len=124, peer_id=<uninitialized>, msg_type=6]
[opcode=6, key_id=0, data_len=124, peer_id=<uninitialized>, msg_type=6]
[opcode=6, key_id=0, data_len=52, peer_id=<uninitialized>, msg_type=6]
$ cat conn.log
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path conn
#open 2021-03-05-15-08-12
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents
#types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string]
1358197736.781122 CW4BxU1d0sa8b429pg 192.168.56.103 39772 192.168.56.102 1194 tcp ssl,openvpntcp 32.021256 6986 7709 S1 - - 0 ShADad 100 12194 95 12657 -
#close 2021-03-05-15-08-12
$ cat ssl.log
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path ssl
#open 2021-03-05-15-08-12
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher curve server_name resumed last_alert next_protocol established cert_chain_fuids client_cert_chain_fuids subject issuer client_subject client_issuer ja3 ja3s
#types time string addr port addr port string string string string bool string string bool vector[string] vector[string] string string string string string string
1358197737.844659 CW4BxU1d0sa8b429pg 192.168.56.103 39772 192.168.56.102 1194 TLSv10 TLS_DHE_RSA_WITH_AES_256_CBC_SHA - - F - - T FIgTSu4XxFLZ2E4pV8,FBAmci2bnwWasltS9i FdSmvb4Z4RmNS42M3g,FHh4nUMj6vdVO6Jtf emailAddress=PRO3,name=PRO3,CN=PRO3,OU=PRO3,O=PRO3,L=PRO3,ST=OE,C=AT emailAddress=PRO3,name=PRO3,CN=PRO3,OU=PRO3,O=PRO3,L=PRO3,ST=OE,C=AT emailAddress=PRO3,name=PRO3,CN=PRO3-Client,OU=PRO3,O=PRO3,L=PRO3,ST=OE,C=AT emailAddress=PRO3,name=PRO3,CN=PRO3,OU=PRO3,O=PRO3,L=PRO3,ST=OE,C=AT f0d20361ae57a5c81d94ac774a736a52 7a2f70a16da750662fc0291d88ebddf8
#close 2021-03-05-15-08-12
Here this example shows TLS certificate information specific for this attack, but in other instances we were able to see the commercial VPN provider’s certificate. With the certificates, it is trivial to tell which commercial VPN services use OpenVPN on your networks. Next is an example of Private Internet Access VPN, a commercial service. Note that this PCAP was processed with the OpenVPN UDP SSL support coming in Zeek v4.1 in order to demonstrate future functionality here. Also note that pia.pcap is not included in the github repo as it is large and for demonstration purposes only.
$ zeek -Cr pia.pcap test.zeek
...
$ grep openvpn conn.log
1613754230.630672 CNNMEZ3azPqXJU1Sqb 192.168.88.3 38827 143.244.46.170 8080 udp openvpn,ssl 44.897121 20826 82406 SF - - 0 Dd 192 26202 170 87166 -
$ grep CNNMEZ3azPqXJU1Sqb ssl.log
1613754230.648009 CNNMEZ3azPqXJU1Sqb 192.168.88.3 38827 143.244.46.170 8080 TLSv12 TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 - - F - - T F5IkX217Cn7btTVSO9 (empty) name=newjersey416,CN=newjersey416,OU=Private Internet Access,O=Private Internet Access,L=LosAngeles,ST=CA,C=US emailAddress=secure@privateinternetaccess.com,name=Private Internet Access,CN=Private Internet Access,OU=Private Internet Access,O=Private Internet Access,L=LosAngeles,ST=CA,C=US
Conclusion
This blog introduced new capabilities of Zeek through the zeek-openvpn protocol analyzer plugin. This analyzer is a package that can be added to your existing Zeek installation to start detecting OpenVPN today! If this blog seemed complicated, it was mainly due to limitations to using Binpac. Luckily, coming soon is a tool called Spicy that will be replacing Binpac. Spicy will make creating protocol analyzers much easier than the process described in this blog. The documentation for Spicy can be found at: https://docs.zeek.org/projects/spicy/en/latest/.