Zeek: Getting started with Zeek (Docker-style): Part 1

Introduction

First of all, welcome to the community! If you are reading this, then you’ve heard all about how great Zeek is and you are interested in finding out for yourself by getting it up and running and playing with it. Good move!

The goal of this article is to make the process of getting Zeek up and running as easy as possible for you, so you can spend more time exploring Zeek, and less time running over speed bumps. To do this, we are going to leverage the extremely convenient Docker toolchain in combination with some cool Docker work that has already been done in the open source community.

Good luck and have fun!

Dockerizing your computer

Some good news: most of the work to get Zeek up and running has already been automated! However, in order to leverage that automation, we need to get your computer to the point where the foundation has been laid for the automation to kick off. Fortunately, that is quite easy: all you need is to have Docker installed and running. In the following, I am going to assume you are running a Mac OSX computer like me (Windows and Linux users, we love you too, but this is the only platform I have validated this on so far — the automation we will review will work on any platform that supports Docker — I just haven’t verified that yet!).

So, let’s get started. On your Mac, you’ll need to install the following:

Homebrew (installation instructions at https://brew.sh/)
Docker Desktop for Mac (installation instructions at https://docs.docker.com/docker-for-mac/install/)

You will also need to make sure you have git installed on your Mac. If it’s not, install it with the following command:

$ brew install git

We’re almost ready to get started. One thing we need to do first is to make sure we give Docker enough RAM to be able to compile Zeek. Docker Desktop for Mac uses a VM behinds the scenes to host the Docker runtime environment. By default it allocated 2 GB of RAM to the VM. That’s not enough to compile Zeek! If you try compiling with the default RAM settings, you will hit a compile error that looks something like this:

c++: internal compiler error: Killed (program cc1plus)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.

This is due to the VM hitting an Out Of Memory condition. To avoid this you will need to allocate more RAM to the VM. Click on the Docker Icon in your menubar and select “Preferences”. Click on the “Advanced” tab and then use the slider to select 8 GB of RAM (6 also works, but use 8 just in case). Docker Desktop will restart and then you will be ready to go!

Checkout the Code

The zeek-docker repo (https://github.com/zeek/zeek-docker) hosts a number of Dockerfiles that can be used to build your favorite version of Zeek. Dockerfiles like the one at https://github.com/zeek/zeek-docker/blob/master/3.0.0.Dockerfile have all the instructions you need to build your Zeek container in a completely repeatable, reliable, and convenient manner. It’s definitely worth taking a read through the Dockerfile to see the steps involved in building and installing Zeek.

If you are new to Docker, the docker documentation is worth checking out. The main documentation page is https://docs.docker.com/. If you are a first timer, the getting started section is very useful: https://docs.docker.com/get-started/. If you’ve had some experience, the Dockerfile reference is always handy to have nearby: https://docs.docker.com/engine/reference/builder/.

Ok, time to download the code so we can start doing cool stuff! Use the following command to clone the code onto your laptop:

$ git clone https://github.com/zeek/zeek-docker.git

Building your first Zeek Container

It’s time! Type in the commands below to build your first Zeek Container (in this case we are building a version 3.0.0 container):

$ cd zeek-docker

$ make build-stamp_3.0.0

That’s all you need to do! Now watch as the wonders of automation unfold, and your Zeek container is built. You should see something like this on your terminal console:

…

Step 24/24 : CMD /bin/bash -l

—> Running in c1263b7d2ea3

Removing intermediate container c1263b7d2ea3

—> 5bc774250a9a

Successfully built 5bc774250a9a

Successfully tagged broplatform/bro:3.0.0

touch build-stamp_3.0.0

Once the container has been built, you will see the container image is available in your local docker registry:

$ docker images | grep -e broplatform -e REPO

REPOSITORY TAG IMAGE ID CREATED SIZE

broplatform/bro 3.0.0 5bc774250a9a 8 minutes ago 215MB

Hello Zeek!

We are ready to run our first Zeek container!

The command below executes the container and sets up an interactive bash shell within the container.It also mounts the current host directory (currently inside the zeek-docker repo) in the /pcap directory of the container so you can copy files into/out of the container.

$ docker run -it -v `pwd`:/pcap broplatform/bro:3.0.0 /bin/bash

At this point we are inside the container. Type the command below and you can see there is a freshly built zeek executable ready to use!

root@3535953ccd99:/# which zeek

/zeek/bin//zeek

The /pcap directory inside the container corresponds to the host working directory (this can be anywhere on your computer, for me it happens to be the zeek-docker repo directory). You’ll probably want to put pcap files in here as well so you can process the data inside the container using zeek while storing the logs in the mounted directory for later access (e.g. after the container is gone!).

root@3535953ccd99:/zeek/bin# ls -1r /pcap

master.Dockerfile

master-dev.Dockerfile

common

build-stamp_3.0.0

Makefile

3.0.0.Dockerfile

…

Congratulations! You are now the proud owner of a brand new Zeek container, which contains an installed Zeek deployment in its own execution environment. (And you built it all by yourself !)

Putting Zeek to Work

There are two ways we can use Zeek:

Run it from the command line against a pre-captured PCAP file
Run it against a live network interface

For this post, we’ll be going through the first use case. In a follow up post, we’ll walk through how to run Zeek against a live network interface.

Now that our execution environment is all setup and we have a Zeek container, let’s process some PCAP files! (My colleague keith.jones@corelight.com has some interesting PCAPs you can take a look at in his article: https://blog.zeek.org/2019/12/how-to-add-jpeg-file-analyzer-to-zeek.html?m=1). Copy your favorite pcap file (let’s assume it’s “test.pcap”) into your current working directory and execute the following:

$ docker run -it -v `pwd`:/pcap broplatform/bro:3.0.0 /bin/bash

root@5ea58f4bb9be:/# cd /pcap

root@5ea58f4bb9be:/pcap# ls *.pcap

test.pcap

Cool! Now we’re running inside the container, and we can see our test.pcap file residing on our host computer via the mounted filesystem on our container’s /pcap.

Time to run Zeek! Execute the following command.

root@5ea58f4bb9be:/pcap# zeek -r test.pcap

root@5ea58f4bb9be:/pcap#

Done! Zeek has quietly processed the pcap file, and output a bunch of data to log files in the current directory. Let’s take a look.

root@5ea58f4bb9be:/pcap# ls *.log

conn.log dnp3.log packet_filter.log

So now we have the log files generated by Zeek using its default set of scripts, cool! If these logs are new to you, then you might want to head over to the official Zeek documentation which contains a deep dive on the various log files and fields: https://docs.zeek.org/en/stable/script-reference/log-files.html.

As an example, we’ll take a look at the contents of the conn.log file (https://docs.zeek.org/en/stable/scripts/base/protocols/conn/main.zeek.html#type-Conn::Info).

root@5ea58f4bb9be:/pcap# cat conn.log

#separator x09

#set_separator ,

#empty_field (empty)

#unset_field –

#path conn

#open 2019-12-04-04-38-26

#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes

resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes

resp_pkts resp_ip_bytes tunnel_parents #types time string addr port addr port

enum string interval count count string bool bool count string count count

count count set[string] 1252963725.444796 Cn0OCi3r0R0aO5nAda 192.168.10.204

1413 192.168.10.140 20000 tcp dnp3_tcp 1.049805 15 17 SF — 0

ShADadFtf 8 343 7 322 – 1252963725.788546 CbSrUpY4Adua4a6Ad 192.168.10.204

1400 192.168.10.21 5450 tcp – 9.006836 60 140 OTH -0 DdA 10 460 5 340 –

#close 2019-12-04-04-38-26

There you have it! Success!

Conclusiion

Here we are at the end of today’s journey, and the beginning of your Zeek journey! We have been able to Dockerize our computer, build a Zeek container, execute Zeek within the container, and process a PCAP file using the Zeek container we built.

I hope this has been useful to you, saved you a lot of time and helped to accelerate you on your Zeek journey. I also hope you had as much fun going through this as I had writing it!

I would love to hear from you, so please feel free to drop me a line (david.lapsley@corelight.com). I really appreciate feedback (especially if it helps to make this tutorial easier for folks to read/follow).

Thank you for following along, good luck, and Happy Zeeking!

About David Lapsley, Ph. D.

Dr. Lapsley has over 20 years of industry experience. Roughly a third of
that has been spent doing applied research for various government agencies,
a third working for large telecom vendors, and a third working at startup
companies. Dr. Lapsley’s expertise includes software development,
data analysis, distributed infrastructure, open source, cloud computing,
and forming and leading high performing teams.

Dr. Lapsley holds a Ph. D. in Electrical and Electronics Engineering from
the University of Melbourne, Australia. He also holds a B. S. (Computer
Science) and B. E. with Honours (Electrical and Computer Systems
Engineering) from Monash University, Australia.

Getting started with Zeek (Docker-style): Part 1

Dockerizing your computer

Building your first Zeek Container

Hello Zeek!

Putting Zeek to Work

Conclusiion

Share this:

Like this: