With Zeek 8.0, we’re taking the first step toward a major shift in cluster communication. A new ZeroMQ backend is now available, making it simpler to run clusters and opening the door for more flexible backends in the future.
Prefer a quick walkthrough? Check out our quick video explainer on the ZeroMQ backend in Zeek 8: Watch it here.
Zeek Clusters: A Quick History
Since roughly 2003 and version 0.8, Zeek has supported running as a multi-process cluster for horizontally scaling of packet processing. Communication between individual processes allows real-time state sharing in a Zeek cluster.
With version 2.4 released in 2015, Zeek’s cluster functionality was reworked towards a more generic publish-subscribe model based on the Broker project–a type-rich publish-subscribe library originating within the Zeek project. With version 2.6 at the end of 2018, Zeek was converted to use Broker exclusively.
Why We Reconsidered Broker
Starting with the Zeek 8.0 development cycle, we took a step back and reconsidered Broker as the communication library. From a high-level perspective, Broker provides similar or even the same primitives and concepts that other open-source messaging libraries and protocols already provide and implement. Popular keywords here are publish-subscribe-pattern, ZeroMQ, MQTT, RabbitMQ, STOMP or NATS.
Given the Zeek project’s focus on network monitoring and protocol parsing, we started experimenting leveraging off-the-shelf messaging libraries and prototyped alternative and pluggable cluster backends at the end of 2023. Zeek version 7.1 saw various infrastructure pieces fall in place as well as the addition of an experimental ZeroMQ cluster backend.
While Broker continues to be the default choice of cluster backend for Zeek 8.0, we’re planning to switch to the ZeroMQ backend by default with version 8.1, aiming for Zeek 9.0 to be the first LTS release with ZeroMQ as the default cluster backend.
What ZeroMQ Brings to Zeek
While Zeek ships with the ZeroMQ cluster backend, this effort resulted in better separation and abstraction of the cluster functionality. This allows advanced Zeek users to build their own cluster backends, leveraging any suitable technology applicable to their environment.
It is important to point out that many ideas and concepts from Broker have been carried forward. Indeed, on the scripting layer, the functions Broker::publish() and Broker::subscribe() have been replaced with Cluster::publish() and Cluster::subscribe()–this is mostly a cosmetic change, however–the topic-based publish-subscribe ideas continue to exist. Additionally, the on-the-wire serialization format introduced by Broker continues to be used in Zeek 8.0.
Compatibility and API Updates
A few APIs have been removed–these changes should surface during testing. For example, we’ve removed the Broker::auto_publish() API without replacement due to its impliciteness. The Broker-specific Python bindings will stop working with Zeek clusters configured to use ZeroMQ or other backends. If you have a use-case where an external application interacts with Zeek using those, consider switching to Zeek’s WebSocket API. While the serialization format of the WebSocket API version 1 can be a bit unusual, sending and receiving events from a Zeek cluster follows a straightforward pattern. Separately, if you’re experienced with ZeroMQ, check out the ZeroMQ Cluster Backend documentation and architecture sketch: We’d certainly appreciate feedback from other ZeroMQ users! For the majority of users, standard Zeek deployments should continue to work without changes.
There are differences in the low-level details of how Broker and ZeroMQ behave, but for most Zeek deployments these should be irrelevant. However, if you maintain complex cluster-aware Zeek scripts distributing state across a Zeek cluster using Broker::publish(), or you have been relying on certain ordering and timing guarantees, you may want to thoroughly test to ensure these continue to work correctly.
In our testing, common cluster setups behaved as expected, so most users should not notice a difference. Please reach out on Slack, Discourse, or Github if you find anything surprising–we may have overlooked certain cases.
Testing and Deployment Guidance
For Zeekctl users specifically, we’ve made it easy to switch to ZeroMQ with 8.0. Putting the following two lines into zeekctl.cfg and redeploying your cluster should do the trick
# zeekctl.cfg
ClusterBackend = ZeroMQ
UseWebSocket = 1If you are in a position to run A/B-testing of real-world Broker vs ZeroMQ deployments receiving the same traffic, any reports on performance and behavioral differences would be appreciated. Internal experiments indicate that in high-volume logging scenarios, the ZeroMQ implementation performs more efficiently.
We believe ZeroMQ is ready for production testing, and we welcome feedback to ensure a smooth transition when ZeroMQ becomes the default with 8.1.
RSS - Posts