Network security monitoring typically focuses on catching threats. But ask Zeek users what they actually discover in their logs, and a different picture emerges: chatty devices wasting bandwidth, misconfigurations creating millions of unnecessary DNS queries, and applications so poorly built they send data in HTTP headers instead of the body.
These aren’t security incidents. They’re operational problems that waste resources, create noise, and make it harder to spot actual threats. We asked the Zeek community to share their discovery stories – the moments when they noticed something odd and tracked it down. Here’s what they found and how they found it.
Start With Summary Data, Then Dig Into Specifics
One user was reviewing daily summary emails and noticed something unusual: close to 60% of traffic on their management VLAN was DNS. For a network segment that mostly handled switches and routers, that seemed high.
Looking into the DNS logs revealed the culprit: 120+ Cisco switches were “phoning home” every 15-20 seconds. Each attempt followed the same pattern – query a Cisco local domain (failed to resolve), append the internal domain and try again (failed to resolve again), append the internal domain a second time and try once more (failed to resolve again, again).
The fix was simple: turn off the “phone home” feature. The result? A reduction of 2.5 million daily DNS lookups on the internal subnets.
Summary data – traffic volume by protocol, top talkers, connection counts – surfaces anomalies that individual log entries hide. A spike in DNS traffic from infrastructure devices is a red flag worth investigating.
Filter for “What Should NOT Be Here”
One user had just installed a Zeek sensor and was browsing raw logs looking for anything notable to share. They opened syslog.log and decided to filter for any syslog being sent outside the network.
Several IP phones kept appearing, all sending logs to the same external destination. A quick lookup revealed the destination was associated with Church’s Chicken – likely an IP that belonged to their previous telco provider and had since been reallocated.
Working with the network engineer, they traced the problem to phones that had been repurposed but never reconfigured, leaking log data to unintended destinations.
The question “What should I NOT be seeing here?” is remarkably effective. Internal services sending data externally, management traffic on user VLANs, or protocols appearing where they shouldn’t all indicate problems worth investigating.
Look at HTTP Traffic Patterns, Not Just Destinations
During routine threat hunting through HTTP logs, someone noticed uncommon user agent strings that didn’t match anything they’d seen before. The traffic was internal-to-internal, which was somewhat reassuring, but still concerning enough to investigate.
Looking closer, they found lots of unique values that shared the same structure: long alphanumeric strings with delimiters. After parsing several days of data, they identified a pattern – key-value pairs, comma-separated, containing metadata about application transactions.
The discovery: a custom application was transferring data between resources using HTTP, but instead of sending data in the request body, a misconfiguration caused it to be sent in the header.
HTTP metadata reveals how applications actually work, not just where they connect. Unusual user agents, especially in internal traffic, often indicate misconfigured applications, debugging code left in production, or automation tools that shouldn’t be running.
Compare Similar Devices to Find Outliers
Several SNMP collectors were polling network devices for metrics. A new collector was added and performed very poorly. After ruling out firewall inspection issues, someone checked Zeek logs and compared the new collector’s behavior to the existing ones.
The problem: the new collector was doing individual SNMP queries instead of bulk queries like all the others. Same device type, same network path, completely different traffic pattern.
When you have multiple devices serving the same function, differences in their traffic patterns usually indicate configuration problems. One device generating 10x the connections of its peers is worth investigating.
Explore Logs You Normally Skip
Most analysts focus on conn.log and dns.log. How often do you check syslog.log? What about snmp.log, or smtp.log? Several discovery stories started with someone opening an unfamiliar log and finding problems immediately.
The Church’s Chicken discovery came from someone who decided to browse syslog.log – a log many analysts skip entirely. Within minutes of filtering for external destinations, they found IP phones leaking data.
Similarly, threat intelligence matching became practical for one team specifically because Zeek logs are consistent across protocols. As one user explained: “Zeek’s data is well-formatted, consistent, and once you can read one log the rest are not hard to learn.” Their analysts could finally scan diverse log types without constant maintenance.
The logs everyone checks are well-understood. The ones nobody checks often contain surprises. Operational problems hide in the places you’re not looking.
What These Discoveries Have in Common
According to a community poll, 35% of valuable discoveries start with proactive hunting or curiosity, not alerts or dashboards. The patterns that emerge:
- They start with questions, not rules: “Why is DNS so high?” “What’s being sent externally?” “Does this user agent make sense?”
- They focus on behavior, not just destinations: Traffic volume, protocol patterns, and metadata reveal problems that IP addresses and domains miss.
- They find operational issues that prevent security incidents: DNS noise masks tunneling attempts. Data in HTTP headers is an exposure waiting to be exploited. External logging is a data leak, even when accidental.
- They require looking, not just monitoring: These problems existed in the logs all along. Someone just had to open the file and ask if what they saw made sense.
Pick a log you don’t normally check this week. Filter for something that shouldn’t be there. Compare similar devices. You probably won’t find sophisticated attacks, but you’ll almost certainly find something worth fixing.
Thanks to everyone who participated in January’s Topic of the Month discussion on our Slack: Eric, Mark, Arne, Johannes, Tom, Jim, Dop, Chris, Sebastian, Fatema, Seth, Kevin, Aashish, David, and Van.
RSS - Posts