Categorygithub.com/jakubd/zeekparse
modulepackage
0.0.0-20210517003635-84c2bf2b83c0
Repository: https://github.com/jakubd/zeekparse.git
Documentation: pkg.go.dev

# README

Actions Status Go Report Card License

zeekparse

A log parser for common zeek text logs in Go. Allows you to query logs using logic written in Go.

This is a WIP and currently supports parsing the following logs:

  • dns.log
  • conn.log
  • http.log
  • ssl.log
  • x509.log

Use Case

This was made because I want to do data analysis on network logs I have been collecting which are mostly in Zeek/Bro IDS text format. These are compact files and can be retained for a longer period compared to full packet captures. Analyzing them quickly is typically done with zeekcut but I wanted to have more control over the logic in order to make it repeatable. This is what zeekparse is meant to do; parse the text logs that Zeek creates by default so that you can write your logic and analyze them in Go.

My plan is to support these logs: dns, conn, http, ssl, ssh and dhcp.

Status

  • handles gz compressed and uncompressed files
  • Can parse values from headers.
  • Can parse log entries into Go structures.
  • Can parse dns.log entries.
  • Can parse conn.log entries.
  • Can parse http.log entries.
  • Can parse ssl.log entries.
  • Can parse x509.log entries.

Still to-do

  • Can parse dhcp.log entries.

# Packages

No description provided by the author

# Functions

DateRange returns a slice of strings of the date strings (in format YYYY-MM-DD) between two datestrings of the same format.
DateStrToTime converts a datestring of the format YYYY-MM-DD to a proper time.Time object.
GetAllConnForDay returns all entries on the given day from the default zeek directory as a slice of parsed ConnEntry objects.
GetAllDnsForDay returns all entries on the given day from the default zeek directory as a slice of parsed DnsEntry objects.
GetAllHttpForDay returns all entries on the given day from the default zeek directory as a slice of parsed HttpEntry objects.
GetAllSSLForDay returns all entries on the given day from the default zeek directory as a slice of parsed SSLEntry objects.
GetAllX509ForDay returns all entries on the given day from the default zeek directory as a slice of parsed X509Entry objects.
GetZeekDir will return the default zeek dir or what is passed in.
IsMulticastOrBroadcastAddress will tell if the given string is either a multicast address or a broadcast.
LastXDays returns a slice of datestrings (in the format YYYY-MM-DD) from the last X months helpful for iterating recent results.
LastXMonths returns a slice of datestrings (in the format YYYY-MM-DD) from the last X months helpful for iterating recent results.
NewConnStateObj instantiates a new ConnStateObj with the given code.
ParseConnLog will parse through the given conn log (passed as a filename string).
ParseConnRecurse will parse through the given directory and recurse further down (passed as a directory string).
ParseDNSLog will parse through the given dns log (passed as a filename string).
ParseDNSRecurse will parse through the given directory and recurse further down (passed as a directory string).
ParseHttpLog will parse through the given single http log (passed as a filename string).
ParseHTTPRecurse will parse through the given directory and recurse further down (passed as a directory string).
ParseSSLLog will parse through the given single http log (passed as a filename string).
ParseSSLRecurse will parse through the given directory and recurse further down (passed as a directory string).
ParseX509Log will parse through the given single x509 log (passed as a filename string).
ParseX509Recurse will parse through the given directory and recurse further down (passed as a directory string).
PathRecurse is used for day recursion type functions.
StrBlankIfUnset is a convenience function for parsers that will return the given value or a blank string if it matches the unset char given.
TimeToDateStr Converts a time.Time object to a date str in the format YYYY-MM-DD.
UnixStrToTime will convert timestamps from unix format to a time.time.

# Constants

No description provided by the author
No description provided by the author
No description provided by the author
ZeekDateTimeFmt is the common format for zeek header datetimes.

# Structs

ConnEntry is a fully parsed conn.log line.
No description provided by the author
DnsEntry is a fully parsed dns.log line.
No description provided by the author
LogFileOpts stores vars in the header of the zeek log.
No description provided by the author
No description provided by the author
ZeekLogField is a generic zeek log field without casts.

# Type aliases

Proto is an enum of tcp protocol, either TCP or UDP.
ZeekLogEntry is a slice of fields referring to a single row in a log.