Usage Notes of

The Bro 2.2 / 2.3

(a network security monitor)

William.L

wiliwe@gmail.com

2015-02-17

Index Basic Information.................................................................................................................................................. 3

Architecture & System Structure ........................................................................................................................ 4

Install Bro .............................................................................................................................................................. 6

Running Bro Without Installing .................................................................................................................. 8

Use Bro Tools ......................................................................................................................................................... 9

Inspect Log Files ......................................................................................................................................... 12

Script Files ................................................................................................................................................... 12

Add Network Application Filter Script..................................................................................................... 14

Read Packet Capture (PCAP) Files........................................................................................................... 15

Communicate With Bro System By Programming.......................................................................................... 17

Default Listen Port Number for Broccoli ................................................................................................. 17

Data Type Mapping between Bro Script and Broccoli Program............................................................ 17

Broccoli Library Documentation............................................................................................................... 18

Broccoli Library Path Setting under 64-bit Environment ...................................................................... 20

Reference ............................................................................................................................................................. 22

Basic Information The Bro official site - https://www.bro.org/index.html.

Bro is a powerful, passive, open-source network traffic analyzer and analysis framework that is much

different from the typical IDS (Intrusion detection system) you may know. It is NOT a classic

signature-based IDS (A signature based IDS will monitor packets on the network and compare them against a

database of signatures or attributes from known malicious threats. This is similar to the way most antivirus

software detects malware.)

Well grounded in more than 15 years of research, Bro has successfully bridged the traditional gap between

academia and operations since its inception.

Bro has originally been developed by Vern Paxson (http://www.icir.org/vern/).

Architecture & System Structure Bro was built based on event-based model. Bro is layered into two major components: event engine and script

interpreter.

Its event engine (or core) reduces the incoming packet stream into a series of higher-level events. These events

reflect network activity in policy-neutral terms, i.e., they describe what has been seen, but not why, or whether

it is significant. The event however does not convey any further interpretation, e.g., of whether that URI

corresponds to a known malware site; it is done by Bro’s second main component, the script interpreter.

The script interpreter which executes a set of event handlers written in Bro’s custom scripting language.

These scripts can express a site’s security policy, i.e., what actions to take when the monitor detects different

types of activity. More generally they can derive any desired properties and statistics from the input traffic.

Bro’s language comes with extensive domain-specific types and support functionality; and, crucially, allows

scripts to maintain state over time. Bro scripts can generate real-time alerts and also execute arbitrary external

programs on demand, e.g., to trigger an active response to an attack.

Bro system contains below components/tools:

Component Description Source Folder

BinPAC A protocol parser generator. Bro-Src-Root/aux/binpac

bro-aux Small auxiliary tools for Bro. Bro-Src-Root/aux/bro-aux

Broccoli The Bro Client Communication

Library.

Bro-Src-Root/aux/broccoli

BroControl An interactive shell for managing

Bro installations.

Bro-Src-Root/aux/broctl

broccoli-python Broccoli Python Bindings. Bro-Src-Root/aux/broccoli/bindings/broccoli-python

broccoli-ruby Broccoli Ruby Bindings. Bro-Src-Root/aux/broccoli/bindings/broccoli-ruby

BTest A unit testing framework. Bro-Src-Root/aux/btest

capstats A command-line tool collecting

packet statistics.

Bro-Src-Root/aux/broctl/aux/capstats

PySubnetTree A Python module for CIDR lookups. Bro-Src-Root/aux/broctl/aux/pysubnettree

trace-summary A script generating break-downs of

network traffic.

Bro-Src-Root/aux/broctl/aux/trace-summary

P.S: "Bro-Src-Root" used here is the Bro source folder. Take mine for example, "/home/william/bro-2.3.2"

Install Bro Here using install-from-source way and steps to build and install Bro tools are described in below link. It also

lists required and optional dependencies for building/compiling Bro source.

https://www.bro.org/sphinx/install/install.html

All operations are done under the Linux distribution Ubuntu 14.04 LTS 64-bit.

1) Download a copy of Bro source archive from the official site shown as below and extract the archive.

https://www.bro.org/download/index.html

Or using GIT to retrieve Bro source:

git clone --recursive git://git.bro.org/bro

The version used in this document is v2.3.2 (bro-2.3.2.tar.gz)

2) Change directory to Bro source root folder (here using "/home/william/bro-2.3.2" for example), configure

building environment and make(compile). There may be auxiliary tools and libraries available in the aux/

sub-directory. Some of them will be automatically built and installed along with Bro.

cd /home/william/bro-2.3.2

./configure --prefix=/home/william/bro

[Note: Cause to that the execution of Bro tools needs root privileges mode, so I configure it to install tools

into a folder named "bro" in my home directory. If you do not use a folder for installation, it will create

folders needed for Bro under your home directory. The default installation path is /usr/local/bro]

make

make install

If you want to uninstall Bro files(only remove script files), you could run below command(cause to that it uses

the Makefile in the sub-directory "build" of Bro source folder.

make -C build uninstall

Set the environemnt variable PATH to include the path to your installed Bro tools.

Ex:

export PATH=$PATH:/home/william/bro/bin

Running Bro Without Installing

For developers that wish to run Bro directly from the build/ directory (i.e., without performing make install),

they will have to first adjust BROPATH environment variable to look for scripts and additional files inside the

build directory.

Sourcing either build/bro-path-dev.sh or build/bro-path-dev.csh as appropriate for the current shell

accomplishes this and also augments your PATH environment variable so you can use the Bro binary directly:

./configure

make

source build/bro-path-dev.sh

bro <options> <script-file>

Use Bro Tools These are the basic configuration(configure files are under the folder Bro-Install-Path/etc ) changes to make

for a minimal BroControl installation that will manage a single Bro instance on the localhost:

# In Bro-Install-Path/etc/node.cfg, set the network interface for monitoring. The variable for setting interface

is “interface” and the network interface name could be found through running command “ifconfig -a”.

# In Bro-Install-Path/etc/networks.cfg, comment out the default settings and add the networks that Bro will

consider local to the monitored environment.

# In Bro-Install-Path/etc/broctl.cfg, change the MailTo variable for email address to a desired recipient and

the LogRotationInterval variable to a desired log archival frequency/period value.

1) Start the BroControl shell, type the command:

$ broctrl

2) When you run BroControl shell for the first time, please perform an initial installation of the BroControl

configuration:

[BroControl] > install

3) Then start up a Bro instance:

[BroControl] > start

Note:

<I> If you encounter a error whose message is similar to below, it means you need root privileges

"error: cannot acquire lock: [Errno 13] Permission denied: '/usr/local/bro/spool/lock"

<II> If it shows message, "bro terminated immediately after starting", it means there have errors and

you could view the detail through the command "diag."

[BroControl] > diag

, or you could inspect the error log file “Bro-Install-Path/logs/current/stderr.log.”

<II> The user starting BroControl needs permission to capture network traffic. If you are not root, you

may need to grant further privileges to the account you’re using. Follow the question and answer of

the Bro's FAQ Web page, https://www.bro.org/documentation/faq.html :

When bro executable runs normally, you could use ps command to observe it.

To stop this Bro instance you would do:

Q: How can I capture packets as an unprivileged user?

A: Fully implemented since Linux kernel 2.6.24, capabilities are a way of parceling super user privileges into

distinct units.

Attach capabilities required to capture packets to the bro executable file like this:

sudo setcap cap_net_raw,cap_net_admin=eip /path/to/bro

where "bro" is a Bro executable tool.

Example:

sudo setcap cap_net_raw,cap_net_admin=eip /home/william/bro/bin/bro

Now any unprivileged user should have the capability to capture packets using Bro provided that they have the

traditional file permissions to read/execute the bro binary.

[BroControl] > stop

Inspect Log Files

By default, logs are written out in human-readable (ASCII) format and data is organized into columns

(tab-delimited).

Logs that are part of the current rotation interval are accumulated in "Bro-Install-Path/logs/current/" (if Bro is

not running, the directory will be empty).

By default, BroControl regularly takes all the logs from "Bro-Install-Path/current/" and archives them to a

directory named by date, e.g. Bro-Install-Path/logs/2011-10-06. For example:

The frequency at which this is done can be configured via the LogRotationInterval option in

Bro-Install-Path/etc/broctl.cfg.

Some logs are worth explicit mention:

conn.log Contains an entry for every connection seen on the wire, with basic properties

such as time and duration, originator and responder IP addresses, services and

ports, payload size, and much more. This log provides a comprehensive record

of the network’s activity.

notice.log Identifies specific activity that Bro recognizes as potentially interesting, odd, or

bad. In Bro-speak, such activity is called a “notice”.

Script Files

Bro includes an event-driven scripting language that provides the primary means for an organization to extend

and customize Bro’s functionality. Virtually all of the output generated by Bro is, in fact, generated by Bro

scripts. It’s almost easier to consider Bro to be an entity behind-the-scenes processing connections and

generating events while Bro’s scripting language is the medium through which we mere mortals can achieve

communication.

Bro scripts effectively notify Bro that should there be an event of a type we define, then let us have the

information about the connection so we can perform some function on it.

Bro ships with many pre-written scripts that are highly customizable to support traffic analysis for your

specific environment. By default, these will be installed into Bro-Install-Path/share/bro/ and can be identified

by the use of a ”.bro“ file name extension.

These files should never be edited directly as changes will be lost when upgrading to newer versions of Bro.

The exception to this rule is the directory Bro-Install-Path/share/bro/site/ where local site-specific files can be

put without fear of being clobbered later. The other main script directories under Bro-Install-Path/share/bro/

are base and policy.

By default, Bro automatically loads all scripts under base (unless the -b command line option is supplied),

which deal either with collecting basic/useful state about network activities or providing frameworks/utilities

that extend Bro’s functionality without any performance cost.

Scripts under the policy directory may be more situational or costly, and so users must explicitly choose if they

want to load them.

The main entry point for the default analysis configuration of a standalone Bro instance managed by

BroControl is the "Bro-Install-Path/share/bro/site/local.bro" script. Adding customized process into this

script file.

Bro has script packages (e.g. collections of related scripts in a common directory). If the package directory

contains a "__load__.bro" script, it supports being loaded in mass as a whole directory for convenience.

Packages/scripts in the "base/" directory are all loaded by default, while ones in "policy/" provide functionality

and customization options that are more appropriate for users to decide whether they’d like to load it or not.

If one wants Bro to be able to load scripts that live outside the default directories in Bro’s installation root,

the BROPATH environment variable will need to be extended to include all the directories that need to be

searched for scripts.

Add Network Application Filter Script

Under the folder "Bro-Install-Path/share/bro/policy/misc/app-stats/plugins"

1) Copy a Bro script to a new one. For example:

cp facebook.bro amazon.bro

2) Change the filtering condition for Amazon site.

3) In "__load__.bro", add a line "@load ./amazon"

4) Using Web browser to link to Amazon site, wait for a while and view the log file

”Bro-Install-Path/logs/current/app_stats.log.“

Read Packet Capture (PCAP) Files

Capturing packets from an interface and writing them to a file can be done like this:

sudo tcpdump -i en0 -s 0 -w mypackets.trace

where en0 can be replaced by the correct interface for your system as shown by e.g. ifconfig. (The -s 0

argument tells it to capture whole packets; in cases where it’s not supported use -s 65535 instead). After a while

of capturing traffic, kill the tcpdump (with ctrl-c), and tell Bro to perform all the default analysis on the capture

which primarily includes :

Or, you could use Wireshark/Ethereal(Linux/Windows) or Microsoft Network Monitor(Windows) to capture

and saved packets into a PCAP format file.

(P.S: Microsoft Network Monitor - http://www.microsoft.com/en-us/download/details.aspx?id=4865)

Run below command to read PCAP file and Bro will output log files into the working directory.

bro -r mypackets.trace

For example:

If you are interested in more detection, you can again load the local script that we include as a suggested

configuration:

bro -r mypackets.trace local

To view the filtering result for application

cat app_stats.log

Communicate With Bro System By Programming Q: What is Broccoli?

A: BRO Client COmmunications LIbrary.

It allows you to write applications that speak the communication protocol of the Bro intrusion detection system

for exchanging Bro events with external programs. Broccoli is free software under terms of the BSD license as

given in the COPYING file distributed with its source code.

From my experiment result of using Bro v2.3 Broccoli and the below link, I changed to use the version of

Broccoli of Bro v2.2.

http://bro.bro-ids.narkive.com/XaJeX1aM/broccoli-not-processing-events

Default Listen Port Number for Broccoli

The default port number for listening Broccoli connecting request is 47760 and it could be confirmed through

running netstat tool:

netstat -ant

If you want to change the port number, it could change the port number value in the Python file “options.py”

under Bro installation path.

After changing the port number, run “broctl” to invoke Bro controller, execute “install” after prompt

“[BroControl] >” to re-generate configuration file and then execute “restart” to restart Bro daemon.

Using netstat tool to verify if the port number is what you set.

Data Type Mapping between Bro Script and Broccoli Program

When you want to test an event provided by Bro scripts(.bro file) in program, it needs to convert data type of

parameters in event handler of Bro script. Bro official site provides this mapping shown as below:

Broccoli Library Documentation

If you want to browse Broccoli library in detail, it provides documentations that could be generated through

Doxygen (http://www.stack.nl/~dimitri/doxygen/). In the Broccoli source folder, there has a sub-folder named

“doc” containing Doxygen configuration file named “Doxyfile”

Change directory to the “doc” sub-folder and run following command to generate HTML-based broccoli

documentation:

doxygen ./Doxyfile

After the generating process accomplished, it create a folder named “html” under “doc” sub-folder.

In “html” folder, the main Web page is index.html, open it in your Web browser to browse Broccoli’s data

structures and functions.

Broccoli Library Path Setting under 64-bit Environment

In 64-bit Linux, it needs to set Broccoli SO (dynamic) library path manually, otherwise a Broccoli application

will fail to run as below.

It could use ldd (List Dynamic Dependencies) tool to see what resulted in this fail:

the loader could not find the location to the Broccoli SO library, libbroccoli.so.

To resolve this problem, it needs to set Broccoli SO library path properly.

In CentOS, add soft links to Broccoli SO library files under folder /usr/lib64.

In Ubuntu, add soft links to Broccoli SO library files under the folder /lib/x86_64-linux-gnu or

/use/lib/x86_64-linux-gnu.

After setting correct path to Broccoli, it could use ldd tool to verify again.

Reference * Official Site

https://www.bro.org/

* On-line Reference/Documentation

https://www.bro.org/sphinx/

* Broccoli library

# https://www.bro.org/sphinx/components/broccoli/README.html

# https://www.bro.org/sphinx/components/broccoli/broccoli-manual.html

* The paper for Bro IDS

<I> http://www.icir.org/vern/papers/bro-CN99.html

<II> ftp://ftp.ee.lbl.gov/papers/bro-CN99-new.pdf.gz