ACSEAlienVault Certified Security Engineer

1

Thursday, May 3, 12

2Thursday, May 3, 12

About this document ACSE (AlienVault Certified Security Engineer)

Author: AlienVault Training Team (trainers@alienvault.com)

Document Version 1.0

Last revision: 01/2012

Product version used: 3.1

Copyright Alienvault 2012All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying,

recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and publisher.

Any trademarks referenced herein are the property of their respective holders

3

Thursday, May 3, 12

Contents Installation

Updates

CLI

Event Collection

Data Sources

Policies & Actions

Logical Correlation Directives

Logger

IDM

HIDS

Secure Connection

Snort

Dimensioning and Deployment

4

Thursday, May 3, 12

Bubba Throughout the document Bubba will give you useful hints and

links for further documentation

We come in peace and security!

5

Thursday, May 3, 12

AlienVault InstallationGetting to speed

6

Thursday, May 3, 12

Products AlienVault Installations

Appliances- Sensors (X1000, X2000, X3000, X4000)

- Loggers (L1000, L2000, L3000)

- SIEM (S1000, S2000, S3000)

Software- analog to the Appliances range, installable on custom hardware

custom restrictions, special purpose environments, etc.

Preferred: AlienVault Appliances- optimum performance and compatibility

7

Thursday, May 3, 12

Installation Guide Find the Installation Guide here

http://www.alienvault.com/docs/Installation_Guide.pdf

8

Thursday, May 3, 12

Hardware recommendations For a production system:

At least 4GB Ram 64 Processor DUAL Core Processor

Depending on the amount of traffic being monitored and the amount of data captured RAM has to be increased, always avoiding SWAP memory usage.

If we dont have the appropriate hardware:

"Divide et vinces"

9

Thursday, May 3, 12

Network hardware Requires Intel E1000 cards for capturing

performance performance performance

Administration interface can be any card with no known problems

10

Thursday, May 3, 12

Best practice Always use the latest installation image

If you need performance you cant use any hardware

disable unused data sources

for the time of installation and customization, try to stick to english for faster support

dont use Sniffers (ntop, snort, p0f) on interfaces without tap or port/mirror

Performance is crucial when sniffing is involved. Get rid of unneeded CPU hoggers!11

Thursday, May 3, 12

Best practice: VmWare VMware installations are popular but

minimal memory: 8GB minimal number of CPUS: 4 take care that all resources are bound singularly to the AlienVault

guest system

12

Thursday, May 3, 12

Best practice: Partitioning SIEM only

use 50-100 GB of disk space for /var/lib/mysql use the maximum available remaining space for /var

- REASON: /var/lib/mysql keeps all the configuration

Logger only

use at least twice the space required per log interval for /var- 50 GB expected logs/day => /var => 100GB

use the remaining rest for /var/ossim/logs- REASON: if /var runs out of space /var/ossim/logs remains intact

13

Thursday, May 3, 12

Best practice: Partitioning Use a small amount of space for the operating system

50-100 GB are absolutely sufficient be sure to configure enough swap space

golden rule: 2x amount of RAM additional swap can be configured later with a loopback mount

the storage killer: /var use at least 80% or more for this partition depending on the usage you may sub-partition /var further

14

Thursday, May 3, 12

Best practice: Partitioning Sensor only

50-100GB for / 2x physical memory (RAM) for swap remaining space: /var

Example: SIEM - S1000

15

path filesystem size partition

/boot ext2 2GB #1

/ ext3 100GB #2

swap swap 16GB #3

/var ext3 834GB #4

/var/lib/mysql ext3 100GB #5

Thursday, May 3, 12

Installation profiles Selecting the server role

Sensor Server (includes SIEM & Logger) Database Framework

can be changed during installation or anytime after installation

16

Thursday, May 3, 12

Profile: Sensor enables Sensor functionality

needs access to all the networks being monitored receives all the network traffic

Port Span- needs to be configured separately

Tap device Hub

17

Thursday, May 3, 12

Profile: Sensor out of the box enabled data sources

Snort (Network Intrusion Detection System)

Ntop (Network and usage Monitor)

OpenVAS (Vulnerability Scanning)

P0f (Passive operative system detection)

Pads (Passive Asset Detection System)

Arpwatch (Ethernet/Ip address parings monitor)

OSSEC (Host Intrusion Detection System)

Nagios (Availability Monitoring)

OCS (Inventory)

18

Thursday, May 3, 12

Profile: Server combines SIEM functionality

correlation risk assessment etc.

with Logger functionality

forensic long-term storage digitally signed

The server installation profile also comes with a Sensor with limited functionality to monitor the Server itself

19

Thursday, May 3, 12

Profile: Database enables a mysql server for usage with Server components

is required in most installations

SIEM only- event and alarm storage

- metadata

- framework

Logger- configuration

- metadata

- framework

20

Thursday, May 3, 12

Profile: Framework enables the Web front-end for the server

is installed on most appliances

low overhead useful in emergency situations

- power outage on central console

- connection to central console lost

- usage as a per department or per location console

21

Thursday, May 3, 12

Profile: All-In-One Enables all AlienVault components on a single appliance

Server (SIEM + Logger) Database (Configuration and event storage + correlation) Sensor Framework (Web-Interface)

Useful for:

Testing Evaluation Small deployments

activated on every automated install

22

Thursday, May 3, 12

Installation methodsAutomated installation

1. Boot the installation system

2. Configure networking

3. Create and mount the partitions on which AlienVault will be installed

4. Watch the automatic download/install/setup/update of the base system.

5. Set up users and passwords

6. Load the newly installed system for the first time

Custom installation

1. Boot the installation system

2. Select the installation language

3. Configure keyboard

4. Configure location

5. Select the installation AlienVault profiles for this installation

6. Configure networking

7. Create and mount the partitions on which AlienVault will be installed

8. Enter the professional license

9. Watch the automatic download/install/setup/update of the base system.

10.Set up users and passwords

23

Thursday, May 3, 12

Installation checklist Rack Space

Power

Network Configuration

Port mirroring IP addresses

Professional Key

Internet Access (Required when installing the professional version)

Role of the device to install

24

Thursday, May 3, 12

INSTALL

25

Thursday, May 3, 12

Installation: Next steps Point your web-browser to https://siem_ip/

username/password = admin Use ssh to login to the appliance

same password as in installation dialog

Be sure to note down the root password you enter in the installation process.26

Thursday, May 3, 12

Hands-On: Installation Fill in the following tables with the partitioning data for the given

profiles

Logger, 4TB of overall disk capacityLogger, 8TB of disk capacityLogger, 8TB of disk capacityLogger, 8TB of disk capacity

Mountpoint Capacity Comment

SIEM, 4 TB of disk capacitySIEM, 4 TB of disk capacitySIEM, 4 TB of disk capacity

Mountpoint Capacity Comment

27

Thursday, May 3, 12

Hands-On: Installation Given the following SIEM design, which profiles need to be

installed on which machines?

SIEM & ConsoleProfiles:__________________________________________________________________________________________________________________

Sensor:Profiles:__________________________________________________________________________________________________________________

Logger:Profiles:__________________________________________________________________________________________________________________

28

Thursday, May 3, 12

AlienVault UpdatesKeeping your system up to date

29

Thursday, May 3, 12

AlienVault: Update channels AlienVault uses the Advanced Packaging Tool (apt) for software

maintenance reliable

tested

Every AlienVault Appliance is configured to retrieve updates

Base System: Debian repositories AlienVault Software: AlienVault repositories

- Binary and package updates

/etc/apt/sources.list.d/alienvault-pro.list

- AlienVault professional feed

/etc/apt/sources.list.d/alienvault-etpro-pro.list

Get more info on how APT works here: http://en.wikipedia.org/wiki/Advanced_Packaging_Tool30

Thursday, May 3, 12

Updating Process Important files (They should never be modified)

/etc/apt/sources.list- Contains the different software repositories

/etc/apt/preferences- Contains the priority configuration between the different repositories

The update system in AlienVault:

Updates the AlienVault components as well as the Debian base system

Allows the AlienVault development team preventing software packages from being upgraded (Unstable versions, software causing troubles to other users.

31

Thursday, May 3, 12

Update AlienVault The system notifies in the Web interface the availability of new

versions of the AlienVault components

The system notifies the management console on the availability of new updates to the components of AlienVault

If the update procedure requires any manual change, it will be explained in the updates notification system

To update the whole system, use the following command:

# alienvault-update

debugging: alienvault-update -v

32

Thursday, May 3, 12

Update AlienVault During the update process:

When prompted always select installing the latest configuration files available

Once the system has been updated, if something is not working run:- # alienvault-reconfig

Snort rules, OpenVas scripts, directives and plugins will be updated automatically using software packages

33

Thursday, May 3, 12

Update AlienVault In case new Snort or OpenVas rules are included manually, you will

need to run the following scripts to update the information in the database

Snort:- # perl /usr/share/ossim/scripts/create_sidmap.pl /etc/snort/rules

OpenVas:- # perl /usr/share/ossim/scripts/update_nessus_ids.pl

And restart the AlienVault Server

# /etc/init.d/ossim-server restart

34

Thursday, May 3, 12

Package Management Following apts principle several tools are available to install

packages and monitor installed software

apt-cache- search for packages (apt-cache search )

- show package requirements, versions etc.

apt-get- install individual packages

dpkg- show installed packages

The Debian APT howtos:http://www.debian.org/doc/manuals/apt-howto/35

Thursday, May 3, 12

Hands-On: Updates Update the system

review the software sources in /etc/apt/sources.list.d

Which versions of the following software is installed:

ossim-server ___________________________ ossim-agent ___________________________ mysql-server-core ___________________________ ossim-framework ___________________________

36

Thursday, May 3, 12

AlienVault CLIpower to the user

37

Thursday, May 3, 12

AlienVault: Base System AlienVault Appliances are built around 64 bit Debian 5 Linux

Open Reliable Secure Innovative

AlienVault uses many packages from Debian

where possible AlienVault repositories

for AlienVault software packages customized Debian packages

38

Thursday, May 3, 12

AlienVault: Filesystem Current standard installation is based on ext3 filesystem

recommended filesystem for usage of the AlienVault software journaled stable

39

Thursday, May 3, 12

AlienVault: system access Get access to the platform

SSH access with superuser root- use the password supplied by AlienVault or the one from the installation

process

HTTPS access with administrative user admin- default password is admin

- can be switched to normal HTTP

40

Thursday, May 3, 12

Configuration Files AlienVault

OSSIM Server: /etc/ossim/server/config.xml OSSIM Agent: /etc/ossim/agent/config.cfg Frameworkd: /etc/ossim/framework/ossim.conf

Snort /etc/snort/snort.ethN.conf

OpenVas /etc/openvas/openvasd.conf

Nagios /etc/nagios3/

Database

/etc/mysql/my.cnf

41

Thursday, May 3, 12

Configuration Files System startup

/etc/rc* Logrotate

/etc/logrotate.d Network configuration

/etc/network/interfaces DNS configuration

/etc/resolv.conf Rsyslog

/etc/rsyslog.conf Monit

/etc/monit/monitrc

42

Thursday, May 3, 12

AlienVault: Services Stop a service

# /etc/init.d/ stop

# service stop

Start a service # /etc/init.d/ start

# service start

Restart a service # /etc/init.d/ restart

# service restart

The parameters that the services will use when starting are usually configured in the following path:

# /etc/default/ 43

Thursday, May 3, 12

alienvault-setup /alienvault-reconfig

ossim-reconfig

AlienVault Components

Database

Integrated Tools

OS Components

ossim_setup.conf

44

Thursday, May 3, 12

ossim-setup.conf interface: Network management interface (eth0, eth1...

language: Language used within AlienVault (en, es, fr...)

profile: Profile or Profiles enabled in the system

version: Version of AlienVault in use

hostname: Name of the system

admin_ip: IP address to manage the system

first_init: Variable to check whether is the first boot or not

email_notify: e-mail to receive notifications

45

Thursday, May 3, 12

alienvault-setup (Database) acl_db, event_db, ossim_db, osvdb_db, ocs_db: Name for the

different databases

db_ip: IP address of the Database

db_port: Listening port of the Database

pass: Password of the Database

type: Type of Database

user: User in the Database

create: If i is set to yes, database will be deleted and created again when running alienvault-reconfig

46

Thursday, May 3, 12

ossim-setup.conf (Sensor) detectors: Enabled detector plugins (Separated by comma and

using the same name that the plugin configuration file has)

interfaces: Listening interfaces (Separated by comma)

ip: IP address that the sensor will use to connect to the AlienVault Server

monitors: Enabled monitor plugins (Separated by comma and using the same name that the plugin configuration file has)

name: Name of the sensor

networks: Local networks that will be monitored from that sensor (In CIDR format and separated by comma)

47

Thursday, May 3, 12

ossim-setup.conf FRAMEWORK

framework_ip: IP address of the Web interface framework_port: Listening port of the frameworkd daemon

SERVER

server_ip: Listening IP address of the AlienVault Server server_port: Listening port of the AlienVault Server server_license: Professional license code server_plugins: Enabled plugins in the Server profile

48

Thursday, May 3, 12

ossim-setup.conf SNMP

snmpd: Enable the Snmp daemon snmptrap: Enable Snmp traps collection community: Snmp community

FIREWALL

active (firewall): Enable or disable iptables VPN

vpn_infrastructure: Enable or disable the VPN between the OSSIM components

vpn_net: VPN Network vpn_port: VPN Port

49

Thursday, May 3, 12

Agent Configuration /etc/ossim/agent/config.cfg [daemon]

daemon: Daemon mode (True or False)

pid: Path to the PID file (Process identifier)

[event-consolidation]

Enable events consolidation at Sensor level

by_plugin: List of plugins that will be consolidated

enable: Enable or disable (True or False)

time: Wait n seconds to consolidate the events before sending them

50

Thursday, May 3, 12

Agent Configuration /etc/ossim/agent/config.cfg [log]

Configures the verbose level and the path to the different log files error: File in which the error events will be stored file: File in which all the agent logs will be stored stats: File in which the agent stats will be stored (Every 5 minutes) verbose: Configures the verbose level (Debug, Info, Warning, Error or Critical)

[output-plain]

Writes in a log file what is being sent to the AlienVault Server (Useful for debugging and developing purposes)

enable: Enable or disable (True or False) file: File in which the output-plain will be stored

51

Thursday, May 3, 12

Agent Configuration /etc/ossim/agent/config.cfg [output-server]

Configures the server to which events are sent enable: Enable or disable sending events to the server (True or

False)

ip: IP address of the AlienVault Server (Logger or SIEM) port: Listening port of the AlienVault Server (Logger or SIEM)

[plugin-defaults]

In this category variables can be defined to be used in the plugins configuration.

52

Thursday, May 3, 12

Agent Configuration /etc/ossim/agent/config.cfg

[plugins]- Defines which Data Source Connectors (detectors and monitors) are

enabled

- name_of_the_plugin=path_to_the_plugin_config_file- device= /etc/ossim/agent/plugins/device.cfg

[watchdog]- Monitor the process associated to each plugin (In case it is running in

the same machine)

- enable: Enable or disable (True or False)- interval: Wait X seconds between checks- restart_interval: Restart the process every X seconds (This has to be

enabled in each plugin)

53

Thursday, May 3, 12

Agent Configuration /etc/ossim/agent/aliases.cfg

Contains predefined regular expressions that can be used when creating new plugins

Data Source Connectors configuration files

/etc/ossim/agent/plugins/*.cfg

54

Thursday, May 3, 12

Data Source Connectors Detector plugin configuration (/etc/ossim/agent/plugins/*.cfg)

[DEFAULT] Any var defined inside this category will be sent to the AlienVault

Server plugin_id: Numerical identifier of the plugin within the AllienVault

system (Data Source ID) [config]

type: detector enable: Enable or Disable plugin (It must be enabled in config.cfg) source: Source of the events (log, database, wmi) location: File in which logs can be found create_file: Create the log file in case it does not exist

55

Thursday, May 3, 12

Data Source Connectors Detector plugin configuration (/etc/ossim/agent/plugins/*.cfg)

[config]- process: Name of the process generating logs (If the process is

running in the same system)

- start: Start the process when the agent starts (yes/no)- stop: Stop the process when the agent stops (yes/no)- startup: Command that starts the process- shutdown: Command that stops the process

The next part of the configuration files includes the regular expressions that collect and normalize the events.

56

Thursday, May 3, 12

Agent Configuration The different configuration variables defined in the config file can be used with the

following syntax to help defining new variables:

%()s

When the variable has been defined in the same file: process=pads

shutdown=killall -9 %(process)s

\_CFG()

When the variable has been defined in the main configuration file(config.cfg)

In /etc/ossim/agent/config.cfg file:restart_interval=3600 ; seconds between plugin process restart

In the Data Source Connector configuration file:restart_interval=\_CFG(watchdog,restart_interval)

57

Thursday, May 3, 12

Server Configuration /etc/ossim/server/config.xml

Path to the AlienVault Server log file

Configuration to access the SQL Database

Configuration to access the Snort Database

Configuration to access the OSVDB Database

58

Thursday, May 3, 12

Server Configuration /etc/ossim/server/config.xml

Path to the correlation directives

Waiting time between each execution of AlienVault Server

scheduled jobs

Listening port, name and listening IP address of the AlienVault

Server

59

Thursday, May 3, 12

Web Interface Configuration Executive panel configuration

/etc/ossim/framework/panel/

/etc/ossim/framework/ossim.conf

Paths to applications and libraries Configure access to the different databases Some tools use this file to get the configuration parameters to

access the database (ossim-db, create_sidmap.pl...)

60

Thursday, May 3, 12

Monit Monit

Process monitors all the important services in the AlienVault machines and restarts services in case of a process crash

Configuration file /etc/monit/monitrc Different configuration based on the profile in use When Stopping any process monit must be stopped first

# Framework check process ossim-framework with pidfile /var/run/ossim-framework.pid group framework start program = "/etc/init.d/ossim-framework start" stop program = "/etc/init.d/ossim-framework stop" if 5 restarts within 5 cycles then timeout

61

When debugging, be sure to turn monit OFF!More Monit infos:http://mmonit.com/monit/

Thursday, May 3, 12

AlienVault: System logging All the AlienVault components offer logging

This can and should be used for

Debugging- errors may not be seen directly in the Web interface but can be spotted in the system

logs

- extra information for the AlienVault Support Team

Diagnosis- are all the services up to date

- did system update generate errors

Verification of Success or Failures- is my data source connector generating events

- are idm-events collected

62

Thursday, May 3, 12

AlienVault: Log files OSSIM Server

/var/log/ossim/server.log

OSSIM Agent

/var/log/ossim/agent.log

OSSIM Frameworkd

/var/log/ossim/frameworkd.log

Snort

/var/log/syslog

/var/log/snort (Binary Format)

Other applications

Check the variable location in the plugin configuration file

63

Thursday, May 3, 12

Debug Mode OSSIM Server

# ossim-server D 6 -d Logfiles in /var/log/ossim/server.log

This does not show information on the terminal, much more info will be logged in the server log file

OSSIM Agent

# ossim-agent vv OSSIM Frameworkd

# ossim-framework vv Never leave an application running in Debug mode in a production

64

Thursday, May 3, 12

Networks Card Information ethtool/mii-tool

Network card stats and link status set link speed and type for not autonegoiating switch ports

iptraf measure throughput and TCP sessions on a network interface

tcpdump Check whether the port mirroring is well configured or not do configured devices really send syslog to the Sensor

65

Thursday, May 3, 12

Network Configuration Rename network interfaces

# apt-get install ifrename

Edit the file /etc/iftab

Insert a line for each network interface with the following format :

eth0 mac 00:17:31:56:BC:2D eth1 mac 00:16:3E:2F:0E:9C

Network cards with more than one interface usually have consecutives MAC addresses

# ifconfig -a | grep HWaddr

66

Thursday, May 3, 12

Network Configuration Additionally rename it with udev

# ifconfig -a |grep HWaddr

Create the file /etc/udev/rules.d/010_netinterfaces.rules

Reboot & check udev entries

# udevinfo -a -p /sys/class/net/eth0

67

Thursday, May 3, 12

Network Configuration The network configuration in Debian is stored in the following file:

/etc/network/interfaces

After modifying the previous file, it is required restarting networking with the following command:

# /etc/init.d/networking restart

68

Thursday, May 3, 12

Network Configuration Each interface is configured in /etc/network/interfaces with the

following template

be sure that the interface is also in the auto section to enable automatic startup on system boot

auto lo0 eth0 ... ethallow-hotplug eth0iface eth0 inet static address 192.168.1.133 netmask 255.255.0.0 network 192.168.0.0 broadcast 192.168.255.255 gateway 192.168.1.1 dns-nameservers 192.168.1.100

69

Thursday, May 3, 12

Network Configuration address: IP address given to the interface (In the example eth0).

netmask: Network Mask

network: It is the part of the IP address which is common between all the IP addresses in the network.

broadcast: Broadcast IP of the network.

gateway: IP address of gateway in our network

dns-nameservers: IP addresses of the DNS servers used in our corporation. More than one DNS server can be used (Separated by comma). In there is a local DNS running in your network, t should be placed in first place.

70

Thursday, May 3, 12

Network Configuration Those interfaces in promiscuous mode used to collect the network

traffic (Port mirroring) should have an entry in the network configuration file with the following format:

up ifconfig eth0 0.0.0.0 promisc -arp

71

Thursday, May 3, 12

Network Recommendations The collector does a lot of queries to the DNS server to normalize

events, so, the local DNS should always be configured in any AlienVault box.

In case there is not a local DNS in your network, the different hostnames and their associated IP addresses should be defined in the the /etc/hosts file in your collectors.

Interfaces in promiscuous mode should only be used to collect network traffic, those interfaces should never have an assigned IP address

72

Thursday, May 3, 12

Disk Space The disk space left in the AlienVault machines should be monitored

The folder /var/ will use the biggest amount of disk space in AlienVault

Databases Log files

The /var/ folder should be separated into another partition with the highest amount of disk available (>80%).

73

Thursday, May 3, 12

Swap Memory Swap memory usage slows down the system so we need to make

sure the system is not using frequently the swap partition

If the system is always using the Swap partition it is required to increase the amount of RAM memory installed in the system.

74

Thursday, May 3, 12

Munin Munin can monitor the status of various parameters within the

operative system such as Interrupts, Load, Memory, network, and processes

Munin is really useful to monitor that our hardware is working properly

Munin can be used distributed since 3.1

75

Thursday, May 3, 12

AlienVault: system recovery Backup & Recovery

simple backup script is provided easy recovery after new installation

Backup script: see next page

76

Thursday, May 3, 12

AlienVault: system recovery Backup script: (Caution!!! - /var/ossim/logs is not included)

#!/bin/bashcd /etc/init.d/

process="monit ossim-agent ossim-server ossim-framework apache2 arpwatch exim4 fprobe nagios3 nessus dnfdump nfsen ntop munin-node openvas-scanner openvassd openvas-server osirisd osirismd pads rsyslog postfix samba snort* snmpd tomcat nfdump"

for i in $processdo /etc/init.d/$i stop > /dev/null;done

chmod 000 /etc/cron.hourly/*chmod 000 /etc/cron.daily/*

d1r="/var/ossim/backup/db/`date +%F-%H_%M_%S`"dbs=`echo "show databases" | ossim-db | grep -v "Database" | grep -v "information_schema"`p4ss=`grep -i pass= /etc/ossim/ossim_setup.conf |awk -F'=' '{print$2}'`h0st=`grep -i db_ip /etc/ossim/ossim_setup.conf |awk -F'=' '{print$2}'`test -z $h0st && h0st="localhost"

for db in $dbs; do test -d $d1r/$db/struct || mkdir -p $d1r/$db/struct mysqldump -d -u root -h $h0st -p$p4ss $db > $d1r/$db/struct/$db-struct-`date +%F-%H_%M_%S`.sql mysqldump -u root -h $h0st -p$p4ss $db > $d1r/$db/$db-`date +%F-%H_%M_%S`.sqldone

tar --preserve -czvf $d1r/../../complete_backup_`date +%F-%H_%M`.tgz $d1r > /dev/nullecho "Generated /var/ossim/backup/complete_backup_`date +%F-%H_%M`.tgz file"

tar --preserve -czvf $d1r/../../config_files_`date +%F-%H_%M`.tgz /etc/* > /dev/nullecho "Generated /var/ossim/backup/config_files_`date +%F-%H_%M`.tgz file"

dpkg -l >> $d1r/../../packages_list_`date +%F-%H_%M`

chmod 755 /etc/cron.hourly/*chmod 755 /etc/cron.daily/*

alienvault-reconfig -c -v

exit 0

77

Thursday, May 3, 12

Hands-On: CLI alienvault-setup

change default admin email address configure monitored networks enable detector plugins: foo bar and baz change the time zone, but if ethx to nonpromiscous mode change the system name

delete and rebuild the complete alienvault-database

WARNING: all your data and configuration will be LOST

78

Thursday, May 3, 12

AlienVault Event Collection

Get the data from the network

79

Thursday, May 3, 12

Syslog Using Syslog is usually the easiest way to forward events to the

AlienVault Sensor

All Linux, BSD and MacOSX include different Syslog implementations by default.

It's simple to configure event forwarding policies using Syslog

A large number of devices and applications allow event logging using the Syslog protocol. If it doesnt the log files can be logged into Syslog using logger:

# tail f /path/to/file | logger t application

80

Thursday, May 3, 12

Snare Agent Snare forwards Windows EventLog events to a remote Syslog

server.

When installing Snare, it can also configure a logging policy in the Windows System. Take care if you have already defined a logging policy.

Once Snare has been installed (Configuration -> Collection -> Downloads) download the .reg file to configure it to forward events to the remote Syslog server

81

Thursday, May 3, 12

WMI WMI (Windows Management Instrumentation) provides an

operating system interface through which instrumented components provide information and notification.

WMI allows scripting languages like VBScript or Windows PowerShell to manage Microsoft Windows personal computers and servers, both locally and remotely.

WMI is preinstalled in Windows 2000 and newer OSs. It is available as a download for Windows 95 and Windows 98.

AlienVault includes two data source connectors that allow collecting information using WMI:

Detector: wmi-system-logger.cfg Monitor: wmi-monitor.cfg

82

Thursday, May 3, 12

Fw1-Loggraber Fw1-Loggrabber allows collecting events from Checkpoint FW-1

devices using the Checkpoints LEA (Log Export Api) protocol

Fw1-Loggraber has to be installed along with the detector. It will stablish a connection to get the event from the Checkpoint FW-1 device

83

Thursday, May 3, 12

Cisco SDEE The AlienVault Sensor can collect events from Cisco devices using

the SDEE protocol. The detector allows collecting event from the following devices:

Cisco Network Detection Systems (IPS) Cisco Switch IDS Cisco IOS routers with Inline Intrusion Prevention

System (IPS) functions Cisco IDS modules for routers Cisco PIX Firewalls Cisco Catalyst 6500 Series firewall services modules (FWSMs) Cisco Management Center for Cisco security agents CiscoWorks

Monitoring Center for Security servers

Data Source Connector configuration file: cisco-ips.cfg

84

Thursday, May 3, 12

Rsyslog Rsyslog is the Syslog implementation shipped with AlienVault

Rsyslog is extremely configurable and allows configuring filtering and forwarding in a really easy way

Rsyslog must allow remote connections to collect logs coming from other Syslog servers. This feature has to be enabled in the Rsyslog configuration file (/etc/rsyslog.conf) including the following lines:

$ModLoad imudp

$UDPServerRun 514

$ModLoad imtcp

$InputTCPServerRun 514

85

Thursday, May 3, 12

rsyslog filters Filter using Rsyslog ( /etc/rsyslog.d/)

Forward certain events to a local file

if $msg contains 'error' then /var/log/error if $syslogfacility-text == 'local0' and $msg startswith 'DEVNAME'

and ($msg contains 'error1' or $msg contains 'error0') then /var/log/somelog

Stop processing some events

if $msg contains 'error' then ~ Regex in Rsyslog

http://www.rsyslog.com/user-regex.php

86

Thursday, May 3, 12

rsyslog customization Separate incoming logs (syslog)

find a phrase in syslog to classify the logs (hostname, ip-address,...)

Send logs to a different logfile Create a file with file extension conf in /etc/rsyslog.d

e.g. /etc/rsyslog.d/customize.conf

possible commands# sends logs with to cisco.log:source, isequal, "HOSTNAME" /var/log/cisco.log& ~if $msg contains STRING then /var/log/xyz.logif $msg contains STRING ~

# ip-address examplesif $fromhost == 'IP_ADDRESS' then -/var/log/ossim/device.log&~if $fromhost-ip isequal 'IP_ADDRESS' then -/var/log/cisco-fw.log& ~:fromhost-ip, isequal, "IP_ADDRESS" -/var/log/cisco-fw.log& ~

The rsyslog webpage provides more useful examples, tricks and howtos:http://www.rsyslog.com/87

Thursday, May 3, 12

Log rotation When creating new log forwarding rules (Rsyslog), it is important to

ensure that the logs will not grow indefinitely

To do this we must create new entries in the logrotate configuration

/etc/logrotate.d/

/var/log/ossim/agent.log /var/log/ossim/agent-plain.log /var/log/ossim/agent_error.log /var/log/ossim/agent_stats.log { daily firstaction test -f /var/log/snort/alert || touch /var/log/snort/alert > /dev/null 2>&1 endscript prerotate /etc/init.d/ossim-agent stop > /dev/null 2>&1 endscript postrotate /etc/init.d/ossim-agent start > /dev/null 2>&1 endscript}

88

Thursday, May 3, 12

Log rotation example If you have new separated logfiles just take this example and add

your new logfiles to it!

/etc/logrotate.d/alienvault

/var/log/xyz.log/var/log/foo.log...{ rotate 7 # Save the last 7 logs daily # rotate daily missingok # if file doesnt exist continue notifempty # if log is empty, the log dont rotate delaycompress # postpone compression of previous log-file to next cycle compress # Compress the log postrotate

invoke-rc.d rsyslog reload > /dev/null

}

89see also > man logrotate

Thursday, May 3, 12

Hands-On: CLI Define a syslog source and filter to /var/log/foo...

enable logrotation on the source you just configured

alienvault-setup

change default admin email address configure monitored networks enable detector plugins: foo bar and baz change the time zone change the system name

delete and rebuild the complete alienvault-database

WARNING: all your data and configuration will be LOST

90

Thursday, May 3, 12

Hands-On: CLI Logfiles

send some example logs to /var/log/syslog- use the following script:

filter the log and send it to a separated log file and be sure that this log is not filling up your disk space (rotate the file daily with compression enabled)

while truedo

cat /var/log/firewall.log | logger -t sleep 10done

91

Thursday, May 3, 12

AlienVault Data SourcesAdapt collection to your organization

92

Thursday, May 3, 12

Types of DS Connectors Two types of Data Source Connectors

Detectors: They offer events (Snort, Firewalls, Antivirus, Web servers, OS events..)

Monitors: They offer indicators (Ntop, Tcptrack, Nmap, Webs, Compromise & Attack...)

93

Thursday, May 3, 12

Files Each DS Connector (monitors and detectors) is built on two files:

Plugin.cfgContains the configuration parameters of the plugins and the rules that an event has to match in order to be collected and normalized.

Plugin.sqlContains the description of every possible event that can be collected using the plugin (Plugin_id, Plugin_sid, Name given to the event, priority and reliability)

94

Thursday, May 3, 12

[DEFAULT]plugin_id=4003

# default values for dst_ip and dst_port# they can be overwritten in each ruledst_ip=\_CFG(plugin-defaults,sensor)dst_port=22

[config]type=detectorenable=yes

source=loglocation=/var/log/auth.log

create_file=false

process=sshdstart=nostop=nostartup=/etc/init.d/ssh startshutdown=/etc/init.d/ssh stop

[ssh - Failed password]# Feb 8 10:09:06 golgotha sshd[24472]: Failed password for dgil from 192.168.6.69 port 33992 ssh2event_type=eventregexp="(\SYSLOG_DATE)\s+(?P[^\s]*).*?ssh.*?Failed password for (?P\S+)\s+from\s+.*?(?P\IPV4).*?port\s+(?P\PORT)"plugin_sid=1sensor={resolv($sensor)}date={normalize_date($1)}src_ip={$src}dst_ip={resolv($sensor)}src_port={$sport}username={$user}

Numerical identifier of the plugin

Type of plugin: Detector

Source of the events (log, mssql,mysql or wmi)

Associated process and start/stop options

Type of event

Regular expressions

Fields that will be sent to the AlienVault Server

Default fields for every event

95

Ds Connector: Detector

Thursday, May 3, 12

Ds Connector: Detector plugin_id

Data Source ID. User reserved range: 9000-10000 E.g.: plugin_id=3000

source log: Text file (E.g: SSH, Sudo, Apache...) mssql: Mssql Database (E.g: panda-se) mysql: Mysql Database (E.g: moodle) wmi: Windows Management Instrumentation (wmi-system-logger)

96

Thursday, May 3, 12

Ds Connector: Detector location

- Files in which the applications store the events

- E.g.: location=/var/log/file.log

create_file- Create the file in case it does not exist

- false/true

process / start / stop / startup / shutdown- Only if the process is running in the same machine that the detector

- If the process is not running in the machine, is there a process helping us to collect those logs? syslog? fw1-loggrabber?

97

Thursday, May 3, 12

Ds Connector: Detector Rules

Rules define the format of each event and how they are normalized It is composed by a regular expression and the list of fields that the

event will include when once it is sent to the AlienVault SIEM or Logger

In some cases only one regular expression will collect every event coming from one application, in some other cases more than one rule will be required

98

Thursday, May 3, 12

DS Connector: Detector Rules

Rules are loading in alphabetical order based on the name given to each rule

Once the log matches one the regex of one rule the ossim agent stops processing the event

Generic rules must be the last loaded in memory as they will probably match all the events

The name of the rule is mandatory

99

Thursday, May 3, 12

DS Connector: Detector The rule must include the event type:

event_type=event

The following fields can be used to normalize the event:

Values in bold are mandatory

Fields in red include values that always have to be defined in the plugin

Fields in green can will be filled by the AlienVault Agent in case they can not be found in the original log (Dont include that line when creating the plugin)

Fields in grey are optional

plugin_id plugin_sid date sensor interface protocol

src_ip src_port dst_ip dst_port username password

filename userdata1 userdata2 userdata3 userdata4 userdata5

userdata6 userdata7 userdata8 userdata9

100

Thursday, May 3, 12

DS Connector: Detector Regexp

The regexp field contains the regular expression that defines the format of the events, and extracts the information to normalize the event.

The regular expressions are written using the Python regular expression syntax:

http://docs.python.org/library/re.html

regexp="(\SYSLOG_DATE)\s+(?P[^\s]*).*?ssh.*?Failed password for (?P\S+)\s+.*?(?P\IPV4).*?port\s+(?P\PORT)"

regexp=(\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d)\S+ (\S+) (\S+) (\S+) (\d+) (\w+) (\S+) \S+ (\d+)

101

Thursday, May 3, 12

Operator Meaningc A non special character matches with itself

\c Removes the special meaning of the character c; The RE \$ matches with $^ Indicates located at the beginning of the line$ Indicates located at the end of the line. Any individual character

[] One or any of the characters ; accepts intervals of the type a-z, 0-9, A-Z[^] A char different from ; Accepts intervals of the type a-z, 0-9, A-Z

102

Regular expressions

Useful tools for testing regular expressions:Online: http://www.regexpal.comWindows: RegEx TesterLinux: http://kodos.sf.net/Regexhibit (OSX): http://homepage.mac.com/roger_jolly/software/

Thursday, May 3, 12

Regular expression Matches with

a.b axb aab abb aSb a#b ...

a..b axxb aaab abbb a4$b ...

[abc] a b c (one character srtings)

[aA] a A (one character srtings)

[aA][bB] ab Ab aB AB (two character srtings)

[0123456789] 0 1 2 3 4 5 6 7 8 9

[0-9] 0 1 2 3 4 5 6 7 8 9

[A-Za-z] A B C ... Z a b c ... Z

[0-9][0-9][0-9] 000 001 .. 009 010 .. 019 100 .. 999

[0-9]* empty_chain 0 1 9 00 99 123 456 999 9999 ...

[0-9][0-9]* 0 1 9 00 99 123 456 999 9999 99999 99999999 ...

^.*$ A full line

103

Regular expressions

Thursday, May 3, 12

Operator Meaningr* 0 or more occurrences of the RE rr+ 1 or more occurrences of the RE rr? 0 or an occurrences of the RE r, and no more

r{n} No occurrences of the RE rr{,m} 0 or at most m occurrences of the RE rr{n,m} N or more occurrences of the RE r, but at most mr1|r2 The RE r1 or the RE r2

Regular expression Matches with

[0-9]+ 0 1 9 00 99 123 456 999 9999 99999 99999999 ..[0-9]? empty_string 0 1 2 .. 9(ab)* empty_string ab ababab abababababab

([0-9]+ab)* empty_string 1234ab 9ab9ab9ab 9876543210ab 99ab99ab ...

104

Regular expressions

Thursday, May 3, 12

Regular expression Matches with Equals

\d Any decimal character [0-9]\D Any non decimal character [^0-9]\s Any space character [ \t\n\r\f\v]\S Any non space character [^ \t\n\r\f\v]

\w Any alphanumeric characterand _ [a-zA-Z0-9_]

\W Any non alphanumeric character [^a-zA-Z0-9_]\Z End of line

105

Regular expressions

Thursday, May 3, 12

Pattern Descriptionb,c,X,8 Ordinary characters just match themselves exactly. The meta-characters which do not match themselves because they

have special meanings are: . ^ $ * + ? { [ ] \ | ( )

. Matches any single character except newline (\n).\w Lowercase w matches a "word" character: a letter or digit or under-bar [a-zA-Z0-9_]. It only matches a single word char,

not a whole word.

\W Uppercase w matches any non-word character.\s Lowercase s matches a single whitespace character -- space, newline, return, tab, form [ \n\r\t\f].\S Upper case s matches any non-whitespace character.\d Lowercase d matches a single Decimal digit [0-9]\D Uppercase d matches any non decimal character\t Matches a tab character\n Matches a newline character\r Matches a return character\Z Matches only at the end of the string.\ Escapes special characters. If you are unsure if a character has special meaning, such as '@', you can put a slash in front

of it, \@, to make sure it is treated just as a character.

106

Regular expressions

Thursday, May 3, 12

Regex aliases /etc/ossim/agent/aliases.cfg

/etc/ossim/agent/aliases.local (For user custom aliases)

This file contains predefined regular expressions that can be used to simplify the process of writing new plugins

Usage Example:

\SYSLOG_DATE\s+\IPV4\s+\IPV4

107

Thursday, May 3, 12

Regular Expressions The information extracted by the regular expression from the log

can be accessed by:

Position: (\d\d):(\d\d):(\d\d)- hour={$1}

- minutes ={$2}

- seconds={$3}

Tags: (?P\d\d):(?P\d\d)(?P\d\d)- hour={$hour}

- minutes ={$minutes}

- seconds={$seconds}

108

Thursday, May 3, 12

Functions The AlienVault SIEM and Logger must receive normalized events, as an

example the addresses have to use IPV4 format and the date has to use the following format YYYY-MM-DD HH:MM:SS (2010-12-31 22:57:00)

To simplify the process of normalizing events some functions can be used

resolv()

Translate hostnames into IPV4 addresess (DNS queries)

normalize_date()

The normalize_date function translate many format dates into the format accepted by the SIEM or Logger

- YYYY-MM-DD hh:mm:ss

More functions can be found and defined in ParserUtils.py

109

A very useful tool for testing plugin is located in /usr/share/ossim/scripts/regexp.py

Thursday, May 3, 12

Translation Tables Translations can be configured to be done once the event has

been collected

E.g.: When the event id is not numeric, but plugin_sid has to be numeric

Translations have to be defined inside a category called [translation]

Translate using the function translate().

110

Even more info can be found here:http://www.alienvault.com/docs/AlienVault%20Building%20Collector%20Plugins.pdf

Thursday, May 3, 12

Testing your plugin Useful tool: regexp.py

will analyze a set of logs to a set of regular expressions or a plugin will give you statistics on matched lines and rules

111

# /usr/share/ossim/scripts/regexp.py /var/log/netscreen.log netscreen-firewall.cfg qMultiple regexp mode used, parsing netscreen-firewall.cfg -----------------------------------------------------------------------------Rule: netscreen-firewall-BA-traffic Matched 5586 timesRule: netscreen-firewall-BB-traffic Matched 0 timesRule: netscreen-firewall-EF-system Matched 0 timesRule: netscreen-firewall-ZZZ-generic Matched 50 timesCounted 5636 lines.Matched 5636 lines.

Thursday, May 3, 12

Hands-On: plugins

firewall logs are sent to /var/log/firewall.log

while truedo

cat /var/log/firewall.log | logger -t sleep 10done

112

Thursday, May 3, 12

Hands-On: plugins write a firewall plugin

copy existing similar plugin: plugin_id number start on custom range write your regex rule to match the loglines

write a firewall sql file

copy existing similar sql file: change the fields to your custom plugin rules

activate your plugin on the CLI (alienvault-setup)

write your sql file to the database

113

Thursday, May 3, 12

AlienVault Policies & Actions

Controlling flow of information between AlienVault Components

114

Thursday, May 3, 12

Event Filtering Levels Policy rules

AlienVault Sensor Configuration

Data Source Connector Configuration

Collection method used

Data Source Configuration

115

Thursday, May 3, 12

SIEM & Logger

Database storage (MySQL) The number of events that can be stored will depend on the

hardware in use As we increase the number of stored events, the analysis gets

slower Max number of stored events: 20 millions Certain events don't offer useful information for correlation

and analysis Need to filter useless events

SIEM

Disk storage No more storage limits apart from the available disk space Everything should be stored in the Logger without

exceptionLogger

116

Thursday, May 3, 12

Filter examples Filter all events generated by a tool

Filter certain events from a tool

Filter all events from one host or network

Filter events generated between two different hosts

Filter some events during the night

Filter some events every weekend

Filter all events from one service

117

Thursday, May 3, 12

Filter in Policy 1. Create a DS group with the types of events that have to be filtered

2. Select source and destination Assets (Hosts, Networks, ANY...)

3. Select the ports

4. Select the DS group

5. Select time period in which the policy applies

6. Select policy consequences

1. Process in the SIEM?2. Process in the Logger?3. Store the event?4. Correlate only?

118

Thursday, May 3, 12

Agent Configuration If we dont want to collect any event from one of the plugins

remove the plugin from the file:

/etc/ossim/ossim_setup.conf

in the variable named:

detectors

And run the following command:

# alienvault-reconfig

119

Thursday, May 3, 12

Filtering in DS Connectors Some events can be filtered during the collection process editing

the configuration file for each plugin:

Using the option exclude_sids (E.g.: Apache DS Connector)

exclude_sids=404,200,403

Modifying the regular expressions to avoid matching certain events

120

Thursday, May 3, 12

Hands-on: Policies Write a policy for a Nagios event / service down

use external command and write to /tmp/alert disable ssh on the device in question does your external command write to /tmp/alert

Send email after a prioritized event

Ticket creation (e.g. if only one user is allowed/responsible to subscribe new tickets)

121

Thursday, May 3, 12

Logical Correlation Directives

Put intelligence into your SIEM

122

Thursday, May 3, 12

Logical Correlation Directives Logical Correlation uses Correlation directives to detect attacks

and problems in the monitored networks

By default AlienVault includes a set of Correlation directives

Users can create custom correlation rules in both Unified and Open Source edition

AlienVault Professional Feed provides more than 500 extra correlation rules

123

Thursday, May 3, 12

Logical Correlation Directives Logical Correlation is implemented in the AlienVault SIEM

Correlation Directives are stored in the following path

/etc/ossim/server/ Web-based Correlation Directives Editor

Intelligence -> Correlation Directives

124

Thursday, May 3, 12

Rules Directives are composed by rules

Each rules defines the conditions that event must match to be correlated by the directive

Whenever the conditions defined by a rule are matched, a new event is generated

Some of the values of the rule will be used to generate the new event (In the event of a successful correlation of that rule)

It is possible to have multiple rules in each correlation level (Except in the first correlation level)

125

Thursday, May 3, 12

Example: Brute Force

Root Login RefusedInvalid Password

Invalid UserUser not allowed

User in DenyUsersIllegal User

Brute-Force Attack against SSH Server

SIEM SENSOR

Normalized Events

ALARM: Brute-Force Attack Against 192.168.1.1Total events captured 12.392

First Event 10-02-2011 23:30:23Last event 10-02-2011 23:56:19

Login AttemptLogin AttemptLogin AttemptLogin Attempt

126

Thursday, May 3, 12

Example: Brute-Force Think about all possible events that may be generated

Root Login not allowed (SSH Server Configuration)

Event generated: Root Login Refused

Root Login allowed but incorrect password

Event generated: Invalid Password

Root account locked

Event generated: User not allowed because account is locked

User: root Password: root

127

Thursday, May 3, 12

Example: Brute-Force Global properties of the correlation directive

Name of the directive- This is the name that will take all events generated within the correlation

of this directive

ID of the directive - All events generated within this directive will use 1505 as plugin_id

(Data Source ID) and the ID of the directive as plugin_sid (Event type) 500000-100000 is the reserver range for user-created directives

Priority of the directive (Impact of this Attack or problem in your network)

- All events generated within this directive will have as their priority the global priority value of the correlation directive

128

Thursday, May 3, 12

Example: Brute-Force Global properties of the directive:

plugin_id=1505 plugin_sid=500000 name=SSH Brute Force Attack Against 192.168.2.2 priority=4

Same plugin_id in all events during Logical Correlation: Plugin_id 1505 The Plugin_sid of the new

event is the ID of the directive that generated the event

The name (Signature) of the new event is given by the name of the directive

The priority of the new event is given by the global priority of

the directive

SIEM

SQL Storage

Correlation

Risk Assessment

Policy

Collection

EVEN

TS

Even

t Gen

erat

ed d

uring

Log

ical C

orre

lation

129

Thursday, May 3, 12

Example: Brute Force Events will first try to get in those directive whose correlation has

started and then they will try to match with the first correlation level of the correlation rules that are enabled while the SIEM runs

The same event can be correlated in multiple correlation directives

SIEMLogical Correlation

Directives in the correlation engine Directives waiting to be correlated

130

Thursday, May 3, 12

Example: Brute-Force First correlation level

Special Conditions

Only detector rules can be used in the first correlation level Only one rule This rule doesnt have time out Only one occurrence (Correlation of the directive will start with the

first event matching the conditions of the rule

131

Thursday, May 3, 12

Example: Brute-Force First correlation level

How many events?- Always a single event in the first correlation level

What type of events should match the rule?- Plugin_id and Plugin_sid of the events

What sources and destinations should match the rule?- Hosts, Host Groups or Networks

- Source and Destination Ports

Any other special condition?- E.g.: Avoid events with a certain username from matching the rule

132

Thursday, May 3, 12

Example: Brute-Force First correlation level

What type of events should match the rule?- Check at Configuration -> Collection -> Data Sources

- Only one Data Source in each rule (One plugin_id with one or multiple plugin_sid)

What sources and destinations should match the rule?- The first rule is usually generic (We do not know yet where the attack is

coming from)

- ANY / HOME_NET (Assets in the AlienVault inventory)

133

Thursday, May 3, 12

Example: Brute-Force First correlation level

type: detector (First rule is always detector) and we will collect the events using a detector DS Connector

reliability: When calculating the risk, it will also be 0 - risk= (Asset Value * Priority * Reliability)/25

from/to: Both are set to any because we don't know yet who will be the attacker and his target

port_from/port_to: Set to ANY, not important in this type of directive

plugin_id/plugin_id: List of the events in the SSHD DS Connector that refer to fail authentications

134

Thursday, May 3, 12

Example: Brute-Force This directive would be useless with only one level

This directive with a single level will never generate an event becoming alarm (Reliability is set to 0)

Directives with one level can be used to rename events

List of the IP Addresses of the Web Servers

WIth a reliability of 0 this event will never become alarm, but it will be stored in the

SIEM SQL (SIEM -> Analysis)

135

Thursday, May 3, 12

Example: Brute-Force Second correlation level

- Authentication successful

Same Source and Same destination that matched the first correlation level

Time out before this possibility is discarded?

Would this indicate a big problem? (Authentication successful after 1 failed)

- 10 more authentication failed

Same Source and Same destination that matched the first correlation level

Time out before this possibility is discarded?

Would this indicate a big problem? (A total of 11 Authentication failed in X seconds)

136

Thursday, May 3, 12

Logical Correlation

137

Thursday, May 3, 12

Logical rule example

138

Thursday, May 3, 12

Logical rule example In the directive with id 27 every time the condition given by a rule is

met an event with priority 2 is generated. The reliability will be taken from the rule that has matched:

Maximum risk will be in the fifth rule Priority 2, Reliability 10 With this directive the event will get as much a risk of 4 (When one

of the assets involved has a value of 5)

(2*10*5) / 25 =4

139

Thursday, May 3, 12

Example: WormSIEM

140

Thursday, May 3, 12

Example: WormAlarm

SIEM

140

Thursday, May 3, 12

Example: WormAlarm

Alarm

SIEM

140

Thursday, May 3, 12

Example: WormAlarm

Alarm

Alarm

SIEM

140

Thursday, May 3, 12

Example: Intrusion

Hi!

At your command!

Persistence

Behaviour

1.1.

2.

3.

4.

F

141

Thursday, May 3, 12

Writing Correlation Rules All directives have a unique numeric identifier, a name and a global

priority that will be used in every event generated during the correlation of the directive.

The events generated when correlation the directive will always have 1505 as plugin_id and the id of the directive as plugin_sid.

Writing Correlation Rules The initial rule does not have a time_out value, and it can be

matched by any event while the SIEM is running

The conditions defined by the first rule are usually so generic because we dont know the values of the event that it is going to start the correlation:

In the following rules in many cases we will have to make sure that some fields have the same value than the event that started the correlation of the directive

Writing Correlation Rules To open a second correlation level the following tags have to be

used: and

More than one rule can be included in each correlation level (Not in the first one)

When one of the multiple rules of each correlation level is matched, the rest of the rules of that level are discarded

144

Thursday, May 3, 12

Writing Correlation Rules Two types of rules can be used when writing directives:

Detector rules: Detector rules for incoming events from the different detector plugins (SSH, Snort, Firewalls...)

Monitor rules: Monitor rules request for information to different applications and devices through the collector (Session information, permissions, availability...)

145

Thursday, May 3, 12

Writing Correlation Rules Common fields (Monitor and detector rules):

plugin_id: Numerical identifier of the tool that provides the information (Events in detector rules and indicators in monitor rules)

plugin_sid: Numerical identifier of the type of even within the tool defined by plugin_id or request or query that has to be executed (In Monitor rules)

reliability: Reliability value of every event generated within the directive. It can be an absolute value 0-10 or incremental +2, +6

time_out: Waiting time before the rule expires and the directive process defined in that rule is discarded. The first rule doesnt have a time_out value.

146

Thursday, May 3, 12

Writing Correlation Rules Detector rule: sensor: ANY

Any IP address Address IP (x.x.x.x)

An IP address (E.g: 192.168.1.9) Several IP addresses

IP address list separated by commas (E.g: 192.168.1.2,192.168.2.3) Sensor name

Sensor name (As configured in Configuration -> SIEM Components) Relative values

It is possible to relate variable values to previous correlation levels. 1:SENSOR refers to the sensor IP address in the first correlation level.

Denied values It is possible to include denied elements as value of the target field: "!192.168.2.203,INTERNAL_NETWORK

147

Thursday, May 3, 12

Detector rule: from/to: ANY

Any IP address Address IP (x.x.x.x)

An IP address (E.g: 192.168.1.9) Several IP addresses

IP address list separated by commas (E.g: 192.168.1.2,192.168.2.3) Network name

Network name (Defined in the paragraph Policy in the Web interface) Relative values

It is possible to relate variable values to previous correlation levels:- 1:SRC_IP refers to the origin IP address of the event in the first correlation level.

- 2:DST_IP refers to the target IP address in the second correlation level.

Denied values It is possible to include denied elements as a value of the origin field:

All networks in the inventory HOME_NET

Writing Correlation Rules

148

Thursday, May 3, 12

Writing Correlation Rules Detector rule: port_to/port_from:

This field can adopt an only port or a list of ports separated by commas as value. The keyword ANY is a list of all the ports.

It is possible to use relative values of ports referring to other correlation values:

1:DST_PORT refers to the values of the target port of the first correlation value.

3:DST_PORT refers to the value of the target port of the third correlation value.

To deny ports we put the symbol ! before the port we want to deny:

port="!22,25,110,!21"

149

Thursday, May 3, 12

Writing Correlation Rules Detector rule: protocol:

The field Protocol refers to the protocol in which the communication was established where the event took place.

Protocol can adopt the following values:

TCP

UDP

ICMP

Host_ARP_Event

Host_OS_Event

Host_Service_Event

Host_IDS_Event

Information_Event

It is also possible to specify the protocol with the protocol number.

150

Thursday, May 3, 12

Writing Correlation Rules Sticky

When the events arrive to the correlation engine they will try to be correlated inside directives whose correlation has been started

Using sticky we avoid those events to start the correlation of the same directive again, as they may also meet the conditions given by the same directive.

Sticky Different

This variable can be associated to any field in rules with more than one occurrence, to make all the occurrences have a different value in one of the fields

E.g.: sticky_different=DST_PORT All the events matching the rule must have a different destination

port (Port scanning detection)

151

Thursday, May 3, 12

Writing Correlation Rules Monitor rules define a condition that has to be checked using monitor

plugins

The following variables to, from, port_to, port_from, protocol, sensor, username, filename, password and userdata1-9 will have the values that will be sent to the collector to be used in the monitor plugin. Those values don not have to be met, they will only be used to build the monitor request.

Monitor rules specific fields:

value condition interval absolute

152

Thursday, May 3, 12

Writing Correlation Rules Monitor rule: condition

Establishes a logical relation between the value field and the value returned in the monitor plugin request.

eq equalne non equallt less thangt greater thanle less or equal thange greater or equal than

153

Thursday, May 3, 12

Writing Correlation Rules Monitor rule: value

This field sets the value that has to be compared with the value returned by the collector after doing the monitor request.

Value must be an integer.

Monitor rule: interval

This value of this field sets the waiting time between each monitor request before the rule is discarded because the time defined by time_out is over.

Monitor rule: absolute

This value sets if the value that has to be compared is relative or absolute.

Absolute true: If the host has more than 1000 bytes sent during the next 60 seconds. There will be an answer if in 60 seconds this value is reached.

Absolute false: If the host shows an increase of more than 1000 bytes sent. There will be an answer if the host shows this increase in 60 seconds.

154

Thursday, May 3, 12

Recommendations The objective should not always be generating alarms

Not all directives should generate alarms with a high risk value (Rate the importance of the attack or problem detected by the directive)

In some cases the last rule level should be used to keep collecting events to avoid having the same directive in the correlation engine so often

A lot of directives can be created using the same directive template (Brute force, Worms, Policy Violation...)

Use sticky to avoid directives from being generated exponentally

155

Thursday, May 3, 12

AlienVault LoggerForensic Storage

156

Thursday, May 3, 12

Hands-On: Logger create the indexed Logger

install logger package, called:- alienvault-logger

check if the indexing process is running after this you should see the indexed search possibility in the UI

157

Future versions of AlienVault will only have one Query button combining Raw and Indexed queries

Thursday, May 3, 12

Hands-On: Logger create a remote logger

ssh-keygen ssh-copy-id -i [pub_identity_file] user@remotelogger_IP example:

try to connect without password create the remote logger under AlienVault Components test the remote logger

158

Thursday, May 3, 12

AlienVault IDMIdentity Monitoring & User awareness

159

Thursday, May 3, 12

AlienVault IDM IDM - Identity Monitoring

Introduced in AlienVault 3.1

every source or destination IP can be enriched with user data

Available user information

username domain hostname IP address MAC address

160

Thursday, May 3, 12

IDM: information flow

ip=192.168.2.1 username=alien

hostname=alienshomedomain=alienvault.com

ip=192.168.24.2 username=Administrator

hostname=dc01domain=alienvault.com

SIEMSensors

Sensors

Sensors

Sensors

Send IDM events

Send IDM events

Send IDM events

Send IDM events

161

Thursday, May 3, 12

IDM: how does it work User information on the network

logins ActiveDirectory LDAP servers VPN access

Information can be retrieved by plugins

event=idm-event Rule will set username, hostname, domain etc.

162

Thursday, May 3, 12

IDM: how does it work This information will then be sent to SIEM and associated with IPs

every Event can have IDM information attached

163

Thursday, May 3, 12

Installing IDM Install alienvault-idm on all the sensors and servers for IDM

apt-get install alienvault-idm On the sensor, tell the agent to send IDM events to the SIEM

sensor: /etc/ossim/agent/config.cfg- section [output-idm] for sending to a single server

- section [idm-server-list] for sending to several SIEM/IDM servers

-[output-idm]enable=Trueip=The_IP_of_the_SIEM_server port=40002

[idm-server-list]FQDN1=The_IP_of_the_Second_SIEM_server;40002FQDN2=The_IP_of_the_Third_SIEM_server;40002

164

Thursday, May 3, 12

Installing IDM on SIEM server

add IDM in /etc/ossim/server/config.xml-

- configure IDM for the GUI

echo update config set value=1 where conf=idm_enable | ossim-db

165

Thursday, May 3, 12

Installing IDM Check if the agent already sends IDM events

works by default in some plugins- OSSEC

start integrating your own IDM data

166

Thursday, May 3, 12

Writing IDM plugins Start with a normal plugin skeleton

Insert rules for log messages generated with IDM relevant data

event-type = idm-event

Event attribute associated IDM value

username IDM username

domain IDM domain-name

hostname IDM hostname

mac IDM mac address

ip IDM associated IP

167

Thursday, May 3, 12

IDM: example plugin Example: Ossec-IDM data source

event-type = idm-event

extracted fields from regex: username, hostname, domain, ip

168

Thursday, May 3, 12

Hands-On: IDM install the required packages on the infrastructure to support IDM

do all the necessary configuration to enable the flow of IDM information across the infrastructure

from these two loglines, develop a IDM data source that reports username, hostname, domainname and IP to the SIEM

User alienvault@example.com logged in via 192.168.2.1. Assigned IP address: 10.10.1.15

Login from trusted source 10.10.2.123, user alice, domain aliceslair.com, hostname aliceslaptop

- do you receive IDM events

- do you see IDM events in the console

169

Thursday, May 3, 12

AlienVault HIDSExtending client security

170

Thursday, May 3, 12

Hands-On: OSSEC Configuration on the server side (AlienVault machine), two

possibilities:

1) creating the agent in the UI

171

Thursday, May 3, 12

Hands-On: OSSEC 2) creating the agent on the terminal

important scripts in: /var/ossec/bin: manage_agents (add agent / extract key / list agents / remove agents)

list_agents (all / connected / not connected)

agent_control (more options than list_agents)

172

Thursday, May 3, 12

Hands-On: OSSEC Install ossec-agent on a windows machine

download ossec-agent from the AlienVault UI install it configure it:

- OSSEC Server IP:

- Authentication key:

173

Thursday, May 3, 12

Hands-On: OSSEC After finishing the OSSEC connection between server and agent:

enable OSSEC plugin (ossec.cfg) in alienvault-setup check if you get the events in the UI or agent.log check if the OSSEC events arrive in the SIEM

If not check the troubleshooting page

next page

174

Thursday, May 3, 12

Hands-On: OSSEC Troubleshoot

To check if events are arriving on the server- /var/ossec/logs/alerts/alerts.log

If not, check if the agent is properly connected- view ossec-agent log file on the windows machine

- check ossec-agent configuration file on the windows machine

- check if the OSSEC udp port 1514 is open

175

Thursday, May 3, 12

AlienVault Secure Connection

Aliens love encrypted transports

176

Thursday, May 3, 12

Hands-on: OpenVPN Connecting a sensor to a SIEM with OpenVPN

SIEM (OpenVPN server) side #alienvault-reconfig --add_vpnnode=

cd /etc/openvpn/nodes

restart/start OpenVPN process: /etc/init.d/openvpn restart

alienvault-vm:/etc/openvpn/nodes# ifconfig tun0+ ifconfig tun0tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 inet addr:10.67.68.1 P-t-P:10.67.68.2 Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:65 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:0 (0.0 B) TX bytes:4340 (4.2 KiB)

177

Thursday, May 3, 12

Hands-on: OpenVPN Connecting a sensor to a SIEM with OpenVPN

send .tar.gz that includes all data we need on the sensor via scp to user@:/etc/openvpn

uncompress on the sensor side: tar -xvzf .tar.gz

restart OpenVPN: /etc/init.d/openvpn restart

alienvault-sensor:/etc/openvpn/# tar -xvzf .tar.gz + tar -xvzf .tar.gz//keys//keys/.csr/keys/.key/keys/ca.crt/keys/.crt.confopenssl.cnf

178

Thursday, May 3, 12

Hands-on: OpenVPN restart OpenVPN: /etc/init.d/openvpn restart tunnel should be established troubleshoot:

- tail -f /var/log/syslog |grep ovpn --color=auto

Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1871]: TUN/TAP device tun0 openedFeb 2 23:46:24 alienvault ovpn-AVinfraestructure[1871]: TUN/TAP TX queue length set to 100...Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: TCPv4_SERVER link local (bound): [undef]:33800Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: TCPv4_SERVER link remote: [undef]Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: MULTI: multi_init called, r=256 v=256Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: IFCONFIG POOL: base=10.67.68.2 size=62Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: MULTI: TCP INIT maxclients=1024 maxevents=1028Feb 2 23:46:24 alienvault ovpn-AVinfraestructure[1879]: Initialization Sequence Completed

179

Thursday, May 3, 12

Hands-on: OpenVPN Adapt sensor configuration to tunnel IP addresses

edit ossim-setup.conf on the sensor change all to mostly 10.67.68.1 run alienvault-reconfig

Check events arriving over the tunnel

Security Events (SIEM) / Logger or tcpdump -i tunX

180

Thursday, May 3, 12

SnortNetwork IDS

181

Thursday, May 3, 12

What is Snort? Snort is an NIDS (Network Intrusion Detection System)

It is Open Source (GPL v2) Snort combines signature, protocol and anomaly-based inspection, It has an active development and new rules are added daily Snort is the most widely deployed IDS technology worldwide

182

Thursday, May 3, 12

Brief History Snort was released as an Open Source product in 1998. Since

then its source code has been distributed using the GPL license

Snort is developed by Sourcefire, company founded by the original creator of Snort (Martin Roesch).

New Snort versions are released often, including new functionalities and fixing bugs and errors.

183

Thursday, May 3, 12

IDS Intrusion Detection System, security tool in charge of monitorizing

events on a system in search of traces of an intrusion.

An intrusion attempt is an attempt to compromise the confidentiality, integrity or availability of a computer system or to circumvent its security measures.

184

Thursday, May 3, 12

NIDS Network level IDS

Monitor network traffic

Advantages

Monitor the entire network traffic No impact on the network

Disadvantages

Unable to analyze encrypted information Requires continuous rule update in order to catch new attacks Requires specific network configuration (Port mirroring or Network

Tap)

185

Thursday, May 3, 12

NIDS: Network Traffic If the network traffic is not forwarded to the NIDS machine, only

the traffic generated that has as source or destination that machine will be analyzed.

Network traffic can be forwarded using the following methods:

Network taps Hubs [+ listen only network cables] Switches with a mirroring or span port

186

Thursday, May 3, 12

Why a NIDS? Some attacks and problems can not be detected using a firewall

Attacks tunneling the network traffic Attacks using vulnerabilities in the applications Attacks from our internal network against the internal or external

network.

187

Thursday, May 3, 12

Snort modes Snort can be used in the following modes:

Sniffer: Monitor the activity of the network Packet logger: All the network activity is stored in capture files IDS : The network traffic is compared with predefined patterns in

search of anomalies, attacks or policy violations

In AlienVault only the IDS mode of Snort is used

188

Thursday, May 3, 12

Snort and AlienVault Snort is a very important tool within AlienVault and it has been used

by AlienVault since the first OSSIM release for several reasons:

Detailled attack information from passive listening

Security problems (Trojans, Virus, Worms...) Policy Violation (Pornography, p2p, messenger) Bad configurations

Snort is the main source of information for correlation directives

Near to Zero configuration

Snort events are used in logical, inventory and cross correlation

189

Thursday, May 3, 12

Snort and AlienVault Using snort, we can collect information that could also be

collected from many applications such as Web servers and Proxies logs.

The main advantage of Snort is that all the events will be generated in a passive mode (URL visited, program versions, Web server response...)

190

Thursday, May 3, 12

Snort and AlienVault Snort is integrated as a detector plugin in AlienVault

The AlienVault Sensor collects logs from the Snort Data Source agent and send them normalized to the SIEM or Logger

Snort should only be used in those interfaces collecting the network traffic

Snort loads all signatures in memory to analyze the network traffic in real time. If all rules are enabled and a lot of traffic has to be analyzed high performance hardware will be required.

AlienVault Appliances (Sensors) are built for this task

All the events generated by the snort rules will have the plugin_id 1001. The events generated by the Snort preprocessors will have a different plugin_id for each preprocessor within the range 1002-1500.

191

Thursday, May 3, 12

Snort Architecture The Snort engine can be divided into the following components:

Decoders Preprocessors Detection engine Output plugins

192

Thursday, May 3, 12

Architecture: Decoders Decoders: The decoders collect the network packets and

prepare them to be analyzed by the preprocessors and the detection engine.

193

Thursday, May 3, 12

Architecture: Preprocessors Snort's preprocessors are divided in two categories.

A number of attacks cannot be detected by signature matching via the detection engine. so some preprocessors are used to examine packets for suspicious activity.

The other preprocessors are responsible for normalizing traffic so that the detection engine can accurately match signatures.

194

Thursday, May 3, 12

Architecture: Preprocessors frag3: Its main objective is avoiding detection evasion techniques. It

reassembles network packets to be analyzed by the detection engine.

stream5: Reassembles TCP traffic and monitors TCP and UDP sessions

http_inspect: Normalizes and searches anomalies in http traffic

rpc_decode: Normalizes and searches anomalies in rpc traffic

bo: Detects back orifice traffic in the network

ftp_telnet: Normalizes and searches anomalies in ftp and telnet traffic

smtp: Normalizes and searches anomalies in smtp traffic

sfportscan: Detects network scans in the network

195

Thursday, May 3, 12

Architecture: Preprocessors arpspoof: Decodes ARP packets and detects ARP attacks,

anomalies and inconsistence's

ssh: Detects the use of different exploits against the ssh protocol

dcerp: Detects and decode SMB and DCE/RCP traffic

dns: Detects the use of different exploits against the DNS service

ssl: Detects the SSL/TSL traffic and determines whether it has to be analyzed or not

196

Thursday, May 3, 12

Architecture: Detection Engine Detection engine: The Detection engine analyzes every network

packet to determine if there is any malicious or forbidden activity on it.

The Detection engine uses rules that are loaded when Snort starts.

Rules are compared with every network packet. If one rule is matched, an event is generated.

197

Thursday, May 3, 12

Architecture: Detection Engine alert tcp $HOME_NET any -> 10.1.1.0/24 80 (flags: SF; msg: SYN-FIN Scan;)

Header:

alert: Action that has to be done if the rule is matched tcp: protocol $HOME_NET any: Source 10.1.1.0/24 80: Destination

Options:

flags: SF; msg: SYN-FIN Scan : Conditions that have to be met and name of the event if the rule is matched.

198

Thursday, May 3, 12

Architecture: Detection Engine Rules to detect access miscelaneous web activities:

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET POLICY Megaupload file download service access"; flow:to_server,established; content:"GET "; depth: 4; uricontent:"/?d="; content:"|0d 0a|Host\: "; content:"megaupload.com"; within:25; nocase; classtype:policy-violation; reference:url,doc.emergingthreats.net/2009301; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_Download_Services; sid:2009301; rev:2;)

alert tcp $HOME_NET any -> $EXTERNAL_NET 5050 (msg:"ET POLICY Yahoo Chat Signin Inside Webmail"; flow:established,to_server; content:"content-length\:"; nocase; depth:15; content:"

Architecture: Output Plugins Output plugins: The events generated by the detection engine or

by the preprocessors can be stored in different formats. The following output plugins can be enabled in Snort:

Syslog Database Tcpdump Unified v1 Unified v2

200

Thursday, May 3, 12

Architecture: Output Plugins More than one output plugin can be enabled at the same time

Syslog Database Tcpdump Unified Unified2 csv, prelude...

In production for optimal performance

Unified (Version 2)

201

Thursday, May 3, 12

Snort Operation

Network traffic

Packets capture

SNORT

Decoders

Output Plugins

Preprocessors

Decoders

Alerts

202

Thursday, May 3, 12

Snort Configuration Snort configuration files in AlienVault can be found in the following

directories:

/etc/snort/ The main configuration file is:

/etc/snort/snort.conf Each interface can have a different configuration file using the

following naming convention

/etc/snort/snort.eth0.conf /etc/snort/snort.eth1.conf

203

Thursday, May 3, 12

Startup Options All the startup options can be found in the Snort man file

# man snort To include any of those options when Snort is started as a service,

the parameter has to be included in the following file:

/etc/default/snort Monitoring networks and listening interfaces are configured in the

file

/etc/snort/snort.debian.conf This file is edited automatically by alienvault-reconfig and should not

be edited manually

204

Thursday, May 3, 12

Snort.conf includes The Snort.conf file can include other configuration files as well as

rule definition stored in different files:

E.g.:

include threshold.conf include classification.config

205

Thursday, May 3, 12

Snort.conf variables In the main configuration files, variables can be defined. This variables

can be used in the same configuration files or in the rules definition:

HOME_NET: Local network monitored. (Automatically edited by alienvault-reconfig based on the networks defined in the OSSIM inventory)

EXTERNAL_NET: External networks (Usually !$HOME_NET -> Networks not in $HOME_NET)

Defining other variables can help us reducing the number of false positives

DNS_SERVERS, SMTP_SERVERS, SSH_PORTS

emerging.rules:alert udp $HOME_NET any -> $DNS_SERVERS 53 (msg:"ET CURRENT_EVENTS Infected System Looking up chr.santa-inbox.com CnC Server"; content:"|03|chr|0b|santa-inbox|03|com"; nocase; classtype:trojan-activity;)

206

Thursday, May 3, 12

Snort.conf Preprocessors Preprocessors are configured in the main configuration file

(snort.conf) as follows:

preprocessor : Some rules use preprocessors so they should never be disabled

as it may affect to the detection engine functionality

Each preprocessor will have a different plugin_id (The range 1000-1500 in plugin_id is reserved for Snort events)

207

Thursday, May 3, 12

Snort.conf Outputs The output plugins can be configured in Snort.conf with the

following syntax:

output : The AlienVault Sensor needs Snort running with the 2the unified

output enabled as follows

output unified2: filename snort, limit 128

Unified Output stores events in Binary format

208

Thursday, May 3, 12

Snort Output - AlienVault Sensor The Snort plugin in the Agent must be configured according to the

configuration of the unified out