SMB Traffic Analyzer @ SDC 2010

2010 Storage Developer Conference. Insert Your Company Name. All Rights Reserved.

SMB Traffic Analyzer

Holger Hetterich

Level 3 Support EngineerSUSE Linux Products GmbH

2010 Storage Developer Conference. Insert Your Company Name. All Rights Reserved.

SMB Traffic Analyzer – use case

The goal of SMB Traffic Analyzer is to find an answer to questions like:Which services are my most used ones?How is my Samba network used in the night?Which services are almost never used?Which users are the most pressing ones on the

Samba network?How much is a specific file being used?When was that specific file renamed and by

whom?

2010 Storage Developer Conference. SUSE Linux products GmbH, a Novell business All Rights Reserved.

What is SMB Traffic Analyzer?

3

We call it SMBTA in the following Module for the Virtual File System layer of Samba

Capture meta data of prominent functions in the VFS layer.

Send the data to a receiver SMBTAD receives the data and builds a SQL

storage from it. SMBTATOOLS, utilities to assist in querying the

database and support real time monitoring.

2010 Storage Developer Conference. SUSE Linux products GmbH, a Novell business All Rights Reserved. 4

World of SMBTA - Overview


Looking at the VFS module


The VFS Module

Version 1, and 2, we are talking about the latterSMBTA v2 going to be released with Samba 3.6.0

Supported VFS operations: Mkdir, chdir, write, read, pread, pwrite, rename, open, close

Fully transparent to the user AES encryption support Extendable protocol Configurable with standard Samba methods

( smb.conf )


A typical transfer

Protocol HeaderSpecifies encryption

and length of the datablock

VFSOperation

ID

Involvedusername

USERSID

InvolvedShare

DomainTime

Stamp

NumberOf byteswritten

Common data Block

Individual VFS function data

VFS function write

VFS Module

SMBTAD

File w/full path


A typical transfer



VFSOperation

ID

Involvedusername

USERSID

InvolvedShare

DomainTime

Stamp


Common data Block


VFS function write

VFS Module

SMBTADThe common data blockis extendable, it's size isspecified in the header.

File w/full path


A typical transfer



VFSOperation

ID

Involvedusername

USERSID

InvolvedShare

DomainTime

Stamp


Common data Block


VFS function write

VFS Module

SMBTADThe common data blockis extendable, it's size isspecified in the header.

And also, the headerIncludes a subversion

Number, and a few extraBytes to be used in future

File w/full path


Transparent and stackable

static ssize_t smb_traffic_analyzer_write(vfs_handle_struct *handle, \ files_struct *fsp, const void *data, size_t n){ struct rw_data s_data;

s_data.len = SMB_VFS_NEXT_WRITE(handle, fsp, data, n); s_data.filename = fsp->fsp_name->base_name; DEBUG(10, ("smb_traffic_analyzer_write: WRITE: %s\n", fsp_str_dbg(fsp)));

smb_traffic_analyzer_send_data(handle, &s_data, vfs_id_write); return s_data.len;}

The VFS „write“ function as implemented by the SMBTA module






Stackable ! Call the NEXT function in the VFS layer.







Transparent ! Send the dataand return the number

of bytes just as anyVFS write function


Stackable ! Call the NEXT function in the VFS layer.


Encryption of data

128 Bit AES

VFS Module SMBTADUsing the same key

Samba 3.6.0 introduces the program „smbta-util“ which will make the SMBTA setup for encryption easy. It is able to generate keys, and to enable encryption or disable it on the fly. The generated keys are easily useable by SMBTAD as a keyfile.


Configuration – via smb.conf

[Distribution Space] vfs object = smb_traffic_analyzer smb_traffic_analyzer:host = localhost smb_traffic_analyzer:port = 3490 smb_traffic_analyzer:protocol_version = V2 comment = Blah inherit acls = Yes path = /distspace read only = No

Example of a share definition that is SMBTA enabled.


This is the ultimate evil !

Exposing user related data is illegal in many countries !Two methods of anonymization built in:

Prefix + hashnumberPrefix only (full anonymization)


World of SMBTA - SMBTAD


SMBTAD – concept overview

Networkhandler

Cache

Databasefeeder

Store incoming VFS data fast

VFS Tra

ffic

SMBTAUTILS

SQLITE

Handle clientRequests


SMBTAD – caching

Temporarily store VFS data in the Systems RAM Be quick : the coolness of talloc_pool ! The database feeder runs as a thread:

Sleep !Check the cache, open a new cache, and

feed the old contents into the database.Sleep !…


SMBTAD – Battle for Performance

Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 8 Run 9 Run 10

0,00 s

20,00 s

40,00 s

60,00 s

80,00 s

100,00 s

120,00 s

140,00 s

160,00 s

180,00 s

Pure Samba ServerSMBTA enabled (talloc_pool patch 3)

SMBTA enabled (talloc_pool patch 2)SMBTA enabled (talloc_pool patch 1)

SMBTA enabled

121,41 s118,84 s 120,20 s 121,16 s 123,35 s

117,65 s114,87 s 114,50 s

123,26 s 121,61 s

127,61 s 127,89 s 129,16 s133,46 s

127,05 s131,12 s

127,29 s132,53 s

125,09 s 125,01 s

SMBTAD Performance test

done by the smbtatorture utility

Pure Samba ServerSMBTA enabled (talloc_pool patch 3)SMBTA enabled (talloc_pool patch 2)SMBTA enabled (talloc_pool patch 1)SMBTA enabled

On average, weAre 8,9 seconds behind, that is a performance decrease of about 7,4 %.

The decrease is much less if SMBTA is run on a dedicated system. Similar tests at SUSE labs with several systems resulted in about 2-3 %.

Battleground : ThinkPad X61 standalone


SMBTAD – maintain the DB

The database needs to be maintained, it would otherwise grow and grow.A configureable maintenance timer and

process is included in SMBTAD.Clean up any data that is older than a given

timespanRun this maintenance process at regular intervalls


World of SMBTA – SMBTATOOLS


SMBTATOOLS

SmbtaqueryProduce reports/statistics from the dataRuns complex queries, may take timeWorks with a simple interpreter to make

querying easy for users. Smbtamonitor

Real time monitoring


SMBTATOOLS - smbtaquery

OBJECTUsername

ShareFile

DomainGlobal

RESULT

Smbtaquery - built-in interpreter

- hides the complexity of the database to the end user- easy to learn syntax- identification of given objects, adds requirements for unique identification automatically

ACTIONTotal,List,Top,

Usage,last_activity


Screenshots of smbtaquery










SMBTAQUERY

Any smbtaquery object understandsFrom … toSince

'global since yesterday, usage r;' 'user holger from 10-23-2010 00:01:00 to today,

total rw;'


SMBTATOOLS - smbtamonitor

Idea: Enable Real-Time by omitting database queries, instead work directly with the incoming data in SMBTAD.SMBTAD includes a subsystem for monitors:

Filter incoming information in realtime for objectsMake internal Database queries to initizalize a monitor objectRun a specific monitor function (such as Throughput per second)

Displays real time information on a given ObjectThroughput R / W / RW by secondTotal numbersLive logging

Runs as many monitor instances as wanted


SMBTA – project outlook

Release 0.1, when it's done :)

What's missingDocumentation!

Open bugs ( bugzilla.novell.com, [SMBTA] in the subject )

Release 0.2 with:

XML support for smbtaqueryExport to openoffice, HTML and others

Web interface for smbtaquery and smbtamonitorUsing smbtaquery as engine

Run a client side round robin database w/ smbtamonitor

AES Encryption SMBTAD ↔ SMBTATOOLS

Additional features in the VFS moduleOptional compression

Support for clustered Samba


SMBTA – Information and Q&A

SMB Traffic Analyzer ( GPL v3 ) http://holger123.wordpress.com/smb-traffic-analyzer/

Core team: Holger Hetterich <[email protected]>

Overall

Michael Haefner

smbtamonitor

Benjamin Brunner

smbtaquery

Björn Geuken

Graphical interfaces

Ralf Schwiete

Port to SOLARIS

Q & AThank you Samba Team!Thanks to Novell/SUSE!

mailto:[email protected]

Documents

SMB Traffic Analyzer @ SDC 2010