ArcSight Variable and OperatorsHOWTO · Specifically for ArcSight the evaluate_velocity_template variable allows interaction via the velocity language with ArcSight event information

Securing Your World

© 2013 Accumuli Security

1

Document Version 1.0

Document Classification Public

Descriptions and examples of some of the ArcSight variables and operators.

ArcSight Variable and Operators HOWTO


Securing Your World

ArcSight Variable and Operators HOWTO 02/08/2013 Document Version 1.0 Document Classification Public

2

Accumuli Security Tuscany House, White Hart Lane, Basingstoke, RG21 4AF, United Kingdom Tel: +44 (0) 1256 303 700 Fax: +44 (0) 1256 303 735 Email: [email protected] Web: www.accumuli.com

Copyright & Confidentiality Statements This document is Copyright © Accumuli Security Networks Ltd All rights reserved. The contents of this document may not be copied or duplicated in any form, in whole or in part, without the prior written permission of Accumuli Security Networks Ltd. The information in this document is subject to change without notice. Accumuli Security Networks Ltd shall not be liable for any damages resulting from technical errors or omissions which may be present in this document, or from use of this document. This document is an unpublished work protected by the United Kingdom copyright laws and is proprietary to Accumuli Security Networks Ltd. Disclosure, copying, reproduction, merger, translation, modification, enhancement, or use of this document by anyone other than authorised employees, authorised users, or licensees of Accumuli Security Networks Ltd without the prior written consent of Accumuli Security Networks Ltd is prohibited. Document Information Name: ArcSight Variable and Operators HOWTO Version: Document Version 1.0 Publication Date: 23 September 2013 Classification: Document Classification Public Prepared by: Mark Johnston Document Status: Final Document Control Version Date Author Change Summary Document Version 1.0 23/09/2013 Mark Johnston Final

Accumuli Corrective Action Accumuli Corrective Action


Securing Your World


3


Contents Copyright & Confidentiality Statements ......................................................................................... 2 Document Information ...................................................................................................................... 2 Document Control ............................................................................................................................. 2 Introduction ....................................................................................................................................... 5 Background ....................................................................................................................................... 5 EvaluateVelocityTemplate ................................................................................................................ 5

General ........................................................................................................................................ 5 Location ....................................................................................................................................... 5 Example – Search for usernames matching the pattern “1 character followed by 5 digits” ......... 5 Example – Compare ActiveList Entry to a regular expression (regex) ........................................ 6 Example – String Substitution ...................................................................................................... 8 Example – String Substitution Expanded (useful for activel:104) ................................................ 9 Example – Sub Strings ................................................................................................................ 9 Example – Conditional String Checks ......................................................................................... 9 Example – Others ........................................................................................................................ 9

JavaMathematicalExpression ........................................................................................................ 10 General ...................................................................................................................................... 10 Location ..................................................................................................................................... 10 Notes ......................................................................................................................................... 10 Example – Add 10 to the value of deviceCustomNumber1 ....................................................... 10 Example – Calculate distance in KM’s given latitude and longitude of two locations ................ 10

Active List Contains ....................................................................................................................... 12 General ...................................................................................................................................... 12 Notes ......................................................................................................................................... 12

Like Logical Operator ..................................................................................................................... 14 General ...................................................................................................................................... 14 Location ..................................................................................................................................... 14 Notes ......................................................................................................................................... 14 Example – Simple Matching ...................................................................................................... 14

Matches Logical Operator .............................................................................................................. 15 General ...................................................................................................................................... 15 Location ..................................................................................................................................... 15


Securing Your World


4


Notes ......................................................................................................................................... 15 Example – Search for an IP address in a String ........................................................................ 15 Example – Search for the domain name “@accumuli.com” in a string ...................................... 15 Example – Search for usernames matching the pattern “1 character followed by 5 digits” ....... 15


Securing Your World


5


Introduction Background ArcSight provides a number of “variables” and operators that can be used within Content to help massage and search through event data. This document looks at some of the more useful variables and operators for which there is little existing documentation. EvaluateVelocityTemplate

General Velocity is a java based template engine developed by the apache foundation. It provides a language to reference objects defined in java code. Further information on Velocity can be found at http://velocity.apache.org/engine/devel/vtl-reference-guide.html Below is a table of the most common operators used in Velocity.

Specifically for ArcSight the evaluate_velocity_template variable allows interaction via the velocity language with ArcSight event information and other ArcSight variables. The variable is available for most resources. As an example, it could be used in an Active Channel (by implementing FieldSets) to give an Analyst an immediate view on rule fire information in a column.

Location The EvaluateVelocityTemplate function is found under Variables (local or global) String EvaluateVelocityTemplate

Example – Search for usernames matching the pattern “1 character followed by 5 digits”

evaluate_velocity_template(#if($message.matches(‘[A-Za-z]\d{5}’))matches#else notmatches#end)


Securing Your World


6


Example – Compare ActiveList Entry to a regular expression (regex)

Create an Active list with sample values

Create global variable with local variables used to extract AL information

Set actual global variable using local variables (calling $userdata which is local to the global)

Set the event field “reason” with the global variable value


Securing Your World


7


Insert some test data, and we see the “reason” field being set with appropriate info

Global variable can also be used in conditions

Setting the “reason” field to “This works” to test use of global variable in conditions


Securing Your World


8


Channel with results

This can also be done locally within the rule if running an older version of ESM

Matches logical operator can also be used to test for the match

Example – String Substitution

evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceAll(‘x’,’y’))

evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceAll(‘[^x]’,’y’))

Assume message string user:mark:stuffhere:otherstuff:12345 and only want to return user value

evaluate_velocity_template(#set($tempvar=$message)$tempvar.replaceAll(‘user:([^:]+).*’,’$1’))


Securing Your World


9


Example – String Substitution Expanded (useful for activel:104)

Assume that activelist expiry comes from a list with two columns, rather than using indexOf …

String1 is evaluate_velocity_template($deviceCustomString4.replaceAll(‘([^\|]*)\|([^\|]*)’,’$1’)

String2 is evaluate_velocity_template($deviceCustomString4.replaceAll(‘([^\|]*)\|([^\|]*)’,’$2’)

Example – Sub Strings

Assume message string of xx|yy|zz

evaluate_velocity_template($message.substring(3,5)), which will return “yy”

evaluate_velocity_template($message.substring(0,$message.indexOf(‘|’))), which will return “xx”

evaluate_velocity_template($message.substring($message.lastIndexOf(‘|’),8)) will return “zz”

Example – Conditional String Checks evaluate_velocity_template(#if($message==”test”)Matches#else Not_Matches#end evaluate_velocity_template(#if($message==”test”)ret_test#elseif($message==”other”)ret_other#else No_Match#end

evaluate_velocity_template(#if($message != “test”)No Match#else Match#end

Example – Others Multiple VM expressions in single template

evaluate_velocity_template(#set($tempvar=$message.length())#if($tempvar > 10)Large#else Small#end


Securing Your World


10


JavaMathematicalExpression

General The function allows for Java mathematical expressions to be applied against fields (numerical only) within ArcSight. This function is not available under all resources, but can be used with Rules, Filters, Data Monitors and FieldSets. By virtue of being available in FieldSets, this function can thus be used within Active Channels. See http://www.singularsys.com/jep/doc/html/functions.html for the available mathematical functions.

Location The JavaMathematicalExpression function is found under Variables (local or global) Arithmetic JavaMathematicalExpression

Notes ArcSight fields are referenced WITHOUT the “$” within the expression. For example “deviceCustomNumber1 + 1”

Example – Add 10 to the value of deviceCustomNumber1

evaluate_jep(deviceCustomNumber1 + 10)

Example – Calculate distance in KM’s given latitude and longitude of two locations

The variables


Securing Your World


11


The screenshot showing the results (validated with online calculator)


Securing Your World


12


Active List Contains

General Active list “contains” is not currently supported by ArcSight. However during our testing it was possible to extract regex expressions from an Active List and have those compared against respective string values. The screenshots below show the implementation.

Notes It may be possible to achieve the “activelist contains” by using the ESM plugin. The plugin allows for the interaction of custom java code with events before they are written to or retrieved from the database.

Create the activelist, must have a key field in order to be looked up.

Insert all regex into regex expr field with a separator.


Securing Your World


13


Extract the regex expr values and then break them down (like activelist:104). Once broken down evaluate each regex expression against the field required.

Screenshot showing the results


Securing Your World


14


Like Logical Operator

General The ArcSight LIKE rule operator functions exactly the same as the SQL LIKE condition with the exception of not being able to use escape characters (escape characters are used to match literal “%” or “_”). For the SQL LIKE condition there are two wildcard matches, these being ‘%’ and ‘_’. The ‘%’ allows matching of any string of any length (including zero length). The ‘_’ allows matching on a single character.

Location Like is a logical operator and thus can be found under Event Conditions, in the Common Conditions Editor.

Notes Variables cannot be used for comparison. For example “Message Like $test”

Example – Simple Matching

Given “event.message = hello” the following will work:

Message Like %ell%

Message Like h%

Message Like hell_

Message Like %ell_


Securing Your World


15


Matches Logical Operator

General The ArcSight MATCHES logical operator provides the capability to use regular expressions in rule conditions only for string matching. The regex engine is based upon Perl 5 syntax, and within ArcSight the regular expression must match the entire string to be true (wildcards can be used).

Location The logical operator can only be used within Rules.

Notes Commas cannot be used within the regular expression (ArcSight limitation). For example you cannot stipulate min and max occurrences \d+{1,3}. If shorthand character classes are used (or any other classes that use a “\”) e.g. “.*\d+\.\d+\.\d+\.\d+.*” the rule editor will automatically insert the escaping “\” and encapsulate the expression in quotes when the OK button is clicked.

Example – Search for an IP address in a String

Assume event.message = Check for IP 10.10.10.10 in this field

Message Matches “.*\\d+\\.\\d+\\.\\d+\\.\\d+.*”

Example – Search for the domain name “@accumuli.com” in a string

Assume event.message = From [email protected] to outside

Message Matches .*@accumuli.com.*

Example – Search for usernames matching the pattern “1 character followed by 5 digits”

Assume event.destinationUserName = I12345

Assume event.destinationUserName = x12345

Destination User Name Matches “[A-Za-z]\\d{5}”

Documents

ArcSight Variable and OperatorsHOWTO · Specifically for ArcSight the evaluate_velocity_template variable allows interaction via the velocity language with ArcSight event information