Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Freezing the Web: A Study of ReDoSVulnerabilities in JavaScript-based Web Servers
Cristian-Alexandru Staicu Michael Pradel
TU Darmstadt
www.software-lab.org
15th August 2018
1/18
Regular Expression Denial of Service (ReDoS)
input: ”Lorem ipsum”
input.match(regexp);
processing time: O(1)
input: ”aoAs,”x 1000
processing tim
e: O(nx ), x
> 1
1/18
Regular Expression Denial of Service (ReDoS)
input: ”Lorem ipsum”
input.match(regexp);
processing time: O(1)
input: ”aoAs,”x 1000
processing tim
e: O(nx ), x
> 1
1/18
Regular Expression Denial of Service (ReDoS)
input: ”Lorem ipsum”
input.match(regexp);
processing time: O(1)
input: ”aoAs,”x 1000
processing tim
e: O(nx ), x
> 1
1/18
Regular Expression Denial of Service (ReDoS)
input: ”Lorem ipsum”
input.match(regexp);
processing time: O(1)
input: ”aoAs,”x 1000
processing tim
e: O(nx ), x
> 1
1/18
Regular Expression Denial of Service (ReDoS)
input: ”Lorem ipsum”
input.match(regexp);
processing time: O(1)
input: ”aoAs,”x 1000
processing tim
e: O(nx ), x
> 1
1/18
Regular Expression Denial of Service (ReDoS)
input: ”Lorem ipsum”
input.match(regexp);
processing time: O(1)
input: ”aoAs,”x 1000
processing tim
e: O(nx ), x
> 1
1/18
Regular Expression Denial of Service (ReDoS)
input: ”Lorem ipsum”
input.match(regexp);
processing time: O(1)
input: ”aoAs,”x 1000
processing tim
e: O(nx ), x
> 1
2/18
This Talk
ReDoS affects librarieswe identify 25 vulnerabilities in popular npm modules
ReDoS affects websiteshundreds of live websites are vulnerable
Novel methodologylibrary vulnerability → website vulnerability
2/18
This Talk
ReDoS affects librarieswe identify 25 vulnerabilities in popular npm modules
ReDoS affects websiteshundreds of live websites are vulnerable
Novel methodologylibrary vulnerability → website vulnerability
2/18
This Talk
ReDoS affects librarieswe identify 25 vulnerabilities in popular npm modules
ReDoS affects websiteshundreds of live websites are vulnerable
Novel methodologylibrary vulnerability → website vulnerability
3/18
Backtracking-based Matching
var regEx = /^a*a*b$/;
start 3 4 5 6
accept
7
891011
ε
ε ε
a
εε
bε ε
ε
ε ε
a
ε
3/18
Backtracking-based Matching
var regEx = /^a*a*b$/;
start 3 4 5 6
accept
7
891011
ε
ε ε
a
εε
bε ε
ε
ε ε
a
ε
3/18
Backtracking-based Matching
var regEx = /^a*a*b$/;
start 3 4 5 6
accept
7
891011
ε
ε ε
a
εε
bε ε
ε
ε ε
a
ε
input: ”ab”
3/18
Backtracking-based Matching
var regEx = /^a*a*b$/;
start 3 4 5 6
accept
7
891011
ε
ε ε
a
εε
bε ε
ε
ε ε
a
ε
input: ”aab”
3/18
Backtracking-based Matching
var regEx = /^a*a*b$/;
start 3 4 5 6
accept
7
891011
ε
ε ε
a
εε
bε ε
ε
ε ε
a
ε
input: ”aab”
3/18
Backtracking-based Matching
var regEx = /^a*a*b$/;
start 3 4 5 6
accept
7
891011
ε
ε ε
a
εε
bε ε
ε
ε ε
a
ε
input: ”aaaaaaaaaaaaaaaaaaaa”
3/18
Backtracking-based Matching
var regEx = /^a*a*b$/;
start 3 4 5 6
accept
7
891011
ε
ε ε
a
εε
bε ε
ε
ε ε
a
ε
input: ”aaaaaaaaaaaaaaaaaaaa”
4/18
Overview
Npm modules (Phase 1)ReDoS analysisof libraries
Exploits creation
Module levelvulnerabilities
Usage scenarios (Phase 2)
List of websitesusing Node.js
ReDoS analysisof websites
Payloads usingHTTP requests
List of vulner-able websites
(Phase 3)
Local machines
Live websites
4/18
Overview
Npm modules (Phase 1)ReDoS analysisof libraries
Exploits creation
Module levelvulnerabilities
Usage scenarios (Phase 2)
List of websitesusing Node.js
ReDoS analysisof websites
Payloads usingHTTP requests
List of vulner-able websites
(Phase 3)
Local machines
Live websites
4/18
Overview
Npm modules (Phase 1)ReDoS analysisof libraries
Exploits creation
Module levelvulnerabilities
Usage scenarios (Phase 2)
List of websitesusing Node.js
ReDoS analysisof websites
Payloads usingHTTP requests
List of vulner-able websites
(Phase 3)
Local machines
Live websites
4/18
Overview
Npm modules (Phase 1)ReDoS analysisof libraries
Exploits creation
Module levelvulnerabilities
Usage scenarios (Phase 2)
List of websitesusing Node.js
ReDoS analysisof websites
Payloads usingHTTP requests
List of vulner-able websites
(Phase 3)
Local machines
Live websites
5/18
Setup
measure in single instancesetup
analyze 2,800 websites fromTop 1 million
manually analyze popularpackages
fifth most-dependent uponnpm package
6/18
Node.js Particularities
Event Loop (JS code)
e1
e2
e3
...
file system
network
process
...
Events Workers
blocking I/O
callback
Regex.match()
6/18
Node.js Particularities
Event Loop (JS code)
e1
e2
e3
...
file system
network
process
...
Events Workers
blocking I/O
callback
Regex.match()
7/18
Node.js Particularities (2)
Node.jsapplication
stringsutility
templatesengine
DBaccess
headersparser
vulnerablemodule
...
...
...
8/18
Ethical Considerations
Few payloads80 requests in total
Iterative probingmost websites use redundancy
Safety mechanismstop after timeout or error
Vulnerabilities disclosurethe majority of them have been fixed
8/18
Ethical Considerations
Few payloads80 requests in total
Iterative probingmost websites use redundancy
Safety mechanismstop after timeout or error
Vulnerabilities disclosurethe majority of them have been fixed
8/18
Ethical Considerations
Few payloads80 requests in total
Iterative probingmost websites use redundancy
Safety mechanismstop after timeout or error
Vulnerabilities disclosurethe majority of them have been fixed
8/18
Ethical Considerations
Few payloads80 requests in total
Iterative probingmost websites use redundancy
Safety mechanismstop after timeout or error
Vulnerabilities disclosurethe majority of them have been fixed
8/18
Ethical Considerations
Few payloads80 requests in total
Iterative probingmost websites use redundancy
Safety mechanismstop after timeout or error
Vulnerabilities disclosurethe majority of them have been fixed
9/18
Phase 1: Npm Analysis
Criterion for vulnerable libraries
We consider a module to be vulnerable iff we find an input that
is at most 80,000 characters long,
whose matching time takes more than 5 seconds.
Manual analysis of regular expressions and information flow
Manually written exploits
9/18
Phase 1: Npm Analysis
Criterion for vulnerable libraries
We consider a module to be vulnerable iff we find an input that
is at most 80,000 characters long,
whose matching time takes more than 5 seconds.
Manual analysis of regular expressions and information flow
Manually written exploits
9/18
Phase 1: Npm Analysis
Criterion for vulnerable libraries
We consider a module to be vulnerable iff we find an input that
is at most 80,000 characters long,
whose matching time takes more than 5 seconds.
Manual analysis of regular expressions and information flow
Manually written exploits
10/18
Phase 1: Vulnerable Regular Expressions
25 ReDoS vulnerabilities
13 advisories
One bug bounty
Example 1: content
/^([^\/]+\/[^\s;]+) (?:(?:\s*;\s*boundary =(?:"([^"
]+)"|([^;"]+)))|(?:\s*;\s*[^=]+=(?:(?:"(?:[^"
]+)")|(?:[^;"]+))))*$/i
Example 2: ua-parser-js
/ip[honead ]+(.* os\s([\w]+)*\ slike\smac |;\ sopera)/
10/18
Phase 1: Vulnerable Regular Expressions
25 ReDoS vulnerabilities
13 advisories
One bug bounty
Example 1: content
/^([^\/]+\/[^\s;]+) (?:(?:\s*;\s*boundary =(?:"([^"
]+)"|([^;"]+)))|(?:\s*;\s*[^=]+=(?:(?:"(?:[^"
]+)")|(?:[^;"]+))))*$/i
Example 2: ua-parser-js
/ip[honead ]+(.* os\s([\w]+)*\ slike\smac |;\ sopera)/
10/18
Phase 1: Vulnerable Regular Expressions
25 ReDoS vulnerabilities
13 advisories
One bug bounty
Example 1: content
/^([^\/]+\/[^\s;]+) (?:(?:\s*;\s*boundary =(?:"([^"
]+)"|([^;"]+)))|(?:\s*;\s*[^=]+=(?:(?:"(?:[^"
]+)")|(?:[^;"]+))))*$/i
Example 2: ua-parser-js
/ip[honead ]+(.* os\s([\w]+)*\ slike\smac |;\ sopera)/
10/18
Phase 1: Vulnerable Regular Expressions
25 ReDoS vulnerabilities
13 advisories
One bug bounty
Example 1: content
/^([^\/]+\/[^\s;]+) (?:(?:\s*;\s*boundary =(?:"([^"
]+)"|([^;"]+)))|(?:\s*;\s*[^=]+=(?:(?:"(?:[^"
]+)")|(?:[^;"]+))))*$/i
Example 2: ua-parser-js
/ip[honead ]+(.* os\s([\w]+)*\ slike\smac |;\ sopera)/
10/18
Phase 1: Vulnerable Regular Expressions
25 ReDoS vulnerabilities
13 advisories
One bug bounty
Example 1: content
/^([^\/]+\/[^\s;]+) (?:(?:\s*;\s*boundary =(?:"([^"
]+)"|([^;"]+)))|(?:\s*;\s*[^=]+=(?:(?:"(?:[^"
]+)")|(?:[^;"]+))))*$/i
Example 2: ua-parser-js
/ip[honead ]+(.* os\s([\w]+)*\ slike\smac |;\ sopera)/
11/18
Phase 2: HTTP-level Payload Creation
Local Node.js installation
For each payload, create a usage scenario
var MobileDetect = require("mobile -detect");
var headers = req.headers["user -agent"];
var md = new MobileDetect(headers);
md.phone();
For each scenario, create HTTP level payloads
In total 8 payloads corresponding to 8 popular modules
11/18
Phase 2: HTTP-level Payload Creation
Local Node.js installation
For each payload, create a usage scenario
var MobileDetect = require("mobile -detect");
var headers = req.headers["user -agent"];
var md = new MobileDetect(headers);
md.phone();
For each scenario, create HTTP level payloads
In total 8 payloads corresponding to 8 popular modules
11/18
Phase 2: HTTP-level Payload Creation
Local Node.js installation
For each payload, create a usage scenario
var MobileDetect = require("mobile -detect");
var headers = req.headers["user -agent"];
var md = new MobileDetect(headers);
md.phone();
For each scenario, create HTTP level payloads
In total 8 payloads corresponding to 8 popular modules
11/18
Phase 2: HTTP-level Payload Creation
Local Node.js installation
For each payload, create a usage scenario
var MobileDetect = require("mobile -detect");
var headers = req.headers["user -agent"];
var md = new MobileDetect(headers);
md.phone();
For each scenario, create HTTP level payloads
In total 8 payloads corresponding to 8 popular modules
11/18
Phase 2: HTTP-level Payload Creation
Local Node.js installation
For each payload, create a usage scenario
var MobileDetect = require("mobile -detect");
var headers = req.headers["user -agent"];
var md = new MobileDetect(headers);
md.phone();
For each scenario, create HTTP level payloads
In total 8 payloads corresponding to 8 popular modules
12/18
Phase 2: Input Dependency
0
500
1000
1500
2000
0 10000
20000
30000
40000
50000
60000
70000
Matc
hin
g t
ime
(ms)
Input size (number of characters)
ua-parser-js
12/18
Phase 2: Input Dependency
0
500
1000
1500
2000
0 10000
20000
30000
40000
50000
60000
70000
Matc
hin
g t
ime
(ms)
Input size (number of characters)
ua-parser-js
useragent
12/18
Phase 2: Input Dependency
0
500
1000
1500
2000
0 10000
20000
30000
40000
50000
60000
70000
Matc
hin
g t
ime
(ms)
Input size (number of characters)
charset
fresh
forwarded
content
mobile-detect
platform
ua-parser-js
useragent
13/18
Phase 3: Websites Analysis
3x
3x
5x
5x
100ms
P1
3x
3x
5x
5x
200ms
P2
3x
3x
5x
5x
500ms
P3
3x
3x
5x
5x
1s
P4
3x
3x
5x
5x
2s
P5
13/18
Phase 3: Websites Analysis
3x
3x
5x
5x
100ms
P1
3x
3x
5x
5x
200ms
P2
3x
3x
5x
5x
500ms
P3
3x
3x
5x
5x
1s
P4
3x
3x
5x
5x
2s
P5
13/18
Phase 3: Websites Analysis
3x
3x
5x
5x
100ms
P1
3x
3x
5x
5x
200ms
P2
3x
3x
5x
5x
500ms
P3
3x
3x
5x
5x
1s
P4
3x
3x
5x
5x
2s
P5
13/18
Phase 3: Websites Analysis
3x
3x
5x
5x
100ms
P1
3x
3x
5x
5x
200ms
P2
3x
3x
5x
5x
500ms
P3
3x
3x
5x
5x
1s
P4
3x
3x
5x
5x
2s
P5
Criterion for vulnerable websites
We consider a website to be vulnerable iff:
statistically significant difference between the response timesto random and crafted inputs,
this difference increases when the input size increases.
14/18
Phase 3: Response Time of a Non-Vulnerable Website
700
800
900
1000
1100
1200
1300
1400
1500
1600
1700
1800
P1
P2
P3
P4
P5
Res
ponse
tim
e (m
s)
Payload number (increasing in size)
Random Crafted
15/18
Phase 3: Response Time of a Vulnerable Website
0
500
1000
1500
2000
2500
P1
P2
P3
P4
P5
Res
ponse
tim
e (m
s)
Payload number (increasing in size)
Random Crafted
16/18
Phase 3: Number of Vulnerable Websites
Exploit Number of sites affected
fresh 241forwarded 99
ua-parser-js 41useragent 16
mobile-detect 9platform 8charset 3content 0
In total: 339 (11%) websites are vulnerable
16/18
Defenses
0
500
1000
1500
2000
2500
3000
1 10 100
1000
10000
100000
Num
ber
of w
ebsi
tes
Accepted header size
charset
16/18
Defenses
0
500
1000
1500
2000
2500
3000
1 10 100
1000
10000
100000
Num
ber
of w
ebsi
tes
Accepted header size
fresh
forwarded
ua-parser-js
useragent
mobile-detect
platform
charset
content
17/18
Defenses (2)
Linear time matching algorithms / hybridRust programming language
Timeout on matching regular expressions[Davis et al., USENIX Security, 2018], .NET framework
Tooling support for identifying ReDoSJava programming language [Wustholz et al., TACAS, 2017]
17/18
Defenses (2)
Linear time matching algorithms / hybridRust programming language
Timeout on matching regular expressions[Davis et al., USENIX Security, 2018], .NET framework
Tooling support for identifying ReDoSJava programming language [Wustholz et al., TACAS, 2017]
17/18
Defenses (2)
Linear time matching algorithms / hybridRust programming language
Timeout on matching regular expressions[Davis et al., USENIX Security, 2018], .NET framework
Tooling support for identifying ReDoSJava programming language [Wustholz et al., TACAS, 2017]
18/18
Conclusions
ReDoS is a widespread problem in npm modules,
Npm modules vulnerabilities can be exploited in live websites
11% of websites using Express are vulnerable to ReDoS
ReDoS vulnerabilities can be used to fingerprint web servers
More tools are needed to mitigate the ReDoS risk
18/18
Conclusions
ReDoS is a widespread problem in npm modules,
Npm modules vulnerabilities can be exploited in live websites
11% of websites using Express are vulnerable to ReDoS
ReDoS vulnerabilities can be used to fingerprint web servers
More tools are needed to mitigate the ReDoS risk
18/18
Conclusions
ReDoS is a widespread problem in npm modules,
Npm modules vulnerabilities can be exploited in live websites
11% of websites using Express are vulnerable to ReDoS
ReDoS vulnerabilities can be used to fingerprint web servers
More tools are needed to mitigate the ReDoS risk
18/18
Conclusions
ReDoS is a widespread problem in npm modules,
Npm modules vulnerabilities can be exploited in live websites
11% of websites using Express are vulnerable to ReDoS
ReDoS vulnerabilities can be used to fingerprint web servers
More tools are needed to mitigate the ReDoS risk
18/18
Conclusions
ReDoS is a widespread problem in npm modules,
Npm modules vulnerabilities can be exploited in live websites
11% of websites using Express are vulnerable to ReDoS
ReDoS vulnerabilities can be used to fingerprint web servers
More tools are needed to mitigate the ReDoS risk
18/18
Conclusions
ReDoS is a widespread problem in npm modules,
Npm modules vulnerabilities can be exploited in live websites
11% of websites using Express are vulnerable to ReDoS
ReDoS vulnerabilities can be used to fingerprint web servers
More tools are needed to mitigate the ReDoS risk
18/18
Is Response Time a Good Estimator?
50
100
150
200
250
300
350
400
450
500
0 50 100
150 200
250 300
350 400
450
Res
ponse
tim
e (m
s)
Matching time (ms)
18/18
Popularity of Vulnerable Websites
0
50
100
150
200
250
300
350
0 100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
900,000
Num
ber
of vuln
erable
web
site
s
Popularity ranking
18/18
Dimensioning Payloads
Module P1 P2 P3 P4 P5100ms 200ms 500ms 1s 2s
fresh 12,000 17,000 27,000 37,500 53,500forwarded 12,000 17,000 26,500 38,000 53,500useragent 500 650 925 1,150 1,450
ua-parser-js 38 39 40 41 42mobile-detect 10,500 15,500 25,000 36,500 50,500
platform 7,500 11,000 17,500 25,000 34,500charset 10,500 15,500 24,000 34,000 48,000content 8,000 11,000 18,000 25,500 35,500