Where Flow Charts Don’t Go -- Website Security Statistics Report (2015)

Where Flow Charts Don’t Go:

© 2015 WhiteHat Security, Inc.

Jeremiah GrossmanFounderWhiteHat Security, Inc.Twitter: @jeremiahg

An Examination of Web Application Security Process Management


Jeremiah Grossman15 years of Application SecurityBrazilian Jiu-Jitsu Black Belt


WhiteHat Security

We help secure the Web by finding application vulnerabilities, in the source code all the way through to production, and help companies get them fixed, before the bad guys exploit them.

Founded: 2001Headquarters: Santa Clara, CAEmployees: 300+


Metric DefinitionsAverage time time to fix:

Average # of days to fix a vulnerability.

Remediation Rate:

# of closed vulnerabilities divided by # of open vulnerabilities.

Days Open:

Average# of days a vulnerability has been open.

Vulnerability Class Likelihood:

# of sites that have at least one open vulnerability in a given class over the total number of active sites.

Window of Exposure:

# of days a site had at least one serious vulnerability open over the analysis period.

Serious Vulnerability: Vulnerability with a severity of 3 or greater as defined by WhiteHat’s Vulnerability Classification System.


Vulnerability Likelihood and Windows of Exposure


Vulnerability Likelihood


• Likelihood of Insufficient Transport Layer Protection has increased in recent years (70% likelihood in 2014)

• Content Spoofing, XSS and Fingerprinting has declined in recent years– Content Spoofing (38% in

2010 to 26% in 2014)– Cross-site scripting (55%

in 2010 to 47% in 2014)– Fingerprinting (23% in

2012 to 5% in 2014)

Vulnerability Likelihood


• A large % of websites are always vulnerable

• 60% of all Retail are always vulnerable

• 52% of all Healthcare and Social Assistance sites are always vulnerable

• 38% of all Information Technology websites are always vulnerable

• 39% of all Finance and Insurance websites are always vulnerable

Windows of Exposure Analysis

Finance and Insurance

Health Care and Social Assistance

Information Retail Trade

39%52%

38%

60%

14%

10%

11%

9%11%

12%

14%

10%18%11%

16%

11%17% 14%

22%11%

Always VulnerableFrequently Vulnerable 271-364 days a yearRegularly Vulnerable 151-270 days a year


Maturity Metrics Analysis


• The analysis is based on 118 responses on a survey sent to security professionals to measure maturity models of application security programs at various organizations.

• The responses obtained in the survey are correlated with the data available in Sentinel to get deeper insights. Statistics pulled from Sentinel are for 2014 timeframe.

Sentinel Customer Survey Overview

Active Customers: ~700Fortune 500: 63

Commercial Banks7 of the Top 18

Largest Banks10 of the Top 50

Software6 of the Top 16

Consumer Financial Services4 of the Top 8


• 24% of the survey respondents have experienced a data or system breach

• Those who have experienced a data or system breach have higher average # of open vulnerabilities than those who haven’t experienced a breach. (20 vs. 26)

• Those who have experienced a breach have lower remediation rate than those who haven’t experienced a breach.

(42% vs. 39%)

Have organizations website(s) experienced a data or system breach resulting from an app layer vulnerability?

All


Information

Retail Trade


76%

83%

80%

50%

0%

24%

17%

20%

50%

100%

NoYes


• 56% of all respondents did not have any part of the organization held accountable in case of data or system breach.

If an organization experiences a website(s) data or system breach, which part of the organization is held accountable and what is it’s performance?

Board

of D

irect

ors

Exec

utive

Man

agem

ent

Softw

are

Devel

opm

ent

Secu

rity

Depar

tmen

t0%

5%

10%

15%

20%

25%

30%

35%

9%

29% 28% 30%


If an organization experiences a website(s) data or system breach, which part of the organization is held accountable and what is it’s performance?

0

15

30

10 10 17 25

Average Number of Vulns Open

0200400

386 364 341 299

Average Time Open (Days)

95110125

129 119 108 114

Average Time to Fix (Days)

34%

38%

42%

46%

44% 43%37%

43%

Remediation Rate


• 15% of the respondents cite Compliance as the primary reason for resolving website vulnerabilities

• 6% of the respondents cite Corporate Policy as the primary reason for resolving website vulnerabilities

• 35% of the respondents cite Risk Reduction as the primary reason for resolving website vulnerabilities

• 19% of the respondents cite Customer or Partner Demand as the primary reason for resolving website vulnerabilities

• 25% of the respondents cite other reasons for resolving website vulnerabilities

Please rank your organization’s drivers for resolving website vulnerabilities. 1 lowest priority, 5 highest.

15%

6%

35%

19%25%

Primary driver for resolving website vulnerabilities

% o

f re

spo

nd

en

ts


Please rank your organization’s drivers for resolving website vulnerabilities. 1 the lowest priority, 5 the highest.

0

15

30

14 21 28 2810

Average # of vulnerabilities

Primary reasons for resolving web site vulnerabilities

0100200300400500600

266 290 283

525355

Average Time Open (Days)


0

100

200

132 86 78163 150

Average Time to Fix (Days)


0%20%40%60%

55%21% 40% 50% 33%

Average Remediation Rate



• % of respondents for frequency of automatic static analysis:

• Daily: 13%• With each major release:

32% • Never: 13%

• # of open vulns for frequency of automatic static analysis:

• Daily: 6• With each major release: 32 • Never: 17

How frequent do you perform automated static analysis during the code review process?


Information

Retail Trade


All

0% 20% 40% 60% 80% 100%

DailyMonthlyNeverOther (please spec-ify)PlannedQuarterlyWeeklyWith each re-lease or ma-jor update

Daily

Never

Weekly

0 10 20 30 40 50 60

All


Retail Trade

Information



• Avg time open for frequency of automatic static analysis:

• Daily: 369 days• Each major release: 273

days • Never: 394 days

• Remediation rate for frequency of automatic static analysis:

• Daily: 39%• Each major release: 38%• Never: 45%


Daily

Never

Quarterly

With each release or major update

0 100200300400500600700

Average Time Open at different frequencies of Automated Static Analysis

All


Retail Trade

Information

Finance and In-surance

Daily

Monthly

Never

Quarterly

Weekly


All

0% 20% 40% 60% 80% 100%

Average remediation rate at different frequencies of Automated Static Analysis

All


Retail Trade

Information



• Time to fix for frequency of automatic static analysis:

• Daily: 74 days• Each major

release: 117 days• Never: 125 days


Daily

Monthly

Never

Other (please specify)

Quarterly

Weekly


0 100 200 300 400 500

Average Time to fix at different frequencies of Au-tomated Static Analysis

All

Health Care and Social Assis-tance

Retail Trade

Information


© 2014 WhiteHat Security, Inc. 19

• % of respondents for frequency of adversarial testing:

Each major release: 32%Quarterly: 11%Never: 21%

• # of open vulns for frequency of adversarial testing:

Each major release: 15Quarterly: 14Never: 34

How frequently does the QA team go beyond functional testing to perform basic adversarial tests (probing of simple edge cases and boundary conditions) example: What happens when you enter the wrong password over and over?


Information

Retail Trade


All

0% 50% 100%

Frequency of Adversarial Testing by Industry

DailyMonthlyNeverOther (please spec-ify)PlannedQuarterlyWeeklyWith each release or major update

Daily

Never

Planned

Weekly

0 10 20 30 40 50 60

Average # of vulns at different frequencies of adversarial testing

AllHealth Care and Social AssistanceRetail TradeInformationFinance and In-surance


• Avg time open for frequency of adversarial testing:

Each major release: 322 daysQuarterly: 375 daysNever: 254 days

• Remediation rate for frequency of adversarial testing:

Each major release: 41%Quarterly: 40%Never: 25%


DailyMonthly

NeverOther (please specify)

PlannedQuarterly

WeeklyWith each release or major update

(blank)

0 200 400 600 800

Average Time Open at different fre-quencies of adversarial testing


Daily

Never

Planned

Weekly

0% 20% 40% 60% 80%

Average remediation rate at different frequencies of adversarial testing



• Time to fix for frequency of adversarial testing:

Each major release: 124 daysQuarterly: 85 daysNever: 102 days


Daily

Never

Planned

Weekly

0 50 100

150

200

250

Average Time to fix at differ-ent frequencies of adversarial

testing

AllHealth Care and Social AssistanceRetail TradeInformationFinance and Insurance


• % of respondents for frequency of pen-testing:

Annually: 21% Quarterly: 26% Never: 26%

• # of open vulns for frequency of pen-testing:

Annually: 12Quarterly: 40Never: 25

How frequently do you use external penetration testers to find problems?


Information

Retail Trade


All

0% 20% 40% 60% 80% 100%120%

Frequency of Penetration Testing by Industry

AnnuallyDailyMonthlyNeverOther (please spec-ify)PlannedQuarterlyWeeklyWith each release or major update

Annually

Monthly

Planned

Weekly

0 10 20 30 40 50 60

Average # of vulns at different frequencies of penetration testing

All


Retail Trade

Information



• Avg time open for frequency of penetration testing:

Annually: 282 daysQuarterly: 273 daysNever: 393 days

• Remediation rate for frequency of penetration testing:

Annually: 49%Quarterly: 44%Never: 34%


AnnuallyDaily

MonthlyNever

Other (please specify)Planned

QuarterlyWeekly

0 100 200 300 400 500 600 700

Average Time Open at different frequencies of penetration testing

Grand TotalHealth Care and Social Assis-tanceRetail TradeInformationFinance and Insurance

AnnuallyDaily

MonthlyNever


QuarterlyWeekly


0%20%

40%60%

80%

Average remediation rate at different fre-quencies of penetration testing

Grand TotalHealth Care and Social AssistanceRetail TradeInformationFinance and Insurance


• Time to fix for frequency of penetration testing:

Annually: 140 daysQuarterly: 102 daysNever: 128 days


Annually

Monthly


Quarterly

With each release or

major update

0 100 200 300

Average Time to fix at different frequencies of penetration test-

ing

Grand TotalHealth Care and Social AssistanceRetail TradeInformationFinance and Insurance


• % of respondents for frequency of operation monitoring feedback:

Daily: 17% With each major release: 17% Never: 9%

• # of open vulns for frequency of operation monitoring feedback:

Daily: 40With each major release: 23Never: 10

How often does your organization use defects identified through operations monitoring fed back to development and used to change developer behavior?


Information

Retail Trade


All

0%50%

100%150%

Frequency of Operations Monitoring Feedback by Industry

Annually DailyMonthly NeverOther (please specify) PlannedQuarterly WeeklyWith each release or major update

Annually

Never

Quarterly

0 10 20 30 40 50 60

Average # of vulns at different fre-quencies of Operations Monitoring

Feedback AllHealth Care and Social AssistanceRetail TradeInformationFinance and In-surance


• Avg time open for frequency of operation monitoring feedback:

Daily: 270 daysWith each major release: 353 daysNever: 243 days

• Remediation rate for frequency of operation monitoring feedback:

Daily: 32%With each major release: 48%Never: 34%


AnnuallyDaily

MonthlyNever


QuarterlyWeekly


0 200 400 600 800

Average Time Open at different frequencies of Operations Monitor-

ing Feedback Health Care and Social AssistanceRetail TradeInformationFinance and In-surance

Annually

Never

Quarterly

0% 10% 20% 30% 40% 50% 60% 70%

Average remediation rate at different frequencies of Operations Monitoring

Feedback Health Care and Social AssistanceRetail TradeInformationFinance and Insurance


• Time to fix for frequency of operation monitoring feedback:

Daily: 76 daysWith each major release: 198 daysNever: 91 days


Annually

Monthly


Quarterly


major update

0 50 100 150 200 250 300

Average Time to fix at different fre-quencies of Operations Monitoring

Feedback

Health Care and Social AssistanceRetail TradeInformationFinance and In-surance


• % of respondents for frequency of ad hoc code reviews:

Never: 21% Planned: 15% With each major release: 15%

• # of open vulns for frequency of ad hoc code reviews:

Never: 41 Planned: 10 With each major release: 13

How frequently does your organization perform ad hoc code reviews of high risk applications in an opportunistic fashion?

Finance and InsuranceInformationRetail Trade

Health Care and Social AssistanceAll

Frequency of Adhoc Code Review by Industry

Annually Daily MonthlyNever Other (please

specify)Planned

Quarterly Weekly With each release or major update

Annually

Never

Quarterly

0 50 100

Average # of vulns at different frequencies of Adhoc code review

All Health Care and Social Assistance

Retail Trade InformationFinance and Insurance


• Avg time open for frequency of ad hoc code reviews:

Never: 309 daysPlanned: 264 daysWith each major release: 278 days

• Remediation rate for frequency of ad hoc code reviews:

Never: 43%Planned: 39%With each major release: 37%


AnnuallyDaily

MonthlyNever


QuarterlyWeekly


0 100200300400500600

Average Time Open at different fre-quencies of adhoc code review


AnnuallyDaily

MonthlyNever


QuarterlyWeekly


0 0.10.20.30.40.50.60.7

Average remediation rate at different frequencies of adhoc code review



• Time to fix for frequency of ad hoc code reviews:

Never: 147 days Planned: 90 daysWith each major release: 102 days


Annually

Monthly


Quarterly


major update

0 20 40 60 80 100

120

140

160

180

Average Time to fix at dif-ferent frequencies of adhoc

code review


Retail Trade

Information



• % of respondents for frequency of security review sharing:

Monthly: 13%With each major release: 28%Never: 19%

• # of open vulns for frequency of security review sharing:

Monthly: 13With each major release: 29Never: 18

How frequently does your organization share results from security reviews with the QA department?


Retail Trade

All

0%50%

100%150%

Frequency of Security Result Sharing by Industry

Daily MonthlyNever Other (please specify)Planned QuarterlyWeekly With each release or major

update

Daily

Never

Planned

Weekly

0 10 20 30 40 50 60

Average # of vulns at different frequencies of Security Result Sharing



• Avg time open for frequency of security review sharing:

Monthly: 282 daysWith each major release: 393 daysNever: 258 days

• Remediation rate for frequency of security review sharing:

Monthly: 49%With each major release: 37%Never: 27%


Daily

Monthly

Never


Planned

Quarterly

Weekly


0 200 400 600 800

All

Health Care and Social Assis-tance

Retail Trade

Information


Daily

Monthly

Never


Planned

Quarterly

Weekly

With each release or major updateAll


Retail Trade

Information



• Time to fix for frequency of security review sharing:

Monthly: 107 daysWith each major release: 162 daysNever: 83 days


Daily

Monthly

Never


Planned

Quarterly

Weekly


0 50 100 150 200

Average Time to fix at different frequencies of Security Result Shar-

ing

AllHealth Care and Social AssistanceRetail TradeInformationFinance and Insurance

Questions?


Jeremiah GrossmanFounderWhiteHat Security, Inc.Twitter: @jeremiahg

Thank you!