Developer Studies

COMP 150-4: Human Factors in Security and Privacy

1

Lecture 19

Prof. Daniel VotipkaSpring 2021

(some slides courtesy of Michelle L. Mazurek and Adam Aviv)

Administrivia• HW3 due on Thursday (4/15) • Bibliographies should follow the SOUPS template • Upcoming Talks:

• NSA Hot Topics on the Science of Security conference• https://cps-vo.org/group/hotsos/registration

• Jeremiah Onaolapo (Next Thursday @ 3pm) — Tales of Malicious Online Activity

• Link: https://tufts.zoom.us/j/98610939077

2

What we did last time!

3

• End User Breach Notifications • How do users respond? • What do typical notifications look like? • Can we make better notifications?

• Enterprise Breach Notifications

What are we doing today?

4

• Developers are not the enemy either! • Secure development ecosystem • Why do developers make mistakes? • Developers support forums

• How can we make them more secure

Recall Users are not the enemy!

• We have to design security systems with users in mind

• Users may be unable to complete security tasks for a variety of reasons, including the design of the system.

Adams, Anne, and Martina Angela Sasse. "Users are not the enemy." Communications of the ACM 42.12 (1999): 41-46.

Why are users stupid or lazy? How can we make

security more usable?

What about developers, are they the enemy?

“Developers, developers, developers!”

Can you think of security/privacy frustrations as an end user that a developer could have prevented? • Confusing password requirements (password policy)

• Privacy policies hard to read

• Developer isn’t knowledgeable enough to be building a system (encryption, authentication)

• Often unknown for users is something’s being done correctly

• Systems are not designed with security in mind, security added as an afterthought (example: financial systems in transit, info sharing via email)

• Breaches: developers make bad choices & that’s why they are happening

Can you name security/privacy requirement that you had to implement (in class/job/for fun)? Was it hard to do? Why?

Where did you get support?

Why are developers stupid,

lazy, or evil? How can we make secure programming

easier?

How to help developers write secure (& privacy-preserving) code?

Personal:

• Security knowledge

• Prior experience

• Personality

Support:

• Language design

• Security analyses

• APIs

• Documentation

Organizational:

• Security vs. profit

• Company culture

• Company processes

Environmental:

• Policies/regulations

• Norms and user expectations

How and why do developers get it wrong?

Research Questions

What types of vulnerabilities do developers introduce?

• What causes these vulnerabilities to be introduced?

• What are the most prevalent vulnerabilities?

How can we measure this?

• Field studies • Field surveys • Lab studies

Build it, Break it, Fix it

Secure programming competition in three parts: • Build-It

‣ Develop to secure spec with open choices ‣ Incentivized for functionality, performance, and security

• Break-It ‣ Teams search for vulnerabilities and produce exploits in each

others’ programs • Fix-It

‣ Fix identified problems

[Ruef et al., 2016]

Mimics real-world in a controlled setting

Secure programming competition in three parts: • Build-It

‣ Develop to secure spec with open choices ‣ Incentivized for functionality, performance, and security

• Break-It ‣ Teams search for vulnerabilities and produce exploits in each

others’ programs • Fix-It

‣ Fix identified problems

[Ruef et al., 2016]

Build it, Break it, Fix it

Analysis Approach

Examined 94 projects with 866 reported breaks • Iterative open coding

‣ Vulnerability type, discovery difficulty, exploitability, and severity

• Used qualitative and quantitative analyses

Vulnerability Types

MistakeNo implementation Misunderstanding

Intuitive Bad Choice

Conceptual ErrorUnintuitive

No implementation

Vulnerability Types

Vulnerability Types

No implementation

• Missed “intuitive” requirement ‣ No encryption ‣ No access control

Intuitive

Vulnerability Types• Missed “intuitive” requirement

‣ No encryption ‣ No access control

• Missed “unintuitive” requirement ‣ No integrity check ‣ No replay protection

No implementation

UnintuitiveIntuitive

Vulnerability Types

Misunderstanding

Vulnerability Types

• Made a “bad choice” ‣ Weak algorithm ‣ Homemade encryption

Misunderstanding

Bad Choice

Vulnerability Types

• Made a “bad choice” ‣ Weak algorithm ‣ Homemade encryption

• Made a “conceptual error” ‣ Fixed value ‣ Disabling protections

Misunderstanding

Bad Choice Conceptual Error

Vulnerability Types

Mistake

Vulnerability Types

• Made a “Mistake” ‣ Control flow mistake ‣ Skipped algorithmic step Mistake

Vulnerability Types

MistakeNo implementation Misunderstanding

Intuitive Bad Choice

Conceptual ErrorUnintuitive Least common

Best practices and code review helped

50% 56% 21%

Prevalence

No implementation Misunderstanding Mistake

Unintuitive Bad Choice Conceptual ErrorIntuitive

44%45% 21%16%

Summary

Developers struggle with security concepts • MOOC lectures covered these topics ‣ Education should include practical

examples to display need and assumptions

So how does the application developer actually do development?

What Resources do Developers Use?

How Android developers look up information

• What do Android developers do when they encounter a security- or privacy-related issue?

• Which information sources do they use to look up security- or privacy-relevant questions?

• Does the use of Stack Overflow really lead to less secure code than the use of other resources?

• Is the official Android documentation really less usable, resulting in less functional code compared to other resources?

Online Survey of Application Developers

• Emailed 50,000 developers who were listed in the Google Play Store

• 302 people completed the survey, 7 removed

• Final: 295 developers with apps listed in the Google Play Store

• Low conversion rate!

Not that much usage of encryption

Side research question: How has this has changed?

Resources used in development

Why might permissions be different?

How would you find out whether these resources have an impact on code security?• Break up devs into groups • Give them different docs • Develop secure app • Test results

Developer Lab Study

Solve 4 programming tasks

How would you choose these tasks for Android? • Critical + short time

Store user login credentials

Change HTTP to HTTPS connections

• Hostname mismatch errors

• Service certificate was issued for a subdomain instead of the top level domain

Least Privilege

Inter Process Communication• Set up an intent so that this

application can be called from another application

Treatments

• Stack Overflow • Only use Stack Overflow for looking

up information

• Official Android Documentation • Restrict access to anything at

developer.android.com

• Books • Only use books (in pdf or print) • Pro Android 4 • Android Security Internals

• Free Use • Start at google.com and do

whatever

Recruitment

• 54 Developers • 12 USA • 43 Germany

• 48 Students • 8 Women • 40 Men

• 6 Professional developers

• 20 minute limit to solve the task

• Four tasks in random order

Anything stand out here?

Confidence in solution

“Free to use any resource” are a lot more confident

“Official resources and Stack Overflow (SO) similar confidents, and maybe official the least

Usability of resourcesFree to use any

resource was the most usable

Book the least usable

Book and Office Documentation were

least helpful

Stack Overflow and Free resources viewed

as helpful

Does this make sense to you?

Books and Official viewed as most correct,

closely followed by Stack Overflow

Free resources viewed perhaps least likely to

be correct

Functional (maybe not secure) Correctness

• Overall Functional Correctness:

• 67% Stack Overflow • 66% book • 52% Free choice • 40% official

documentation

What does this say about the task as well as the resources?

Functional (and secure) Correctness

• Security: • 86% official

documentation • 73% book • 66% Free choice • 52% Stack

Overflow

Why was this so good?

Remember, this task was one of the hardest!

How much did developers look up?

35 Webpages, a lot or little?

Discussion

• What happened? Was this the result you were expecting?

• How can we explain the disparity between usability and secure code?

• Any obvious and problematic limitations in the study?

• Time limit issues?

• Tests for success?

Now that we have the problem (insecure code comes from SO), what would you do with that information? How would you set up a follow-up study? • How much of the code is out there, static analysis on app code • Code reviews to help developers develop secure code • Flag insecure code on stack overflow

How much Stack Overflow code actually make it into Android applications?

• How do security related snippets in Stack Overflow proliferate?

• Actually find all relevant Stack Overflow post for Android and analyze those for security.

IEEE S&P’17

What do you expect to see?

• A lot of copy pasting / direct usage • Time crunch -> a lot of different variety of snippets

Processing Pipeline

Filtering

Classification

Program Dependency

Graph

Extraction

Clone Detection

Security Code Snippets Found Online

• Cryptography • Java Cryptography Architecture • Java Cryptography Extension

• Secure network communication • Java Secure Socket Extension • Java Generic Security Service • Simple Authentication and Security

Layer

• Reference Libraries (w/usability) • Keyczar • jaysprt

• Public Key Infrastructure • X.509 and Certificate Revocation List

in java.security.cert • Java certificate path API, PKCS#11,

OCSP

• Authentication and Access Control • Java Authentication and

Authorization Service

• Apache TLS/SSL • HttpClient

Snippets broken down by libraryNotice anything interesting?

Some Examples

Classification of Secure vs. Insecure

Insecure Snippets in Applications What’s up with games?

What’s up with business?

High Level Findings

• 15% of survey apps contain at least one insecure snippet • Top snippets can be found in 180K (13.8%) of apps • 183K apps were related to TLS (14%) • 21K related to symmetric cryptography (1.4%)

• 500K applications contain a secure snippet (38.82%)

Discussion

• Reflections from your own work flow? • How do you use stack overflow?

• How easily can these snippets be fixed? • What is the responsibility of Stack Overflow?

• What follow up work would you consider?

What’s other work you would do to help developers develop secure code? • Test first development: does this impact code security?

• “gut feeling”: helpful to write code to test, if it passes: secure! • Might increase the time initially • How common might this be? ! probably not, pushed in certain places

69

What we did today!

• Developers are not the enemy either! • Secure development ecosystem • Why do developers make mistakes? • Developers support forums

• How can we make them more secure

What’s next?

• Other Security Professionals • Hackers • Network admins

70