Sharing Sensitive Data Securely

Managing Security – Sharing DataTed Dunning

Agenda• Two kinds of security failure

– Buried treasure• But what could go wrong?

– Horror stories• Sharing into controlled environments

– Views, masking and fine-grained control• Sharing without sharing

– When masking is not sufficient• Summary

Locked Up Tight – The Cheapside Hoard• Between 1640 and 1666 somebody hid

a cache of jewels under the floorof 30-32 Cheapside Road

• They never came back for them …

• The hoard was found by workmen in 1910• Did the owners forget where they were?• Why didn’t their heirs or partners recover them?

The Other Kind of Security Failure• Security can fail when there is a leak

– Enigma decryption– Retail data compromise– Klaus Fuchs

• Security also fails when data is not shared– AKA siloing– The many threads of 9/11– The Cheapside hoard– Invisible technological opportunity cost

Netflix• Shared anonymized data• Huge boost in state of the art for some kinds of

recommendations

• Anonymization shown to be weak barrier

• Lawsuit, security clamp-down everywhere

Reference Data Attack

The Moral

• If there is something to correlate, anonymization may fail

• When I say “may”, you should read “will”

NY Cab• Hack license and medallion number hashed using MD-5• No correlation data to work with

• But cab (medallion) numbers have only a few forms

• So we can generate hashes for all 20 million (or so) medallions

So What?• What correlations are there?

• NYC medallions are public information anyway

• Taxis operate in the public realm

So What?

Paparrazo + Timestamp + Taxi = Who and Where

See http://gawker.com/the-public-nyc-taxicab-database-that-accidentally-track-1646724546http://research.neustar.biz/2014/09/15/riding-with-the-stars-passenger-privacy-in-the-nyc-taxicab-dataset/

Extended Moral• Correlations are more common than we thought

• Masking PII is not sufficient for public datasets

• Theoretically, no solution is possible• Pragmatically, never bet against cleverness

• Must change the game

Alternative Strategies

Public disclosure + Simple masking

Key Elements of Masking• Opaque or format preserving?• Random or reversible or one-way?

• Simple omission?

• Right to be forgotten?

Releasing Public Data• Why?

– Required– For research– For support

• How?– New technology based on KPI-preserving random data

• Three use cases

Secure Development is Hard

Outside collaborators are outside the security perimeter

They can’t see the data and they can’t tune new algorithms to fit reality

How To Make Realistic Data

Parametric Simulation

Parametric matching of failure signatures allows emulation of complex data properties

Matching on KPI’s and failure modes guarantees practical fidelity

The Method• Pick realistic and important KPI’s and failure measures

– False positive rate– Scale invariant score distribution– Internal performance metrics (# of candidates searched, similar)

• Build emulation roughly based on real system• Tune data spec to match KPI’s using real models• Export data spec to alternative models• Re-tune data spec to match on alternative models

Example #1 – Query failure• Performance index is query failure with particular stack signature

• Tuning knobs include– Table sizes– Data distributions– (potentially) field value realism– (potentially) field cross correlations

The Original ConversationThem UsHive broke, fix it.

The Original ConversationThem UsHive broke, fix it. Sure! Can I see the data?

No. OK. Can I see the stack trace?

No. Can I log in to the system?

No. What do you want me to do?

Fix it.

The Broken Query

A Simpler Example Schema

A Simpler Example[ {"name":"customer_id", "class":"id"}, {"name":"name", "class":"name", "type":"first_last"}, {"name":"street", "class":"address"}, {"class":"flatten", "value": { "class":"zip", "fields":"city,state,zip"}}]

[ {"name":"sales_id", "class":"id"}, {"name":"customer_id", "class":"foreign-key", "size":"$customers"}, {"name":"time_id", "class":"foreign-key", "size":"$times"}, {"name":"store_id", "class":"foreign-key", "size":"$stores"}, {"name":"item_id", "class":"foreign-key", "size":"$items"}, {"name":"quantity", "class":"int", "skew":0.5}, {"name":"unit_price", "class":"gamma", "dof":1, "scale":10}, {"name":"discount", "class":"uniform", "min":0, "max":20}, {"name":"exact_time", "class":"event", "start": "2014-01-01", "format":"yyyy-MM-dd HH:mm:ss", "rate": "10/d"}]

Data Flow

Sample Data

customer_id,name,street,zip,city,state0,"Mark Long","8578 Pied River Flats","02630","BARNSTABLE","MA"1,"Chris Lanier","90018 Lost Treasure Corner","06083","ENFIELD","CT"2,"Bryant Brandon","30712 Bright Shadow Stroll","93922","CARMEL","CA"3,"Norman Horn","66871 Dewy Bird Shoal","59727","DIVIDE","MT"4,"Carmen Nowell","6053 Velvet Barn Glen","29329","CONVERSE","SC"

Results• We had to match size, number of records, rough levels of skew

• Bug was in query planner– For particular values of relative table size, planner messed up

• Once we had the fault, we could slim down the tables– Final example had 3 tables, 1000 records in larges

Common Point of Compromise• Scenario:

– Merchant 0 is compromised, leaks account data during compromise– Fraud committed elsewhere during exploit– High background level of fraud– Limited detection rate for exploits

• Goal:– Find merchant 0

• Meta-goal:– Screen algorithms for this task without leaking sensitive data

Simulation Setup

Simulation Strategy• For each consumer

– Pick consumer parameters such as transaction rate, preferences– Generate transactions until end of sim-time

• If merchant 0 during compromise time, possibly mark as compromised• For all transactions, possible mark as fraud, probability depends on history• Merchants are selected using hierarchical Pittman-Yor

• Restate data– Flatten transaction streams– Sort by time

• Tunables– Compromise probability, transaction rates, background fraud, detection

probability

Performance Indicators to Match• User and merchant population• Transaction count/consumer• Merchant propensity skew• Level of detected fraud• Spectrum of meta-model scores

Real bad guys

Results• We matched general mechanism, rough transaction rates

• Model was tuned on synthetic data, tested on live data

• We found real bad guys on the first try

Summary• Security can fail through too much and

too little access• Sharing widely can have significant benefits and

substantial risks• New levels of control available for masking and filtering

of big data via Drill views• Synthetic data with KPI matching provides sharing of

realistic data without risk

Questions

Thank You@mapr maprtech

tdunning@mapr.comtdunning@apache.org

Ted Dunning, Chief Application Architect

MapRTechnologies

maprtech

mapr-technologies

Sharing Sensitive Data Securely

Software

Recommendations for Implementing Open Science · Sharing Sensitive Data with Confidence: The DataTags System, Technology Science OpenDP: A New Project for Sensitive Data A community

Load Sharing Module - Precision · PDF fileManual 26011 Load Sharing Module Woodward 5 Chapter 2. Electrostatic Discharge Awareness All electronic equipment is static-sensitive,

Improving how people and companies do business with UC San · Securely manages sensitive registration information (like Social Security numbers or bank account numbers) through encryption

Secure Multi-owner data Sharing for dynamic group using ... · e cloud computing is secure data sharing in dynamic cloud computing. It implies that users in the group can securely

Enabling data driven outcomes: Safe havens · data •Handling sensitive data Talks •National ethics policy for managing and sharing data •Enabling access to sensitive data at

Secure audited email file sharing: SECURELY SHARE FILES FROM ALMOST ANY COMPANY OR CLOUD DATA SOURCE

SHARING SENSITIVE DATA WITH CONFIDENCE: THE DATATAGS SYSTEMdatatags.org/files/datatags/files/tip-tots-datatags.pdf · 2016-08-08 · SHARING SENSITIVE DATA WITH CONFIDENCE: THE DATATAGS

Monitoring and protecting sensitive data in Office 365€¦ · Web viewMicrosoft IT created a solution to manage the risk of sharing sensitive data, while still promoting collaboration

MANAGING SENSITIVE DATA FOR SHARING − THE UK DATA ARCHIVE EXPERIENCE ……………………………………………………

Publishing and sharing sensitive data 28 June

12-Sep-15 Virtual Private Network. Why the need To transmit files securely without disclosing sensitive information to others in the Internet

Protection, Privacy And Peace Of Mind For Sharing Sensitive … · For Sharing Sensitive Enterprise Information EXTERNAL FILE SHARING SECURITY & GOVERNANCE PLATFORM. 2 | | sales@accellion.com

Sharing Sensitive Information without Compromising Data · Title: Sharing Sensitive Information without Compromising Data The Federal government is working to create a central repository

KOMSCO JK62 V1.0 Certification Report...Secure operation of application instances through Java Card Platform 3.0.4, Securely clearing and destroying of sensitive information (such

PRIVACY TOOLS FOR SHARING RESEARCH DATA · DataTags and their respective policies Sweeney L, Crosas M, Bar-Sinai M. Sharing Sensitive Data with Conﬁdence: The Datatags System. Technology

Sharing Sensitive Health Data in a Federated Data ......Sharing Sensitive Health Data in a Federated Data Consortium Model 5 Establish and sustain trust 1 Generating trust is more

Feature Sheet · Sharing large files securely can be a challenging and complicated process that often results in sensitive data being risked in the interest of expediency. With CipherPost

Sharing Securely SIMposium 2010

Making better use of data to identify customers in vulnerable … · 2018-11-01 · Customer trust is crucial – sharing sensitive information means maintaining data securely is

Safeguard eHealth with Legislation · Safeguard eHealth ... Safeguard eHR Sharing with Legislation. ... “Will my health data be securely kept in the Electronic Health Record Sharing