Engineering Privacy Engineering

..

Engineering Privacy Engineering

Dr. Ian OliverSecurity ResearchNokia Networks

16 April 2015

1 © Nokia Solutions and Networks

A confession...

2 © Nokia Solutions and Networks

I am an engineer

3 © Nokia Solutions and Networks

This Talk

An appreciation about how an engineer conceptualises a system engineering problem

or

How do we bring engineers and privacy professionals together?

or

How do we make privacy engineering a discipline?

4 © Nokia Solutions and Networks

Question

You have a data breach - of any kind - whose fault was it?

..1 the privacy officer?

..2 the privacy policy itself?

..3 the engineers for not complying?

5 © Nokia Solutions and Networks

Question

You have a data breach - of any kind - whose fault was it?

..1 the privacy officer?

..2 the privacy policy itself?

..3 the engineers for not complying?

5 © Nokia Solutions and Networks

Question

You have a data breach - of any kind - whose fault was it?

..1 the privacy officer?

..2 the privacy policy itself?

..3 the engineers for not complying?

5 © Nokia Solutions and Networks

Question

You have a data breach - of any kind - whose fault was it?

..1 the privacy officer?

..2 the privacy policy itself?

..3 the engineers for not complying? ← HINT!

6 © Nokia Solutions and Networks

Question - Why?

Was your data breach due to:

..1 A failure in security? (typical RCA result)

..2 Bad programming?

..3 Bad design?

..4 Wrongly worded privacy policy?

..5 Miscommunication?

7 © Nokia Solutions and Networks

Speaking with Engineers

..1 Conversation 1• Privacy Officer: “Does your system collect PII?”• Engineer: “No!”

..2 Conversation 2• Privacy Officer: “Does your system collect GPS coordinates?”• Engineer: “Maybe...”

..3 Conversation 3• Privacy Officer: “What is that struct{float x, float y} thing?”• Engineer: “OK, you got me!”

8 © Nokia Solutions and Networks

Speaking with Engineers

..1 Conversation 1• Privacy Officer: “Does your system collect PII?”• Engineer: “No!”

..2 Conversation 2• Privacy Officer: “Does your system collect GPS coordinates?”• Engineer: “Maybe...”

..3 Conversation 3• Privacy Officer: “What is that struct{float x, float y} thing?”• Engineer: “OK, you got me!”

8 © Nokia Solutions and Networks

Speaking with Engineers

..1 Conversation 1• Privacy Officer: “Does your system collect PII?”• Engineer: “No!”

..2 Conversation 2• Privacy Officer: “Does your system collect GPS coordinates?”• Engineer: “Maybe...”

..3 Conversation 3• Privacy Officer: “What is that struct{float x, float y} thing?”• Engineer: “OK, you got me!”

8 © Nokia Solutions and Networks

Complexity• DNT

• 0 or 1

• Cookies• Context, Usage and Type vs Bytes

• Cloud• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics

• Notice and Consent• Policies

• Legalese vs Mathematics• POLICY ∈ P

{Type× Usage× Purpose× Provenance× . . .

}− ∅

• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .

9 © Nokia Solutions and Networks

Complexity• DNT

• 0 or 1• Cookies

• Context, Usage and Type vs Bytes

• Cloud• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics

• Notice and Consent• Policies

• Legalese vs Mathematics• POLICY ∈ P

{Type× Usage× Purpose× Provenance× . . .

}− ∅

• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .

9 © Nokia Solutions and Networks

Complexity• DNT

• 0 or 1• Cookies

• Context, Usage and Type vs Bytes• Cloud

• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics

• Notice and Consent• Policies

• Legalese vs Mathematics• POLICY ∈ P

{Type× Usage× Purpose× Provenance× . . .

}− ∅

• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .

9 © Nokia Solutions and Networks

Complexity• DNT

• 0 or 1• Cookies

• Context, Usage and Type vs Bytes• Cloud

• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics

• Notice and Consent

• Policies• Legalese vs Mathematics• POLICY ∈ P

{Type× Usage× Purpose× Provenance× . . .

}− ∅

• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .

9 © Nokia Solutions and Networks

Complexity• DNT

• 0 or 1• Cookies

• Context, Usage and Type vs Bytes• Cloud

• Alan Turing and his Turing Machines• Encryption and Key Management• Deletion Semantics

• Notice and Consent• Policies

• Legalese vs Mathematics• POLICY ∈ P

{Type× Usage× Purpose× Provenance× . . .

}− ∅

• TY PE =def ({⊤,⊥, F in,Hlt, Id, . . . },≤, . . .})• SY STEM |= POLICY• . . .

9 © Nokia Solutions and Networks

Towards a Solution

• Terminology and Semantics• Modelling and Context• Requirements• Risk• Compliance and Auditing

10 © Nokia Solutions and Networks

The Semantics of PII

11 © Nokia Solutions and Networks

Understanding PII/Personal Data

12 © Nokia Solutions and Networks

Terminology• Terminological or Ontological Structures

• Security classification• Information type (versus machine type or PII/personal data)• Provenance, Jurisdiction• Usage, Purpose• Controller, Processor and Data Subject• Identity and Authority, Notice and Consent• Requirements and Risk

• System Architecture, as a data flow• Data Processing and Storage, Environment and Breaches• Data Subject• Data Transformation• Architectural Partitioning and System Boundaries• Boundary Crossings

• Inference Rules and Calculi• ⋄

(x : MachineAddress ⊢ x : Location

)• Policy Calculation, Refinement and Compliance Checking

13 © Nokia Solutions and Networks

Terminology• Terminological or Ontological Structures

• Security classification• Information type (versus machine type or PII/personal data)• Provenance, Jurisdiction• Usage, Purpose• Controller, Processor and Data Subject• Identity and Authority, Notice and Consent• Requirements and Risk

• System Architecture, as a data flow• Data Processing and Storage, Environment and Breaches• Data Subject• Data Transformation• Architectural Partitioning and System Boundaries• Boundary Crossings

• Inference Rules and Calculi• ⋄

(x : MachineAddress ⊢ x : Location

)• Policy Calculation, Refinement and Compliance Checking

13 © Nokia Solutions and Networks

Terminology• Terminological or Ontological Structures

• Security classification• Information type (versus machine type or PII/personal data)• Provenance, Jurisdiction• Usage, Purpose• Controller, Processor and Data Subject• Identity and Authority, Notice and Consent• Requirements and Risk

• System Architecture, as a data flow• Data Processing and Storage, Environment and Breaches• Data Subject• Data Transformation• Architectural Partitioning and System Boundaries• Boundary Crossings

• Inference Rules and Calculi• ⋄

(x : MachineAddress ⊢ x : Location

)• Policy Calculation, Refinement and Compliance Checking

13 © Nokia Solutions and Networks

Modelling

The purpose of any model is to explore and answer questions about a system

or

So you really want to talk to an engineer?

14 © Nokia Solutions and Networks

Modelling

15 © Nokia Solutions and Networks

Notices and Consent

16 © Nokia Solutions and Networks

Requirement Aspect

17 © Nokia Solutions and Networks

Requirement Structure

18 © Nokia Solutions and Networks

Risk

after Solove, Anton-Earp, et al

19 © Nokia Solutions and Networks

Analysing The Model

• Let C be the set of components of a system,. . .

• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .• Let S be a system with the signature (C, I),. . .

• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the

whole system• Why?!

• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}

• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk

20 © Nokia Solutions and Networks

Analysing The Model

• Let C be the set of components of a system,. . .• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .

• Let S be a system with the signature (C, I),. . .

• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the

whole system• Why?!

• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}

• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk

20 © Nokia Solutions and Networks

Analysing The Model

• Let C be the set of components of a system,. . .• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .• Let S be a system with the signature (C, I),. . .

• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the

whole system• Why?!

• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}

• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk

20 © Nokia Solutions and Networks

Analysing The Model

• Let C be the set of components of a system,. . .• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .• Let S be a system with the signature (C, I),. . .

• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the

whole system• Why?!

• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}

• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk

20 © Nokia Solutions and Networks

Analysing The Model

• Let C be the set of components of a system,. . .• Let I be the set of interconnections {(c1, c2)}, c1, c2 ∈ C,. . .• Let S be a system with the signature (C, I),. . .

• Audit(c : C,…)→ B ∪ {⊥}, c ∈ S• Unfortunately doesn’t always produce the correct answers, even when C refers to the

whole system• Why?!

• CalcPolicy(ds : DataSubject)→ Policy =∑{(InfoType.ds.endpoints× Usage.ds.endpoints× . . .)}

• Can be compared against the ‘official’ privacy policy• Overconstrained system• Weakening implies additional risk

20 © Nokia Solutions and Networks

Metricisation

• Risk → (Cost× V alue), Cost ∝ V alue

• Requirements→ Risk

•(InformationType× Usage× . . .

)→ Requirements⇔ POLICY

• Policy ⊑ Architecture ⊑ Design ⊑ Code

21 © Nokia Solutions and Networks

Metricisation

• Risk → (Cost× V alue), Cost ∝ V alue

• Requirements→ Risk

•(InformationType× Usage× . . .

)→ Requirements⇔ POLICY

• Policy ⊑ Architecture ⊑ Design ⊑ Code

21 © Nokia Solutions and Networks

Metricisation

• Risk → (Cost× V alue), Cost ∝ V alue

• Requirements→ Risk

•(InformationType× Usage× . . .

)→ Requirements⇔ POLICY

• Policy ⊑ Architecture ⊑ Design ⊑ Code

21 © Nokia Solutions and Networks

Metricisation

• Risk → (Cost× V alue), Cost ∝ V alue

• Requirements→ Risk

•(InformationType× Usage× . . .

)→ Requirements⇔ POLICY

• Policy ⊑ Architecture ⊑ Design ⊑ Code

21 © Nokia Solutions and Networks

Quick Summary

• Terminology is the key - no more talking about PII in isolation

• Infomation Type, Usage, Purpose, Provenance etc

• Model your system

• Formalise requirements - polices are too abstract

• Risk management through metrics derived from the system model

• System model as a data flow becomes the binding and communication medium

22 © Nokia Solutions and Networks

Quick Summary

• Terminology is the key - no more talking about PII in isolation

• Infomation Type, Usage, Purpose, Provenance etc

• Model your system

• Formalise requirements - polices are too abstract

• Risk management through metrics derived from the system model

• System model as a data flow becomes the binding and communication medium

22 © Nokia Solutions and Networks

Quick Summary

• Terminology is the key - no more talking about PII in isolation

• Infomation Type, Usage, Purpose, Provenance etc

• Model your system

• Formalise requirements - polices are too abstract

• Risk management through metrics derived from the system model

• System model as a data flow becomes the binding and communication medium

22 © Nokia Solutions and Networks

Quick Summary

• Terminology is the key - no more talking about PII in isolation

• Infomation Type, Usage, Purpose, Provenance etc

• Model your system

• Formalise requirements - polices are too abstract

• Risk management through metrics derived from the system model

• System model as a data flow becomes the binding and communication medium

22 © Nokia Solutions and Networks

Quick Summary

• Terminology is the key - no more talking about PII in isolation

• Infomation Type, Usage, Purpose, Provenance etc

• Model your system

• Formalise requirements - polices are too abstract

• Risk management through metrics derived from the system model

• System model as a data flow becomes the binding and communication medium

22 © Nokia Solutions and Networks

Quick Summary

• Terminology is the key - no more talking about PII in isolation

• Infomation Type, Usage, Purpose, Provenance etc

• Model your system

• Formalise requirements - polices are too abstract

• Risk management through metrics derived from the system model

• System model as a data flow becomes the binding and communication medium

22 © Nokia Solutions and Networks

Putting it all Together

?

23 © Nokia Solutions and Networks

Putting it all Together

Culture

24 © Nokia Solutions and Networks

Lessons - Privacy as a Safety Critical Aspect

• Aviation• Medicine: Surgery, Anaesthesia• Civil Engineering

25 © Nokia Solutions and Networks

The End

Thankyou

26 © Nokia Solutions and Networks