Lit Review Talk - Signal Processing and Machine Learning with Differential Privacy Algorithms and challenges for continuous data

Signal Processing and Data Privacy

Literature Review – Talk

By Kato Mivule

COSC891 Fall 2013

Signal Processing and Machine Learning with Differential Privacy

Algorithms and challenges for continuous data

Sarwate, A.D.; Chaudhuri, K., "Signal Processing and Machine Learning with Differential Privacy: Algorithms and Challenges for

Continuous Data," Signal Processing Magazine, IEEE , vol.30, no.5, pp.86,94, Sept. 2013

doi: 10.1109/MSP.2013.2259911

Bowie State University Department of Computer Science

Agenda

• Introduction.

• Privacy Definition Challenge.

• Differential Privacy Definition.

• Differential Privacy Challenges.

• Differential Privacy Applications in Signal Processing.

• Conclusion.



Introduction: Privacy Preserving Data Mining

• Maintaining the privacy of individuals is imperative.

• Individuals expect their data to be kept private despite willingness to share

such info.

• The Challenge is how extract knowledge from large scale data while

maintaining privacy.



Introduction: Privacy Definition

• Privacy definition is problematic.

• There are different meaning for these words across different communities:

• Privacy

• Confidentiality

• Security

• “There is no real separation between individuals’ identity and their data—the

pattern of data associated with an individual is itself uniquely identifying.”

Sarwate and Chaudhuri (2013)



Differential Privacy Definition

• Cryptographically Motivated.

• Proposed by Cynthia Dwork (2006).

• Imposes confidentiality by returning perturbed query responses from

databases:

• 𝒇 𝒙 + 𝑳𝒂𝒑𝒍𝒂𝒄𝒆(𝟎, 𝒃)

• 𝒃 =∆𝒇

𝜺

• ∆𝒇 = 𝑴𝒂𝒙 𝒇 𝑫𝟏 − 𝒇 𝑫𝟐

• The end user of the database cannot know if a data item has been altered.

• An attacker cannot gain information about any data item in the database.



Differential Privacy Definition:

ε-differential privacy is satisfied if the results to a query run on database D1 and D2 should

probabilistically be similar, and meet the following condition:

𝑷[𝒒𝒏(𝑫𝟏)∈𝑹]

𝑷[𝒒𝒏 𝑫𝟐 ∈𝑹] ≤ 𝒆𝜺

Where D1 and D2 are the two databases

P is the probability of the perturbed query results D1 and D2.

qn() is the privacy granting procedure (perturbation).

qn(D1) is the privacy granting procedure on query results from database D1.

qn(D2) is the privacy granting procedure on query results from database D2.

R is the perturbed query results from the databases D1 and D2 respectively.

𝒆𝜺 is the exponential e epsilon value.



Differential Privacy Definition – Types

• Query-based Differential Privacy.

• Input perturbation based differential privacy – add Laplace noise to the data.

• Output perturbation – add Laplace noise to the query results.



Differential Privacy – Input and Output Perturbation



Image Source: Sarwate, A.D.; Chaudhuri, K., "Signal Processing and Machine Learning with Differential Privacy: Algorithms and Challenges for

Continuous Data," Signal Processing Magazine, IEEE , vol.30, no.5, pp.86,94, Sept. 2013 doi: 10.1109/MSP.2013.2259911

Differential Privacy Challenges

• There is a tension between data privacy and data utility.

• Evaluating the impact of privacy restrictions on utility.

• Data dimensionality.

• Trade-off points between data privacy and utility.



Differential Privacy Challenges – Utility Quantification

• Mean Squared Error – for statistical estimation.

• Computed as follows: Original and privatized datasets 𝑋 − 𝑋′

• (𝑥𝑖𝑗−𝑥𝑖𝑗

′ )2𝑛𝑖=1

𝑑𝑗=1

𝑛𝑑

• Expected loss – for classification.

• Comparative analysis – quantify various differential privacy algorithms.

• Where n is the number of records and d the number of variables in the table.



Differential Privacy Challenges – Limitations

• Time Series and Filtering problems.

• Understanding the fundamental limits for continuous data may shed some

light on which signal processing tasks are possible under differential privacy.

• The optimal differential privacy parameter adjustment for acceptable levels of

data utility – adjustment of the 𝜺 value.

• A single data set could be used in multiple computations – challenge is how

to keep the same differential privacy across the board.



Differential Privacy – Signal Processing Applications

• Apply differential piracy in signal processing problems.

• Apply signal processing to the differential privacy problem of data utility.

• Time Series and Filtering problems.

• Differential privacy of a query sequence in Fourier domain and use homomorphic

encryption for distributed noise addition.

• Kalman filtering applied to a differentially private time series.

• Kalman filtering used on aggregated signals after input and output perturbation.



Differential Privacy – Applications

• Integrate differential privacy in signal processing.

• Use signal processing to enhance data utility and privacy.

• Use signal processing to filter out unneeded noise.

• Research areas – applying differential privacy in signal processing for:

• Image processing

• Network information systems

• Cryptographic approaches

• Social networks



Conclusion

• A general overview of the differential privacy is given.

• The paper focused on differential privacy and the applications in signal processing.

• The paper suggests research areas for applying differential privacy in signal

processing.

• Implementation is left to the readers.



References • Sarwate, A.D.; Chaudhuri, K., "Signal Processing and Machine Learning with Differential Privacy: Algorithms

and Challenges for Continuous Data," Signal Processing Magazine, IEEE , vol.30, no.5, pp.86,94, Sept. 2013

doi: 10.1109/MSP.2013.2259911

• Mivule, K; Turner, C; and Y. Ji, S., “Towards A Differential Privacy and Utility Preserving Machine Learning

Classifier,” in Procedia Computer Science, 2012, vol. 12, pp. 176–181.

• Mivule, Kato, "Utilizing Noise Addition for Data Privacy, an Overview", Proceedings of the International

Conference on Information and Knowledge Engineering (IKE 2012), Pages 65-71, Las Vegas, NV, USA.



Technology

Lit Review Talk - Signal Processing and Machine Learning with Differential Privacy Algorithms and challenges for continuous data