Upload
others
View
12
Download
0
Embed Size (px)
Citation preview
10/7/2019
1
Governance for Artificial Intelligence/
Machine Learning
Akbar Siddiqui
Technical Director
Civil Liberties, Privacy, and Transparency Office
National Security Agency
1
2
10/7/2019
2
The NSA Mission
Signals Intelligence
The National Security Agency is responsible for:
Providing our nation’s policy
makers and military commands
with foreign intelligence to gain a
decisive advantage.
U.S. CybersecurityProtecting and defending
sensitive information systems
and networks critical to national
security and infrastructure.
What is AI/ML?
Unsupervised
Learning
MACHINE
LEARNING
Reinforcement
Learning
Supervised
Learning
Game AI
SkillsAcquisition
Learning Tasks
Robot Navigation
Real-time Decisions
Estimating Life Expectancy
PopulationGrowth Prediction
Market Forecasting
Weather Forecasting
Advertising Popularity Prediction
Classification
Regression
Diagnostics
Customer Retention
Image Classification
Identity Fraud DetectionMeaningful
Compression
Clustering
Big Data Visualization
Structure Discovery
Feature Elicitation
Recommender Systems
Targeted Marketing
Customer Segmentation
Dimensional
Reduction
Raw Data
Labeled Training Data
Experimental Parameters
Models
Output
3
4
10/7/2019
3
Governance in the Process
Purpose Collect Process Evaluate Retain Disseminate
Authority
Training
Guidance
Compliance Controls
Technical Safeguards
Governance in the Process
Purpose Collect Process Evaluate Retain Disseminate
Authority
Training
Guidance
Compliance Controls
Technical Safeguards
5
6
10/7/2019
4
Governance in the Process
Purpose Collect Process Evaluate Retain DisseminatePurposeTraining
DataDevelop Model
Apply to Data
Use Outputs
Feedback
Machine Learning Process Safeguards
Purpose
• Authorities, Ethical Use
• Explainable Purpose and Methods
Training Data
• Collect or generate training data and test data
• Documentation: Datasheets for Datasets
Develop Model
• Explainability
• Testing/Validation
• Confidence Levels
• Documentation: Model Cards
Apply to Data
• Use Limitation
• User Interpretation and Validation
Use Outputs
• Accountability
• Explainability and Redress
• Human control
• Confidence level follows outputs
Feedback
• Build user into workflow
• Check for adversarial techniques
7
8
10/7/2019
5
Defined Purpose and Use
• Governance Bodies
• Check for Authorities
• Check for Ethical Use (Principles)
• Explainable Purpose and Methods
Purpose
Training Data
• Collect or generate training data and test data• Data Selection, Feature Engineering, Labeling
• Issue: Collecting and maintaining “negative” examples
• Documentation: Datasheets for Datasets• Identify and document features, purpose, limitations, and
known issues
• biases (explicit and implicit)
Training DataPurpose
9
10
10/7/2019
6
Model Development
• Explainability
• Testing and Validation
• Check for bias in weights/methodology
• ID situations where model performs poorly/unreliably or is vulnerable to
adversarial techniques
• Confidence Level
• Documentation: Model Cards
Develop Model
Training DataPurpose
Stakeholders
ModelMission Review
Peer ReviewSenior Operations
Data Officer (SODO),
Senior Operations Analytics Officer (SOAO)
Sharing Equities
(Equities Review Board)
Mission Risk Acceptance
(Mission Element Owners)
Security
Business Intelligence Metrics
(ROI)
Labeled Data
Civil Liberties, Privacy,
and Transparency (CLPT)
Compliance for Dissemination (LPOC)
Legal
Deployment Compliance (AVG)
Chief Data Officer (CDO)
11
12
10/7/2019
7
Applying Models to Data
• Use Limitation
• User Interpretation and Validation
Develop Model
Training DataPurpose Apply to Data
Using Outputs
• Accountability
• Explainability and Redress
• Human control
• Confidence level follows outputs
Develop Model
Training DataPurpose Apply to Data Use Outputs
13
14
10/7/2019
8
Feedback
• Build user corrections into workflow
• Drift
• Biased weights
• Check for adversarial techniques
Develop Model
Training DataPurpose Apply to Data Use Outputs
Feedback
Q & A
15
16
10/7/2019
9
18
Mitigating Adversarial Machine Learning
*Popular Science: Fooling The Machine, The
Byzantine science of deceiving artificial intelligence; Dave Gershgorn (March 30, 2016)
Original outcome →
Tricked outcome →
Machine learning (ML) can be a solution to scalable defensive and offensive measures for cybersecurity. These can range from semi-automated decision support to fully-automated capabilities. However, ML models can be exploited in at least four ways. Adversaries can:
(a) poison training data used to train ML algorithms to degrade prediction quality, or redirect predictions, altogether;
(b) evade by manipulating runtime data to ensure ML models misclassify malicious behavior as benign;
(c) infer records into the training data; and
(d) reconstruct the ML model for further analysis and exploitation.
When ML models of varying qualities are integrated into an ensemble, an adversary can exploit weaknesses in individual models to coordinate a malicious effect in the overall system.
17
18