Embed Size (px)
Citation preview
Securing Search Data in the CloudSameer Maggon
Measured SearchHarry OchiaiHitachi Solutions
New York Enterprise Cloud Meetup
Jan 25, 2017
2
Agenda
• About the Speakers • About Measured Search & Hitachi Solutions • What is Apache Solr? • Where is Apache Solr used? • How Search Data is stored • Data Security Challenge in the Cloud • Protecting Confidential Search Data • Challenges of Encrypted Search Index • Encryption Solution • Demo • Q&A
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
3
About the Speakers
Harry Ochiai
• Senior Business Development Manager of Hitachi Solutions• Worked on networking, cyber security, and storage• Focus on cloud encryption solutions since 2013• New Yorker
Sameer Maggon
• Founder / Technologist at Measured Search• Been working in Open Source Search since 2001 (Lucene/Solr/Elastic)• USC Engineering Alumni• Works and Lives in Los Angeles, CA
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
4
About Hitachi Solutions
Global IT Solutions Company
• A Hitachi Company• Japan(HQ), North America, Europe, China, India and Southeast Asia• 12,000 Employees
Leading security solution provider in Japan
• Innovator and leading provider of encryption technology for over 20 years • HIBUN: 40% market share in the endpoint encryption segment in Japan• Launched new security solution Credeon globally in 2013
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
5
About Measured SearchMeasured Search® enables companies to elevate the experience of Search based applications faster and with more confidence.
Managed Services & Support
SearchStax® Platform as a Service
On-Demand Expertise & Consulting
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
6
by Measured SearchSearchStax®
SearchStax® Solr Cloud Manager
SearchStax® Pulse
SearchStax® Analytics
Comprehensive Solr Monitoring & Alerting with service level reporting to proactively manage your clusters.
Realtime feedback & user insights to help optimize your Search Experience
Easiest way to run & manage Solr in the cloud - saves time, money and reduces risk.
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
7
What is Apache Solr?
Solr is the popular, blazing-fast, open source enterprise search platform built on Apache
Lucene™
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
8
Where is Apache Solr used?
Government
eCommerce
Education
Life Sciences
Entertainment
HealthcareFinancial Services
High Tech
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
9
Where is Apache Solr used?
findin
g tick
ets
finding job
finding restaurant/services
Enterprise Search
Media Search
Retail Customer Search
Fraud Analytics Publishing
RecruitingTravelResearch
Business Intelligence
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
10
Search IndexSearch platforms maintains internal indices of terms and properties of each indexed document in plaintext.
Plain Search Index Encrypted Search Index© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
11
Data Security Challenges in the Cloud
Solr / Search Cluster Backups
Threats
Managed Service Provider (MSP)Rogue EmployeeManaged Service Provider Rogue Employee Hacker Accidental Data Access
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
12
Challenges of Encrypted Search Index
Challenges
• To search through encrypted data, data must be decrypted • Decryption slows down the process• Encryption limits usability
Goals
• Maintain encrypted state without sacrificing security level• Encrypt using your own key• Maintain search performance and usability• Protect against unauthorized users and rogue system administrators • Regulatory compliance
Solution
• Searchable Encryption technology
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
13
Protecting Confidential Search Data
Solr/Lucene
text Keyword
result
value
Solr/Lucene
text Keyword
result
File System Encryption
value
Solr/Lucene
text Keyword
result
value
Simple Encryption
Solr/Lucene
text Keyword
result
value
SearchableEncryption
text Keyword
result
value
Searchable Encryption
Storage Storage Storage Storage Storage
Client
ServerApp
ServerOS
No Encryption OS Encryption Simple Encryption with Solr Plugin
Client-Side Searchable Encryption
Searchable Encryption with Solr Plugin
Low Security High Security
No Security Decryption at storage layerX Difficult to separate key
Decrypt first and matchX Very slowX plaintext in memory
Match first and decryptO Key separationO High Performance(1)
O Semantically Secure(2)
X plaintext in memory
Client Client Client Client Client
Decryption at client-sideO Key separationO High Performance(1)
O Semantically Secure(2)
O no plaintext on server
Solr/Lucene
(1) Use of Symmetric Key(2) Probabilistic Encryption Scheme
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
14
Search Encryption 101
Searchable Encryption is an encryption technology for searching data in an encrypted state.
• Fully Homomorphic Encryption• Homomorphic Encryption• Functional Encryption• Hitachi Searchable Encryption
Practical
CKA
CPA
Tokenization
103 10610010-3
HitachiSearchable Encryption
FunctionalEncryption
HomomorphicEncryption
Fully HomomorphicEncryption
Secu
rity
Performance (Search / sec)
EncryptedPlain
CKA: Chosen Keyword AttackCPA: Chosen Phrase Attack
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
15
Search Encryption 101
Encryption Schemes
• Deterministic
• Constant value
• Vulnerable to statistical attacks
• Probabilistic
• Random value
• Semantically secure
Encryption Key Exchange
• Symmetric
• Asymmetric / PKI
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
16
Solution: Searchable Encryption with Solr Plugin
Hitachi Credeon Secure Full-Text Search
• Searchable Encryption plugin for Apache Solr and Elasticsearch• Probabilistic Encryption Scheme• 128 bit randomization• AES 256, FIPS 140-2
• Symmetric Key• Real-time search (15%+ overhead)
• Key Management System, Java KeyStore
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
17
Solution: Client Side Searchable Encryption
Credeon Secure Document Solution for SharePoint Online
• Client-side encryption for search index and data• Searchable encryption on Solr• Search Engine and Key Management are independent of Microsoft
Search Server
SharePoint Server
Key ManagementServer
Search Engine
SharePoint Online
Client PC
1. Get a key
2. Index the file contents and encrypt index
4. Upload encrypted file
4. Upload encrypted index
3. Encrypt file
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
18
Demo: Securing Solr Search in the CloudSearchStax with Credeon
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
19
Q&A
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
20
Contact Info
Sameer Maggon
https://www.measuredsearch.com
Harry Ochiai
https://psg.hitachi-solutions.com/credeon/overview
© Hitachi Solutions America, Ltd. and Measured Search, Inc. 2017, All rights reserved.
