View
337
Download
2
Category
Preview:
Citation preview
©2015 Couchbase Inc. 2
Agenda
What’s New in Couchbase Server 4.0 Querying with N1QL Indexing Multi-Dimensional Scaling ForestDB: Next-Generation Storage Engine Security
©2015 Couchbase Inc. 3
And There’s More …
Feature Description
XDCR Filtering Key-based filtering on XDCR streams addressing reference data & geo-based replication use cases.
Bloom Filters Significantly improved latency for low % working set (DGM) situations
Memory Defragmentation
Use memory according to one’s needs
Improved Cluster Robustness
Rogue view queries no longer cause cluster manager outage
Breakpad Compact and readable crash logs for memcached
SDK First class SDK support across Couchbase Server 4.0 features
SUSE, Oracle Enterprise Linux, CentOS 7, RHEL 7, Ubuntu 14.04
Expand the supported production platforms
©2015 Couchbase Inc. 7
N1QL is Expressive: E-Commerce Examples
SELECT product.name, SUM(items.count) AS unitsSold FROM purchases UNNEST purchases.lineItems AS items JOIN product ON KEYS items.product GROUP BY product ORDER BY unitsSold DESC LIMIT 10
SELECT SUBSTR(purchases.purchasedAt, 0, 7) AS month,
ROUND(SUM(product.unitPrice * items.count)/1000000, 3) revenueMillion
FROM purchases UNNEST purchases.lineItems AS items
JOIN product ON KEYS items.product
GROUP BY SUBSTR(purchases.purchasedAt, 0, 7)
ORDER BY month
©2015 Couchbase Inc. 8
N1QL is Expressive: Image Processing Example
SELECT
m.image AS gallery_image,
s.image AS source_image, s.x, s.y,
m.r AS gallery_r, m.g AS gallery_g, m.b AS gallery_b,
s.r AS source_r, s.g AS source_g, s.b AS source_b,
FROM SourcePixels s
JOIN GalleryMeans m ON KEY s.rgb
Source: http://www.metanautix.com/tr01
Learn more about N1QL @ query.couchbase.com/tutorial
©2015 Couchbase Inc. 9
Sessions on N1QL Introducing N1QL: Query Without Compromise
May 3 @ 1:00 pm – 1:45 pm, Architecture TrackMay 4 @ 11:15 am – 12:00 pm, Architecture Track
N1QL and SDK Support for Java, .NET, and Node.jsMay 3 @ 4:30 pm - 5:15 pm, Developer Track
Migrating MySQL Queries to N1QLMay 4 @ 1:00 am – 1:45 pm, Developer Track
Big Data Query Landscape: N1QL and MoreMay 4 @ 4:30 pm – 5:15 pm, Architecture Track
Interactive Data Analytics with Couchbase N1QL – NielsenMay 3 @ 1:45 pm - 2:30 pm, Developer Track
Agility with Data Modeling at DIRECTV Using N1QL in Couchbase with Node.js - DIRECTVMay 4 @ 3:45 pm - 4:30 pm, Developer Track
©2015 Couchbase Inc. 10
N1QL is Declarative: What vs. How
You specify WHATCouchbase Server figures out HOW
©2015 Couchbase Inc. 11
N1QL is Declarative: Top 10 Selling Products
SELECT product.name, SUM(items.count) AS unitsSold FROM purchases UNNEST purchases.lineItems AS items JOIN product ON KEYS items.product GROUP BY product ORDER BY unitsSold DESC LIMIT 10
WHAT
HOW
©2015 Couchbase Inc. 12
Sessions on N1QL – Deep Dive Under the Hood: Couchbase Server 4.0 Architecture
May 3 @ 11:30 pm - 12:15 pm, Architecture Track Deep Dive into N1QL: Power Features and Internals in Couchbase 4.0
May 3 @ 2:30 pm - 3:15 pm, Architecture Track Tuning Query Performance with N1QL in Couchbase Server 4.0
May 4 @ 1:45 pm – 2:30 pm, Architecture Track Enterprise Architect’s Perspectives of Couchbase with N1QL
May 4 @ 5:15 pm – 6:00 pm, Architecture Track
©2015 Couchbase Inc. 13
N1QL Connects To Your Enterprise Ecosystem
Standards-based drivers
Integrations, partnerships
ODBC / JDBC
App
CB Node
ODBC / JDBC
ETL
ODBC / JDBC
BI
ODBC / JDBC
Visualization
CB Node CB Node
©2015 Couchbase Inc. 14
Sessions on N1QL – Partners Operational Analytics with Looker and N1QL- Looker
May 3 @ 1:00 pm – 1:45 pm, Big Data Track Enterprise Data Integration: Using Informatica ETL with Couchbase –
InformaticaMay 3 @ 3:45 pm – 4:30 pm, Big Data Track
Visual Analytics with Tableau & Couchbase – Tableau SoftwareMay 3 @ 4:30 pm - 5:15 pm, Architecture Track
Open Connectivity: BI, Integration, and Applications on Couchbase using ODBC and JDBC - SimbaMay 4 @ 1:45 pm – 2:30 pm, Big Data Track
Metanautix Quest and Couchbase: Scalable Analytics Across NoSQL, RBDMS, Hadoop – MetanautixMay 4 @ 3:45 pm – 4:30 pm, Big Data Track
©2015 Couchbase Inc. 16
Why Do We Need Indexes?
Indexes are used to quickly look up objects
meeting user-specified criteria without having to search over every
object in the database.
©2015 Couchbase Inc. 17
What Are the Different Types of Indexes?
Primary IndexesSecondary Indexes
Spatial IndexesText Indexes
©2015 Couchbase Inc. 18
Index Types … Let’s Use the Library Analogy
Source: http://www.montel.com/en/markets/library/
©2015 Couchbase Inc. 19
Secondary Indexes
Local secondary indexes – Map/Reduce views in 3.0 Co-located with data Higher write performance Lower read performance: scatter-gather Scaling bottleneck with a high number of indexes or data
nodes
Global secondary indexes (GSI) – New in 4.0 Higher query performance Async writes to a large number of global indexes Independently scaled and partitioned Isolated from Key-Value operations
©2015 Couchbase Inc. 20
Sessions on Global Secondary Indexes Introducing Global Secondary Indexes: The New High Performance
Indexer with Couchbase 4.0May 3 @ 1:45 pm - 2:30 pm, Architecture Track
Deep Dive into Global Secondary Indexing Architecture in Couchbase Server 4.0May 3 @ 3:45 pm - 4:30 pm, Architecture Track
©2015 Couchbase Inc. 21
Box on Points Box Intersects Multi Dimensions Geo + Others
Spatial Indexes
Experimental in 3.0, now GA in 4.0 Performance and stability improvements
GeoJSON output Bounding-box and range queries on multiple
dimensions
©2015 Couchbase Inc. 22
Session on Spatial Indexes Introducing Spatial Views for Location Aware Applications
with Couchbase Server 4.0May 4 @ 4:30 pm – 5:15 pm, Architecture Track
©2015 Couchbase Inc. 23
Text Indexes (Developer Preview)
…
wise
…
engineer…
…
…
…, akay1980, …
…, akay1980, …
engineers
engineer Exact
Match
Inverted Index
Apply the sametext analysisat search timethat we usedat index time.
©2015 Couchbase Inc. 24
Session on Text Indexes A Sneak Peek of CBFT, Full Text Search for Couchbase
May 3 @ 5:15 pm –6:00 pm, Architecture Track
©2015 Couchbase Inc. 26
Workload and Scaling
Horizontal scaling Partitions a dataset onto one or more homogenous nodes Each node runs the same mixed workloads Re-partition dataset with additional hardware capacity
Node 8
Index Service
Couchbase Cluster
Query Service
Data Service
Node 1
©2015 Couchbase Inc. 27
Multi-Dimensional Scaling
Isolated Service for minimized interference Independent “zones” for query, index, and data
services
Minimize indexing and query overhead on core key-value operations.
Index Service
Couchbase Cluster
Query Service Data Service
Node 1 Node 8
©2015 Couchbase Inc. 28
Multi-Dimensional Scaling
Independent scalability for the best computational capacity per service
Couchbase Cluster
node1 Node 8 Node 9
Data Service
Index ServiceQuery Service
More storage for indexing: scale up index service nodes.
More cores for query processing:scale up query service nodes.
Node 1
©2015 Couchbase Inc. 29
Session on Multi-Dimensional Scaling Multi-Dimensional Scaling: A New Architecture for Scaling
Big Data ApplicationsMay 4 @ 10:30 am – 11:15 am, Architecture Track
©2015 Couchbase Inc. 31
Common Storage Structures – B+-Tree
31
Reads Writes Engines
B+-Tree Good read performance if the fan-out is high (“short” tree) for small fixed-length keys. Read performance degrades for variable-length keys.
Update-In-Place results in random writes and bad write latency.
Append-Only file improves write performance but requires periodic compaction.
BerkeleyDB, InnoDB, Couchstore (append-only), WiredTiger B+
04/26
…
…
…Key
Value (or Pointer)
longer keys
©2015 Couchbase Inc. 32
Common Storage Structures – LSM-Tree
32
Reads Writes EnginesLSM-Tree
Reads may have to traverse multiple trees – typically worse than B+ tree.
WAL(Write-Ahead Log) improves writes within-memory trees that are appended to the end of the log.
LevelDB, RocksDB, Cassandra, WiredTiger LSM
…
In-memory
Sequential log
flush/merge merge
C1 tree C2 tree
merge
Capacity increases exponentially
©2015 Couchbase Inc. 33
Fast Storage – ForestDB
Fundamentally rethinking the requirements for fast storage Compact and efficient storage for a variety of data Unified engine that performs well for various workloads Unified engine that scales from small devices to large
servers Optimized for new SSD storage technology
33
ForestDB provides a compact index structure
that significantly reduces the disk I/Oand write amplification.
©2015 Couchbase Inc. 34
ForestDB
ForestDB Evaluations: LevelDB, RocksDB
34
Throughput
UP TO
FASTER6X
Future Proof
FITTED
FORSSD
Efficiency
UP TO
5X
COMPACT
©2015 Couchbase Inc. 35
Trie (prefix tree) for which the nodes are B+-Trees HB+-Trie was originally presented at ACM SIGMOD 2011 Programming
Contest, by Jung-Sang Ahn who works at Couchbase(http://db.csail.mit.edu/sigmod11contest/sigmod_2011_contest_poster_jungsang_ahn.pdf)
HB+-Trie (Hierarchical B+-Trie)
©2015 Couchbase Inc. 36
Sessions on ForestDB Next Generation Storage Engine: ForestDB
May 3 @ 4:30 pm – 5:15 pm, Architecture Track
Under the hood ForestDB: Performance on SSDs and File SystemsMay 4 @ 2:30 pm – 3:15 pm, Architecture Track
Intel: Next Generation SSDs and ForestDB: Next Generation Storage EngineMay 4 @ 2:30 pm – 3:15 pm, Architecture Track
©2015 Couchbase Inc. 38
Previously… In 2.2 In 2.5 In 3.0 New in 4.0
SASL AuthN with Bucket Passwords
Admin User
Secure Build Platform
Read-Only User
Easy Admin Password
Reset
Non-Root User
Deployments
Secure Communication for XDCR
Encrypted Client- Server
Communication
Encrypted Admin Access
Access Log
Data-at-Rest Encryption
• Simplified compliance with admin auditing
• External identity management for admins using LDAP
Couchbase Security Features
In a fewslides ..
38
©2015 Couchbase Inc. 39
External Identity Management Using LDAP
39
Centralized identity management Define multiple read-only admins and full-admins Centralized security policy management for
admin accounts for stronger passwords, password rotation, and auto lockouts
Individual accountability; simplified compliance Define UIDs in LDAP, and map UIDs to the read-
only/full admin role in Couchbase Comprehensive audit trails with LDAP UIDs in
audit records
©2015 Couchbase Inc. 40
Admin Auditing in Couchbase
40
Rich audit events Over 25+ different, detailed admin audit events Auditing for tools including backup
Configurable auditing Configurable file target Support for time-based log rotation and audit
filtering
Easy integration JSON format allows for easy integration with
downstream systems using flume, logstash, and syslogd
©2015 Couchbase Inc. 41
Auditing a successful login
41
{ "timestamp":"2015-02-20T08:48:49.408-08:00", "id":8192, "name":"login success", "description":"Successful login to couchbase cluster", "role":"admin", "real_userid": { "source":"ns_server", "user":"bjones” }, "sessionid":"0fd0b5305d1561ca2b10f9d795819b2e", "remote":{"ip":"172.23.107.165", "port":59383}}
WHEN
WHO
WHAT
HOW
©2015 Couchbase Inc. 42
Sessions on Security Security Features in Couchbase Server
May 3 @ 3:45 pm – 4:30 pm, Operations Track
Securing Your Couchbase Server Deployment Using VormetricMay 4 @ 10:30 pm –11:15 pm, Operations Track
Querying with N1QLIndexing
Multi-Dimensional ScalingForestDB: Next-Generation
Storage EngineSecurity
Recommended