Couchbase

Lorraine JUG - 19-JUN-2013

Thursday, June 20, 13


Send a mail to:[email protected]

LorraineJUG in the subject...

Join the French Couchbase Meetuph3p://www.meetup.com/Couchbase-‐France/


mailto:[email protected]


http://www.meetup.com/Couchbase-France/



Introduc)on to NoSQL with Couchbase

Tugdual Grall

Technical Evangelist



About me...

• Tugdual “Tug” Grall Couchbase

• Technical Evangelist

eXo

• CTO

Oracle

• Developer/Product Manager

Mainly Java/SOA Developer in consul)ng firms

• Web

@tgrall

hIp://blog.grallandco.com

tgrall

NantesJUG co-‐founder

Pet Project :

hIp://www.resultri.com


http://blog.grallandco.com

http://blog.grallandco.com

http://www.resultri.com

http://www.resultri.com


Agenda

• Why NoSQL ?

• NoSQL Landscape

• Document Design

• Use Cases

• Couchbase Server 2.0



$30B Database Market Being Disrupted

<50%?

2027

95%RelaVonal

Technology

2012

All new database growth will be NoSQL

RelaVonal Technology



NoSQLTechnology

Other



Three Macro Trends Driving DisrupVon

Big Data SaaS/Cloud CompuVngBig Users



Smartphone UsersHours Spent Online

35Billion Hours

1Billion

+

Global Online PopulaVon

2Billion

+



Big DataHigh Data Variety and Velocity

Source: IDC 2011 Digital Universe Study (hIp://www.emc.com/collateral/demos/microsites/emc-‐digital-‐universe-‐2011/index.htm)

Trillions of G

igabytes (ZeI

abytes)

0

0.50

1.00

1.50

2.00

2000 2006 2011

Unstructured and Semi-‐Structured Data

Structured Data

Text, Log Files, Click Streams, Blogs, Tweets, Audio, Video, etc.

More Flexible Data Model Required



RDBMS are not Enough?



Growth is the New Reality

• Instagram gained nearly 1 million users overnight when then expanded to Android



Draw Something Viral Growth

191715131197533/12826242220181614121082/6

Draw Something by OMGPOPDaily Ac)ve Users (millions)

21

2

4

6

8

10

12

14

16



RDBMS is good for many thing, but hard to scale

RDBMS Scales UpGet a bigger, more complex server

Users

ApplicaVon Scales OutJust add more commodity web servers

Users

System CostApplica)on Performance

RelaVonal Database

Web/App Server Tier


Won’t scale beyond this point

How do you take this growth?



Web/App Server Tier

Memcached Tier

MySQL Tier

Scaling out RDBMS

• Run Many SQL Servers

• Data could be sharded

Done by the applica)on code

• Caching for faster response Vme



Scaling out flafens the cost and performance curves

NoSQL Database Scales OutCost and performance mirrors app Ver

Users

NoSQL Distributed Data Store

Web/App Server Tier

ApplicaVon Scales OutJust add more commodity web servers

Users


Applica)on Performance System Cost

NoSQL Technology Scales Out



DynamoOctober 2007

CassandraAugust 2008

BigtableNovember 2006

VoldemortFebruary 2009

Very few organizaVons want to (fewer can) build and maintain database sojware technology.But every organizaVon building interacVve web applicaVons needs this technology.

A New Technology?

• Building new database to answer the following requirements No schema required before inser)ng data

No schema change required to change data format

Auto-‐sharding without applica)on par)cipa)on

Distributed queries

Integrated main memory caching

Data synchroniza)on ( mul)-‐datacenter)



Lack of flexibility/rigid schemas

Inability to scale out data Performance challenges Cost All of these Other

49%

35%

29%

16%12% 11%

What Is Biggest Data Management Problem Driving Use of NoSQL in Coming Year?

Source: Couchbase Survey, December 2011, n = 1351.



NoSQL CatalogKey-‐Value

Memcached

Membase

Redis

Data Structure Document Column Graph

MongoDB

Couchbase Cassandra

Cache

(mem

ory on

ly)

Database

(mem

ory/disk) Neo4j

HBase InfiniteGraph

Coherence



OperaVonal vs. AnalyVc Databases

ClouderaHortonworks

Mapr

CouchbaseMongoDB

CassandraHbase

Analy9cDatabases

Get insights from data

Real-‐9me, Interac9ve Databases

Fast access to data

NoSQL



Hadoop



Use CasesKey Value • Session Management

• User Profile/Preferences• Shopping Cart

Document • Event Logging• Content Management • Web AnalyFcs• E-‐Commerce ApplicaFon

Columns • Event Logging• Content Management• Counters

Graph • Connected Data / Social Networks• RouFng, Dispatch• RecommendaFons based on Social Graph



Data Models



RelaVonal vs Document Data Model

RelaVonal data model Document data modelCollec)on of complex documents witharbitrary, nested data formats and

varying “record” format.

Highly-‐structured table organiza)on with rigidly-‐defined data formats and

record structure.

C1 C2 C3 C4

JSONJSON

JSON

{

}



The RelaVonal Approach to Storing Data

Rela)onal databases were not designed with clusters in mind, which is why people have cast around for an alterna)ve. Storing aggregates as fundamental units makes a lot of sense for running on a cluster.

hIp://mar)nfowler.com/bliki/AggregateOrientedDatabase.html


http://martinfowler.com/bliki/AggregateOrientedDatabase.html

http://martinfowler.com/bliki/AggregateOrientedDatabase.html


Document Database by Comparison

o::1001{uid: “ji22jd”,customer: “Ann”,line_items: [

{ sku: 0321293533, quan: 2, unit_price: 48.0 },{ sku: 0321601912, quan: 1, unit_price: 39.0 },{ sku: 0131495054, quan: 1, unit_price: 51.0 }

],payment: { type: “Amex”, expiry: “04/2001”,

last5: 12345 }}

Easy to distribute dataMakes sense to applicaFon programmers



Objects Serialized to JSON and Back

User Object

string uid

string firstname

string lastname

int age

array favorite_colors

string email

u::[email protected]{“uid”: 123456,“firstname”: “John”,“lastname”: “Smith”,“age”: 22,“favorite_colors”: [“blue”, “black”],“email”: “[email protected]”}

User Objectstring uid

string firstname

string lastname

int age

array favorite_colors

string email

u::[email protected]{“uid”: 123456,“firstname”: “john”,“lastname”: “smith”,“age”: 22,“favorite_colors”: [“blue”, “black”],“email”: “[email protected]”}

set()

get()









Use Cases



Common Use Cases

Social Gaming• Couchbase stores player and game data

• Examples customers include: Zynga

• Tapjoy, Ubisor, Tencent

Mobile Apps• Couchbase stores user info and app content

• Examples customers include: Kobo, Play)ka

Ad TargeVng• Couchbase stores user informa)on for fast access

• Examples customers include: AOL, Mediamind, Convertro

Session store• Couchbase Server as a key-‐value store

• Examples customers include: Concur, Sabre

User Profile Store• Couchbase Server as a key-‐value store

• Examples customers include: Tunewiki

High availability cache• Couchbase Server used as a cache )er replacement

• Examples customers include: Orbitz

Content & Metadata Store

• Couchbase document store with Elas)c Search

• Examples customers include: McGraw Hill

3rd party data aggregaVon • Couchbase stores social media and data feeds

• Examples customers include: Sambacloud



QuesVons ? Concerns ?

• Data DuplicaVon?

• Schema ?

• TransacVon ?

• DBA ?

• Backup/Restore ?

• ...?



Couchbase ServerNoSQL Document Database



Easy Scalability Consistent High Performance

Always On 24x365

Grow cluster without applica)on changes, without down)me with a single click

Consistent sub-‐millisecond read and write response )mes with consistent high throughput

No down)me for sorware upgrades, hardware maintenance, etc.

Flexible Data Model

JSON document model with no fixed schema.

Couchbase Server



Open Source Project



Couchbase Handles Real World Scale



Heartbeat

Process m

onito

r

Glob

al singleton supe

rviso

r

Confi

gura)o

n manager

on each node

Rebalance orchestrator

Nod

e he

alth m

onito

r

one per cluster

vBucket state and

replica)

on m

anager

hSp

REST m

anagem

ent A

PI/W

eb UI

HTTP8091

Erlang port mapper4369

Distributed Erlang21100 -‐ 21199

Erlang/OTP

storage interface

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

Memcached

New Persistence Layer

8092Query API

Que

ry Engine

Data Manager Cluster Manager

Couchbase Server Architecture



Heartbeat

Process m

onito

r

Glob

al singleton supe

rviso

r

Confi

gura)o

n manager

on each node

Rebalance orchestrator

Nod

e he

alth m

onito

r

one per cluster

vBucket state and

replica)

on m

anager

hSp

REST m

anagem

ent A

PI/W

eb UI

HTTP8091

Erlang port mapper4369

Distributed Erlang21100 -‐ 21199

Erlang/OTP

storage interface

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

Memcached

New Persistence Layer

8092Query API

Que

ry Engine

Couchbase Server Architecture



Couchbase Opera)ons



33 2

Single node -‐ Couchbase Write Opera)on

Managed Cache

Disk Que

ue

Disk

Replica)on Queue

App Server

Couchbase Server Node

Doc 1Doc 1

Doc 1

To other node



COUCHBASE SERVER CLUSTER

Basic Opera)on

• Docs distributed evenly across servers

• Each server stores both ac9ve and replica docsOnly one server ac)ve at a )me

• Client library provides app with simple interface to database

• Cluster map provides map to which server doc is onApp never needs to know

• App reads, writes, updates docs

•Mul9ple app servers can access same document at same 9me

User Configured Replica Count = 1

READ/WRITE/UPDATE

ACTIVE

SERVER 1

ACTIVE

Doc 4

Doc 7

Doc

Doc

Doc

SERVER 2

Doc 8

ACTIVE

Doc 1

Doc 2

Doc

Doc

Doc

REPLICA

Doc 4

Doc 1

Doc 8

Doc

Doc

Doc

REPLICA

Doc 6

Doc 3

Doc 2

Doc

Doc

Doc

REPLICA

Doc 7

Doc 9

Doc 5

Doc

Doc

Doc

SERVER 3

Doc 6

Doc 5

Doc 2

Doc

Doc

DocDoc 9

APP SERVER 1

COUCHBASE Client LibraryCLUSTER MAP

COUCHBASE Client LibraryCLUSTER MAP

APP SERVER 2



Client SDK

www.couchbase.com/develop

Clojure

Python

Ruby

libcouchbase

Go


http://www.couchbase.com/develop

http://www.couchbase.com/develop


Demo Time



Cross Data ReplicaVon

US DATA CENTER EUROPE DATA CENTER ASIA DATA CENTER

Replica9on Replica9on

Replica9on

• Data close to users

• MulVple locaVons for disaster recovery

• Independently managed clusters serving local data



www.couchbase.com/download

Couchbase Server



Q & A

MeetuphIp://meetup.com/Couchbase-‐France/

Contacts@tgrall / [email protected]


http://meetup.com/Couchbase-France/

http://meetup.com/Couchbase-France/




Thank [email protected]

@tgrall

Join the French Couchbase Meetuph3p://www.meetup.com/Couchbase-‐France/




