Embed Size (px)
Citation preview
MongoDB and Spring DataChris Harris
Twitter : cj_harris5Email : [email protected]
© 2012 SpringOne 2GX. All rights reserved. Do not distribute without permission.
22
Solution Architect at 10gen (the company behind MongoDB)
- Previous Roles:
• EMEA Architect at SpringSource• EMEA Principal JBoss Consultant at RedHat
About Me
http://images.mirror.co.uk/upl/m4/mar2009/0/2/satnav-cliff-hanger-pic-ross-parry-image-3-213726463.jpg
http://www.simpleadhdexpert.com/wp-content/uploads/2009/08/frustration.jpg
http://daddytypes.com/archive/duct_tape_baby_mianro.jpg
Is there a better way?
Terminology
RDBMS MongoDBTable CollectionRow(s) JSON DocumentIndex IndexJoin Embedding & LinkingPartition ShardPartition Key Shard Key
As simple as possible, but no simpler
Depth of functionality
Sca
labi
lity
& P
erfo
rman
ceMemcached
Key / Value
RDBMS
MongoDB
Lets build a Spring Data MongoDB App!
Welcome to Dev News• User Story 1:
– As a user I should be able to create “News Articles”
What is a News Article?{ title: “What is new in 2.2 MongoDB”, date : new Date(), author : “Fred” text : “.......” }
Example 1 - Create
Welcome to MongoDB News Dev• User Story 1:
– As a user I should be able to create “News Articles”• User Story 2:
– As a user I should be able to find a “News Articles” by Name
Example 2 - Query
Example 3 - Repo
Welcome to MongoDB News Dev• User Story 1:
– As a user I should be able to create “News Articles”• User Story 2:
– As a user I should be able to find a “News Articles” by Name• User Story 3:
– As a user I should be able to comment on “News Articles”
What is a Comment?{ title: “What is new in 2.2 MongoDB”, date : new Date(), author : “Fred” text : “.......” tags : [“mongodb”, “nosql”] comments : [ {author : “chris”, text: “this is great” , date : new Date()}, {author : “tom”, text: “rock on”, date : new Date()} ] }
Example 4 - Comments
Welcome to MongoDB News Dev• User Story 1:
– As a user I should be able to create “News Articles”• User Story 2:
– As a user I should be able to find a “News Articles” by Name• User Story 3:
– As a user I should be able to comment on “News Articles”• User Story 4:
–As a user I want to know how many comments are on a “Articles”
What is a Comment?{ title: “What is new in 2.2 MongoDB”, .... comments : [ {author : “chris”, text: “this is great” , date : new Date()}, {author : “tom”, text: “rock on”, date : new Date()} ] , comments: count : 2}
Example 5 - Inc
“MongoDB lost my data!!”
Data: 125647383885969795743
Least durability - Don't use!
Driver MongoDB
apply in memory
write
Wait for Journal Sync - Same as RDBMSDriver MongoDB
apply in memory
write
j:trueWrite to journal
Replication
http://www.flickr.com/photos/10335017@N07/4570943043
Replica Set• Data Protection• Multiple copies of the data• Spread across Data Centers, AZs• High Availability• Automated Failover• Automated Recovery
Primary
Secondary
Secondary
Read
Write
Java App
Asynchronous Replication
Replica Sets
Replica Sets
Primary
Secondary
Secondary
Read
Write
Java App
Replica Sets
Primary
Primary
Secondary
Read
Write Automatic Election of new PrimaryJava App
Replica Sets
Recovering
Primary
Secondary
Read
Write New primary serves dataJava App
Replica Sets
Secondary
Primary
Secondary
Read
WriteJava App
Durability SummaryMemory Journal Secondary
Other Data Center
RDBMS
Default"Fire & Forget"
w=1
w=1j=true
w="majority"w=n
w="myTag"
Less More
Example 6 - Write Concern
Eventual Consistency
• http://www.flickr.com/photos/26095468@N04/3779692985
Understanding Eventual Consistency
Primary Secondary
v1
Thread #1
Insert
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
v1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Readv1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Readreads v1v1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
v2Update
Readreads v1v1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Update
Read
v2
reads v1
v2
v1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Update
Read
Read
reads v1
v2
v2
v1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Update
Read
Readreads v2
reads v1
v2
v2
v1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Update
Read
Read
Thread #2
v2
v2
v1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Update
Read
Read
Thread #2
v2
v2
v1
v1
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Update
Read
Read
Thread #2
v2
v2
v1
v1
v1 does not exist on
Secondary
Understanding Eventual Consistency
Primary SecondaryThread #1
Insert
Update
Read
Read
Thread #2
v2
v2
v1
v1
v1 does not exist on
Secondary
Reads v1 but Primary at v2
Read v2
Example 7 - Eventual Consistency
http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg
http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg
MongoDB Sharding• Automatic partitioning and management
• Range based
• Convert to sharded system with no downtime
• Fully consistent
How mongoDB Sharding works
• Range keys from -∞ to +∞ • Ranges are stored as "chunks"
-∞ +∞
> db.runCommand({addshard: "shard1"});> db.runCommand({shardCollection: "mydb.users", key: {age: 1}})
How mongoDB Sharding works
• Data in inserted• Ranges are split into more "chunks"
-∞ +∞
-∞ 40 41 +∞
> db.users.save({age: 40})
How mongoDB Sharding works
-∞ +∞
-∞ 40 41 +∞
41 50 51 +∞
61 +∞ 51 60
> db.users.save({age: 40})> db.users.save({age: 50})> db.users.save({age: 60})
-∞ 4041 50
61 +∞
51 60
> db.runCommand({addshard: "shard2"});> db.runCommand({addshard: "shard3"});
How mongoDB Sharding works
-∞ 4041 50
61 +∞
51 60
shard1
> db.runCommand({addshard: "shard2"});> db.runCommand({addshard: "shard3"});
How mongoDB Sharding works
-∞ 4041 50
61 +∞
51 60
shard1 shard2 shard3
> db.runCommand({addshard: "shard2"});> db.runCommand({addshard: "shard3"});
How mongoDB Sharding works
Architecture
C1
C2
C3
Config Serversmongos mongos
app app
mongod
Shard 1
mongod
mongod
mongod
Shard 2
mongod
mongod
Shard 4
mongod
Shard 3
mongod
mongod
Replica Set
mongod
mongod
mongod
Balancing
config
config
config
Shard 1 Shard 2 Shard 4Shard 3
mongosbalancer
5
9
1
6
10
2
7
11
3
8
12
4
13 14 15 16 17 18 19 20 21 22 23 24
ImbalanceData inserted all in a range
mongos
Balancing
config
config
config
Shard 1 Shard 2 Shard 4Shard 3
mongosbalancer
5
9
1
6
10
2
7
11
3
8
12
4
13 14 15 16 17 18 19 20 21 22 23 24
Move chunk 1 to Shard 2
mongos
Balancing
config
config
config
Shard 1 Shard 2 Shard 4Shard 3
balancer
5
9
6
10
2
7
11
3
8
12
4
13 14 15 16 17 18 19 20 21 22 23 24
1
mongos
Balancing
config
config
config
Shard 1 Shard 2 Shard 4Shard 3
balancer
5
9
6
10
2
7
11
3
8
12
4
13 14 15 16 17 18 19 20 21 22 23 24
1
mongos
Balancing
config
config
config
Shard 1 Shard 2 Shard 4Shard 3
mongosbalancer
5
9
16
10
2
7
11
3
8
12
4
13 14 15 16 17 18 19 20 21 22 23 24
mongos
Example 8 - Sharding
• MongoDB 2.2– Improved concurrency–Tag-aware sharding–Aggregation framework–TTL collections
• Free online MongoDB training–Develop–Deploy–Classes start Oct. 2012
Questions
