@doanduyhai
Cassandra Drivers And Tools DuyHai DOAN, Technical Advocate
@doanduyhai
Agenda!
2
Drivers • architecture, policies, Java driver API
DevCenter (live coding demo!) Cassandra Unit (+ live coding demo!) Object Mapper module (+ live coding demo!) Achilles Object Mapper (+ live coding demo!)
Cassandra Drivers Architecture!
Architecture!Policies!
Java driver API!
@doanduyhai
Drivers list!
4
• Java • C# • Python • Node.js • Ruby (1.0.0.rc1) • C++ (beta) • ODBC (beta) • Clojure (community) • Go (community) • PHP (to be announced)
@doanduyhai
Connection pooling!
5
n2
n3
n4
Driver
Pool1
Pool2
Pool3
Client Thread1
Client Thread2
Client Thread3
@doanduyhai
Connection pooling!
6
n2
n3
n4
Driver
Pool1
Pool2
Pool3
Client Thread1
Client Thread2
Client Thread3
1
2 3
4 5
6
@doanduyhai
Request Pipelining!
7
Client Cassandra
@doanduyhai
Request Pipelining!
8
Client Cassandra
StreamID
StreamID
@doanduyhai
Nodes Discovery!
9
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Control Connection
@doanduyhai
Round Robin Load Balancing!
10
n2
n3
n4
n5
n6
n7
n8
n1 Client 1
2 3
4
@doanduyhai
DC Aware Load Balancing!
11
Client1 DC1
DC2 Client2
⤫
@doanduyhai
DC Aware Load Balancing!
12
Client1 DC1
DC2 Client2
⤫
@doanduyhai
Token Aware Load Balancing!
13
n2
n3
n4
n5
n6
n7
n8
n1 Client
1
2
3
⤫
@doanduyhai
Combining Load Balancing Policies!
14
Load Balancing Policy
Round Robin DC Aware Round Robin
Token Aware
extends
wraps
Default config
@doanduyhai
Automatic Failover!
15
n2
n3
n4
Driver
Pool1
Pool2
Pool3
Client Thread
⤫
1
2 3
4 5
6 7 8
@doanduyhai
Other policies!
16
Retry policy • write/read timeout • node unavailable Reconnection policy • constant schedule • exponential schedule
@doanduyhai
Statements!
17
Plain statement • convenient, one-off query • plain string ☞ parsing overhead
INSERT INTO user(login, name, age) VALUES(‘jdoe’, ‘John DOE’, 33)’;
@doanduyhai
Statements!
18
Prepared statements • avoid parsing overhead • query structure should be known ahead of time • bound values • named parameters
INSERT INTO user(login, name, age) VALUES(?, ?, ?)’;
INSERT INTO user(login, name, age) VALUES(:login, :name, :age)’;
@doanduyhai
Statements!
19
Parameterized statements • same as plain statement • pass bound values as bytes ☞ avoid ser/deser of values
INSERT INTO user(login, name, age) VALUES(?, ?, ?)’;
@doanduyhai
Java Driver!
20
Reference implementation Base on asynchronous Netty library Configurable policies Query tracing support Client-node compression & SSL
@doanduyhai
Maven dependency!
21
<dependency> <groupId>com.datastax.cassandra</groupId> <artifactId>cassandra-driver-core</artifactId> <version>2.1.3</version> </dependency>
depends on Netty, Guava, Metrics
Available on Maven Central
@doanduyhai
Connect and Write!
22
Cluster cluster = Cluster.builder() .addContactPoints("127.0.0.1", “another-host").build();
seed nodes (IP or DNS name)
Session session = cluster.connect("my_keyspace");
session.execute("INSERT INTO user (user_id, name, email) VALUES (12345, 'johndoe', '[email protected]’)");
@doanduyhai
Configuration!
23
Cluster cluster = Cluster.builder() .addContactPoints("127.0.0.1", “another-host") .withLoadBalancingPolicy(new DCAwareRoundRobinPolicy("DC1") .withRetryPolicy(DowngradingConsistencyRetryPolicy.INSTANCE) .withReconnectionPolicy(new ConstantReconnectionPolicy(1000)) .build();
@doanduyhai
Read!
24
ResultSet resultSet = session.execute("SELECT * FROM user");
List<Row> rows = resultSet.all();
for (Row row : rows) { Long userId = row.getLong("user_id"); String name = row.getString("name"); String email = row.getString("email"); }
Stateless ☞ thread-safe
@doanduyhai 25
Asynchronous Read!
ResultSetFuture future = session.executeAsync("SELECT * FROM user");
ResultSet resultSet = future.get(); //blocking call
List<Row> rows = resultSet.all();
for (Row row : rows) { Long userId = row.getLong("user_id"); String name = row.getString("name"); String email = row.getString("email"); }
@doanduyhai 26
Asynchronous Read with CallBack!
ResultSetFuture future = session.executeAsync("SELECT * FROM user");
future.addListener(new Runnable() { public void run() { // Process the results here } }, executor);
executor = Executors .newCachedThreadPool(); or
executor = Executors .sameThreadExecutor();
@doanduyhai 27
Query Builder!
Query query = QueryBuilder .select() .all() .from( "my_keyspace", "user") .where(eq("login", "jdoe"));
query.setConsistencyLevel(ConsistencyLevel.ONE);
ResultSet rs = session.execute(query);
@doanduyhai
Old manual paging!
28
Some time you need to fetch all table content Manual paging:
SELECT * FROM users WHERE token(login) >= token(<last_fetched_login>) LIMIT 100;
@doanduyhai
New automatic paging!
29
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Query
Page 1 + paging state 1
@doanduyhai
New automatic paging!
30
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Query
Page 2 + paging state 2
@doanduyhai
New automatic paging!
31
Paging state ≈ stateless cookie Resilient to node failure
@doanduyhai
Paging during node failure!
32
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Query
Page 1 + paging state 1
@doanduyhai
Paging during node failure!
33
n2
n3
n4
n5
n6
n7
n8
n1 Driver
Query
Page 2 + paging state 2
⤫
Q & R
! " !
Dev Center Demo
@doanduyhai
Cassandra Unit!
36
• start an embedded Cassandra server
• useful for unit testing
• mature project, created at August 5th 2011
• designed around Thrift & Hector
• propose a JUnit rule for CQL
Cassandra Unit Demo
@doanduyhai
Java Driver Object Mapper!
38
• simple mapper
• KISS
• annotations à-la JPA (but no JPA dependencies)
• templating system à-la Spring Data
@doanduyhai
Java Driver Object Mapper!
39
@Table(keyspace = "mapper_module", name = "users") public class User {
@PartitionKey private String login;
private String name; // getters and setters omitted... }
Mapping
@doanduyhai
Java Driver Object Mapper!
40
MappingManager manager = new MappingManager(session); Mapper mapper = manager.mapper(User.class);
User user = mapper.get("[email protected]");
mapper.saveAsync(new User("[email protected]"));
mapper.delete("[email protected]");
Usage
@doanduyhai
Java Driver Object Mapper!
41
@Accessor interface UserAccessor { @Query("SELECT * FROM users LIMIT :max") Result<User> firstNUsers(@Param("max") int limit); }
Accessors (SpringData template-like) definition
@doanduyhai
Java Driver Object Mapper!
42
Accessors usage
UserAccessor accessor = manager.createAccessor(UserAccessor.class); List<User> users = accessor.firstNUsers(10).all(); for (User user : users) { System.out.println( profile.getAddress().getZip() ); }
Java Driver Object Mapper Demo
@doanduyhai
Achilles!
44
Why ? • started in late 2012, when mapper module did not exists
• more involved and more features than the mapper module
• different annotations set (may converge)
Achilles Demo
@doanduyhai
Dirty Checking!
46
Dirty checking, why is it important ? • 1 user ≈ 8 mutable fields
• × n denormalizations = n update combinations
• and not even counting multiple fields updates …
@doanduyhai
Dirty Checking!
47
• Are you going to manually generate n prepared statements for all possible updates ?
• Or just use dynamic plain string statements and get some perf
penalty ?
@doanduyhai
Dirty Checking!
48
//No read-before-write ContactEntity proxy = manager.forUpdate(ContactEntity.class, contactId); proxy.setFirstName(…); proxy.setLastName(…); //type-safe updates proxy.setAddress(…);
manager.update(proxy);
@doanduyhai
Dirty Checking!
49
Proxy Setters interception
Empty Entity
DirtyMap
PrimaryKey
@doanduyhai
Dirty Checking!
50
• Dynamic statement generation
UPDATE contacts SET firstname=?, lastname=?,address=? WHERE contact_id=?
prepared statements are cached, of course
@doanduyhai
Main API!
51
manager.insert(entity) manager.update(entity) manager.remove(entity) manager.find(Entity.class, primaryKey)
@doanduyhai
Advanced Features!
52
Counter Batch mode Strategies (insert, naming) Options Asynchronous
@doanduyhai
Documentation!
53
Comprehensive Github WIKI Twitter-clone demo app (demo.achilles.io) Versioned documentation (HTML & PDF) JavaDoc
@doanduyhai
RoadMap!
54
C* 2.1 user defined types (UDT) Query-templates à-la Spring Data Reactive ? (RxJava) ElasticSearch integration (@olivierbourgain)
Q & R
! " !