©2013 DataStax Confidential. Do not distribute without consent.
@PatrickMcFadin
Patrick McFadinChief Evangelist for Apache Cassandra
Keynote: The State of Cassandra
1
Five Years of Cassandra
Jun-09 Mar-10 Jan-11 Nov-11 Sep-12 Jul-13
0.1 0.3 0.6 0.7 1.0 1.2...
2.0
DSE
Jul-08
•Massively scalable •High performance • Reliable/Available
Core values Cassandra HBase Redis MySQL
CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int);!!CREATE INDEX ON users(state);!SELECT * FROM users WHERE state=‘Texas’ AND birth_date > 1950;
New Core Value
•Massively scalable •High performance •Reliable/Available •Productivity + ease of use
CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int);
CREATE TABLE users_addresses ( user_id uuid REFERENCES users, email text);!SELECT *FROM users NATURAL JOIN users_addresses;X
Collections
UPDATE usersSET email_addresses = email_addresses + {‘[email protected]’, ‘[email protected]’};
CREATE TABLE users ( id uuid PRIMARY KEY, name text, state text, birth_date int, email_addresses set<text>);
Collections
Cassandra Community
Your community
We are growing!
Great projects!
•Storage backend for Graphite •Replacement of whisper •Plugs into carbon •Real users: Love it!
Cyanite
https://github.com/pyr/cyanite
Great projects!
The Kiji Project
http://www.kiji.org/2014/02/11/working-on-cassandra-integration-with-datastax/
•Real-time application framework •Product and content recommendation systems •Risk analysis and fraud monitoring •Customer profile and segmentation applications
Get Involved!
•Talk at a meet up •5 minute interview? •Stack overflow •Mailing list •Apache Jira
Cassandra 2.0
Reads in a cluster
Client Coordinator
40% busy
90% busy
30% busy
A failure
Client Coordinator
40% busy
90% busy
30% busyXtimeout
Rapid read protection
Client Coordinator
40% busy
90% busy
30% busyXsuccess
Rapid Read Protection
NONE
Latency (mid-compaction)
Cold data
10,000 req/s 5,000 req/s
4,000 req/s 10 req/s
Cold data
10,000 req/s 5,000 req/s
4,000 req/s 10 req/s
Cold data compaction
10 req/s
10,000 req/s
Cold Data Compaction•Disabled in 2.0
•Default in 2.1
•Configuration per Table:
ALTER TABLE mykeyspace.mytable WITH compaction = {'class': 'SizeTieredCompactionStrategy', 'cold_reads_to_omit': 0.05};
http://www.datastax.com/dev/blog/optimizations-around-cold-sstables
Cassandra 2.1
User defined typesCREATE TYPE address ( street text, city text, zip_code int, phones set<text>)!CREATE TABLE users ( id uuid PRIMARY KEY, name text, addresses map<text, address>)!SELECT id, name, addresses.city, addresses.phones FROM users;! id | name | addresses.city | addresses.phones--------------------+----------------+-------------------------- 63bf691f | jbellis | Austin | {'512-4567', '512-9999'}
Collection indexingCREATE TABLE songs ( id uuid PRIMARY KEY, artist text, album text, title text, data blob, tags set<text>);!CREATE INDEX song_tags_idx ON songs(tags);!SELECT * FROM songs WHERE tags CONTAINS 'blues';! id | album | artist | tags | title----------+---------------+-------------------+-----------------------+------------------ 5027b27e | Country Blues | Lightnin' Hopkins | {'acoustic', 'blues'} | Worrying My Mind!!
(UDT indexing?)
Counters++•simpler implementation, no more edge cases
•possible to properly repair now
•significantly less garbage and internode traffic generated
•better performance for 99% of uses •RF>1, replicate_on_write=true
•topology changes not leading to data loss (#4071)
•commitlog now 100% safe to replay (#4417)
•Internal format overhaul still coming in 3.0 (#6506)
What hasn’t changed•same API
•same average throughput
•same restrictions on mixing counter and non-counter columns
•same restrictions on mixing counter and non-counter updates
•same restrictions on counter deletes
•same retry limitations
Writes (low contention)
Writes (high contention)
Data directories (2.0)/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-CompressionInfo.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Data.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Filter.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Index.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Statistics.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-Summary.db/var/lib/cassandra/data/foo/bar/foo-bar-jb-1-TOC.txt
Data directories (2.1)/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-CompressionInfo.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Data.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Filter.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Index.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Statistics.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Summary.db/var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-TOC.txt
Inefficient bloom filters
+
= ?
+
=
Inefficient bloom filters
+
=
Inefficient bloom filters
Inefficient bloom filters
HyperLogLog applied
HLL and compaction
HLL and compaction
More-efficient repair
More-efficient repair
More-efficient repair
Implications for LCS (and STCS)
Performance•Memtable memory use cut by 40% (85% by beta2)
• larger sstables, less compaction
•~50% better write performance
•Full results after beta2
The new query cache
The new row cacheCREATE TABLE notifications ( target_user text, notification_id timeuuid, source_id uuid, source_type text, activity text, PRIMARY KEY (target_user, notification_id))WITH CLUSTERING ORDER BY (notification_id DESC) AND caching = 'rows_only' AND rows_per_partition_to_cache = '3';!
The new row cachetarget_user notification_id source_id source_type activity
nick e1bd2bcb- d972b679- photo jbellis liked
nick 321998c- d972b679- photo rbranson commented
nick ea1c5d35- 88a049d5- user mbulman created account
nick 5321998c- 64613f27- photo jbellis commented
nick 07581439- 076eab7e- user thobbs created account
rbranson 1c34467a- f04e309f- user jbellis created account
The new row cachetarget_user notification_id source_id source_type activity
nick e1bd2bcb- d972b679- photo jbellis liked
nick 321998c- d972b679- photo rbranson commented
nick ea1c5d35- 88a049d5- user mbulman created account
nick 5321998c- 64613f27- photo jbellis commented
nick 07581439- 076eab7e- user thobbs created account
rbranson 1c34467a- f04e309f- user jbellis created account
2.1 roadmap•Beta1 today •Beta2 beginning of April •RC end of March •Final release end of April?
Questions?