Upload
eric-evans
View
1.062
Download
0
Tags:
Embed Size (px)
DESCRIPTION
OpenNMS User Conference Europe presentation on using Apache Cassandra and Newts for time-series data storage.
Citation preview
It’s Not You...Ending a 15 year relationship with RRD
OpenNMS User Conference EuropeApril 11, 2014
Eric [email protected]
@jericevans
Five Stages of Grief
1. Denial2. Anger3. Bargaining4. Depression5. Acceptance
RRDTool
● Round robin database● First released 1999● Time-series storage● File-based● Constant-size● Automatic, amortized aggregation
Graph All The Things
Consider
● 2 5+ IOPs per update (read-modify-write)!● 1 RRD per data source (storeByGroup=false)● 100,000s of data sources, 1,000s IOPS● 1,000,000s of data sources, 10,000s IOPS● 15,000 RPM SAS drive, ~175-200 IOPS
Also
● Not everything is a graph● Inflexible● Incremental backups impractical● ...
Bottleneck!?
● We need to be collecting even more!● We need to be collecting more frequently!● The Internet of Things is upon us!!
But can’t we … ?
● Serialize RRD writes?● Cache?● Distribute the RRDs?● … ?
How about:● Distributed, decoupled architecture● High throughput● Horizontally scalable● Pluggable, extensible graphing● Facilitate new forms of analytics● More?
Starting Over Can Be Fun!
Observation #1
We collect and write a great deal; We read (graph) relatively little.
We are read-optimized.
Observation #2
Grouping samples that are accessed together is an easy optimization.
Project: NewtsGoals:● Stand-alone time-series data store● High-throughput● Horizontally scalable● Grouped metric storage/retrieval● Late-aggregating
Cassandra
● Apache top-level project● Distributed database● Tunable consistency● NoSQL MoSQL
Partitioning
A
B
C
Key: Apple
...
Placement
A
B
C
Key: Apple
...
Replication
A
B
C
Key: Apple
...
Cap Theorem
Consistency
Availability
Partition tolerance
Consistency
A
?
?
W=1
Consistency
A
B
C
R=3
Consistency
A
B
?
W=2
Consistency
?
B
C
R=2
R+W > N
Properties
● Symmetrical● Linearly scalable● Redundant● Highly available
SSTables
Writes
Commitlog
Memtable
SSTable
DiskMemory
Properties
● Optimized for write throughput● Sorted on disk● Perfect for time series!
Gist
● Samples stored as-is.● Samples can be retrieved as-is.● Measurements are aggregations calculated
from samples (at time of query).
Samples vs. Measurementssam·ple/ˈsampəl/noun
1. a small part or quantity intended to show what the whole is like."investigations involved analyzing samples of handwriting"synonyms: representative, illustrative, selected, specimen, test, trial, typical
meas·ure·ment/ˈmeZHərmənt/noun
1. the action of measuring something."accurate measurement is essential"synonyms: quantification, computation, calculation, mensuration
Sample{ ‘timestamp’: 1395278097, ‘resource’ : ‘localhost.eth0’, ‘name’ : ‘ifInOctets’, ‘type’ : ‘COUNTER’, ‘value’ : 457283782231}
NewtsCREATE TABLE newts.samples ( resource text, collected_at timestamp, metric_name text, metric_type text, value blob, attributes map<text, text>, PRIMARY KEY(resource, collected_at, metric_name));
Behind the scenes...
KSAT (1970-02-10 12:42:00,dewPoint,value): 0xc01a0000
(1970-02-10 12:42:00,maxTemp,value): 0x40280000
...
Ascending Order
Newts
resource | collected_at | metric_name | value
---------+---------------------+--------------+-----------
KSAT | 1970-02-10 12:42:00 | dewPoint | 0xc01a0000
KSAT | 1970-02-10 12:42:00 | maxTemp | 0x40280000
KSAT | 1970-02-10 12:42:00 | maxWindGust | 0x7ff80000
KSAT | 1970-02-10 12:42:00 | maxWindSpeed | 0x40180000
KSAT | 1970-02-10 12:42:00 | meanTemp | 0xbfe00000
POSTPOST /samples HTTP/1.1Host: example.comContent-Type: application/json
[ { ‘timestamp’: 1395278097, ‘resource’ : ‘localhost.eth0’, ‘name’ : ‘ifInOctets’, ‘type’ : ‘COUNTER’, ‘value’ : 457283782231 }, { ... },]
GET samplesGET /samples/localhost.eth0?start=900000000 HTTP/1.1
[ [ {“name”: “ifInOctets”, “timestamp”: 900000000, “type”: “COUNTER”, “value”: 12345678900}, {“name”: “ifOutOctets”, “timestamp”: 900000000, “type”: “COUNTER”, “value”: 87654321000} ], [ {“name”: “ifInOctets”, “timestamp”: 900000300, “type”: “COUNTER”, “value”: 23456789000}, {“name”: “ifOutOctets”, “timestamp”: 900000300, “type”: “COUNTER”, “value”: 98765432100} ]]
GET measurementsGET /measurements/octets/localhost.eth0?resolution=20m HTTP/1.1
[ [ {“name”: “ifInOctets”, “timestamp”: 900000000, “value”: 102400.00}, {“name”: “ifOutOctets”, “timestamp”: 900000000, “value”: 409600.00} ], [ {“name”: “ifInOctets”, “timestamp”: 900001200, “value”: 102400.00}, {“name”: “ifOutOctets”, “timestamp”: 900001200, “value”: 409600.00} ]]