39
Redis, 7 years and more @antirez Redis Labs

Redis Day Keynote Salvatore Sanfillipo Redis Labs

Embed Size (px)

Citation preview

Page 1: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Redis, 7 years and more

@antirez Redis Labs

Page 2: Redis Day Keynote Salvatore Sanfillipo Redis Labs

2009

Page 3: Redis Day Keynote Salvatore Sanfillipo Redis Labs

• EXPIRE

• Sets and Sorted Sets

• Non blocking replication

• AOF

Page 4: Redis Day Keynote Salvatore Sanfillipo Redis Labs

2010

Page 5: Redis Day Keynote Salvatore Sanfillipo Redis Labs

• Hash data type

• Virtual Memory project started

• Pub/Sub

• Redis Cluster project started

• VM abandoned, Diskstore project started (!)

Page 6: Redis Day Keynote Salvatore Sanfillipo Redis Labs

2011

Page 7: Redis Day Keynote Salvatore Sanfillipo Redis Labs

• More work to Redis Cluster (alpha at the time)

• IPv6

• Lua scripting

• Diskstore abandoned. No new attempts.

Page 8: Redis Day Keynote Salvatore Sanfillipo Redis Labs

2012

Page 9: Redis Day Keynote Salvatore Sanfillipo Redis Labs

• Bit operations

• Special encodings for small values

• Sentinel V1

• SCAN family commands

Page 10: Redis Day Keynote Salvatore Sanfillipo Redis Labs

2013

Page 11: Redis Day Keynote Salvatore Sanfillipo Redis Labs

• Keyspace events

• Replication PSYNC support

• CONFIG REWRITE

• Performance boosts

• Sentinel V2

Page 12: Redis Day Keynote Salvatore Sanfillipo Redis Labs

2014

Page 13: Redis Day Keynote Salvatore Sanfillipo Redis Labs

• Lexicographical ranges

• HyperLogLog

• Diskless replication

• New List type encoding introduced

Page 14: Redis Day Keynote Salvatore Sanfillipo Redis Labs

2015

Page 15: Redis Day Keynote Salvatore Sanfillipo Redis Labs

• I join RedisLabs !!!

• Redis 3.0 with Cluster released

• Geo indexing API

• Non blocking DEL and FLUSH commands

• Lua debugger

Page 16: Redis Day Keynote Salvatore Sanfillipo Redis Labs

2016

Page 17: Redis Day Keynote Salvatore Sanfillipo Redis Labs

• BITFIELD command

Page 18: Redis Day Keynote Salvatore Sanfillipo Redis Labs

What’s new in 3.2

Page 19: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Lua new replication modes

• Now you can write script with side effects, random commands, …

• Selective replication of certain commands to slaves.

Page 20: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Expires consistency

• Slave and Master now agree much better about what keys already expired.

• Expires are still up to the master, but the slave, for read operations, can show a sane behavior.

Page 21: Redis Day Keynote Salvatore Sanfillipo Redis Labs

RDB improvements

• RDB has now auxiliary informations inside.

• It’s faster to load since it resizes the hash table before loading the keys.

• So… no rehashing needed (is costly).

Page 22: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Geo commands

• 49 commits, 899 insertions, 1429 deletions.

• API simplified, JSON API removed entirely.

• An useless indirection layer removed. Geo indexing API taken from Ardb simplified & improved.

• Radius queries fixed to really report items in range.

• Much faster (may be yet a lot faster… work needed)

Page 23: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Quicklists• Idea initially conceived by Pieter & me, never

implemented.

• Later we saw Twitter implementing it, but just the minimum they needed.

• Finally Matt Stancliff provided a good implementation.

• Sun He and I refined the implementation and improved the on-disk serialization.

Page 24: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Quicklists

Page 25: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Improved ZADD• NX, XX, CH, INCR.

• Better capabilities via options.

• Newcomers can ignore them.

• Complexity is disclosed incrementally.

• Commands list does not becomes huge.

• What about Markov Chains?

Page 26: Redis Day Keynote Salvatore Sanfillipo Redis Labs

SPOP with count

• Ability to pop multiple items from set in a single call.

• Small change, but required some reworking of the replication code.

Page 27: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Lua debugger

• Writing complex scripts is no longer a nightmare.

Page 28: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Cluster improvements

• Rebalancing of the cluster.

• CLUSTER API improvements.

Page 29: Redis Day Keynote Salvatore Sanfillipo Redis Labs

BITFIELD

• Arrays of counters

• Overflow handling

Page 30: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Lazy freeing of objects

Page 31: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Lazyfree basics

• It means: O(1) DEL synchronous work.

• Plus: O(N) DEL asynchronous work.

• The result: DEL is no longer blocking.

• API: UNLINK and ASYNC option for FLUSHDB and FLUSHALL.

Page 32: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Shared objectsSet 1 Set 2 Output

buffer

Page 33: Redis Day Keynote Salvatore Sanfillipo Redis Labs

More stuff• Better SDS library.

• More useful and polished crash reports.

• CLIENT REPLY to suppress replies for bulk inserts.

• DEBUG RESTART and CRASH-AND-RECOVERY.

• HSTRLEN command.

• Too many to show them all.

Page 34: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Ideas for the future

Page 35: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Threaded I/O• Spawn N threads at startup.

• Each thread does multiplexing like now.

• Each client is pinned to a thread when connecting.

• We can finally scale write, read, multiplexing.

• Global Lock. Similar to memcached.

Page 36: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Threaded slow ops• Client is executing SUNION foo bar.

• Block it (like in blocked.c).

• Lock the key in a lock hash table: no mutex per key! Single mutex for the lock hash table.

• Perform operation, including sending the output in the client output buffer, in a different thread.

• Unblock the client.

Page 37: Redis Day Keynote Salvatore Sanfillipo Redis Labs

N-dimensional range queries

• Already implemented as a Ruby library.

• Uses bit to bit interpolation.

• We want to add it as a native API.

• NADD myindex 3 1.4 200.4 4321 element

• NRANGE myindex [0 1] [10 20] (30 40]

Page 38: Redis Day Keynote Salvatore Sanfillipo Redis Labs

Bloom filters

• API is the thing that matters.

• I would love no parameters (like HyperLogLogs).

• Auto scaling, without false negatives.

• Alternatives to bloom filters with same semantics look promising compared to bloom filters.

Page 39: Redis Day Keynote Salvatore Sanfillipo Redis Labs

The endask me anything here or mentioning @antirez