Upload
jordi-boggiano
View
4.321
Download
2
Embed Size (px)
Citation preview
Jordi Boggiano@seldaekhttp://nelm.io/
Redis - Your AdvancedIn-Memory Key-Value Store
About Me
Belgian living in Zürich, Switzerland
Weby stuff for 10 years http://seld.be
Symfony2, Composer and other OSS contributions http://github.com/Seldaek
Working at Nelmio http://nelm.io Symfony2 & frontend performance consulting
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Agenda
IntroFeaturesUse CasesUsing it with PHP
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Redis
Wut?
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Store
You put data in it
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Key-Value Store
Like NoSQL?
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
In-Memory Key-Value Store
Like Memcached?
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
In-Memory Key-Value Store
Memory is fast, but ephemeral
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Persist to Disk
Fast and lasting
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Created by @antirez
Sponsored by VMWare
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
So how does it work?
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Simple Text Protocol
Human Readable!
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Example - Client Library1 SET key value
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Example - Client Library1 SET key value2 > OK
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Example - Client Library1 SET key value2 > OK3 GET key
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Example - Client Library1 SET key value2 > OK3 GET key4 > "value"
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Example - Low Level1 *32 $33 SET4 $35 key6 $57 value
1 +OK
1 *22 $33 GET4 $35 key
1 $52 value
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Example - Low Level1 *3\r\n2 $3\r\n3 SET\r\n4 $3\r\n5 key\r\n6 $5\r\n7 value\r\n
1 +OK\r\n
1 *2\r\n2 $3\r\n3 GET\r\n4 $3\r\n5 key\r\n
1 $5\r\n2 value\r\n
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Data Types
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Strings1 SET name Bob2 SET age 203 MGET name age4 > Bob5 > 20
1 GETSET name Alice2 > Bob
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Strings1 SETEX age 3 202 GET age3 > 204 // .. 3 seconds later ..5 GET age6 > null
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Integers1 INCR count2 > 13 INCR count4 > 25 INCRBY count 36 > 5
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Hashes01 HMSET user name Alice email
[email protected] HGET user email03 > [email protected] HKEYS user05 > name06 > email07 HGETALL user08 > name09 > Alice10 > email11 > [email protected]
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Lists01 RPUSH admins Alice02 > 103 RPUSH admins Bob04 > 205 LINDEX admins 006 > Alice07 LLEN admins08 > 209 RPOP admins10 > Bob
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Sets01 SADD page:3:visitors 13402 > 103 SADD page:3:visitors 25304 > 105 SADD page:3:visitors 25306 > 007 SCARD page:3:visitors08 > 209 SMEMBERS page:3:visitors10 > 13411 > 25312 SISMEMBER page:3:visitors 34913 > 0
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Sets1 SADD page:6:visitors 2532 SADD page:6:visitors 9233 SADD page:6:visitors 134 SINTER page:3:visitors page:6:visitors5 > 253
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Sorted Sets1 ZADD highscores 2930 Alice2 ZADD highscores 1546 Bob3 ZREVRANGE highscores 0 10 WITHSCORES4 > Alice5 > 29306 > Bob7 > 1546
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Sort of Lists? Listed Sets?Lists
Sets
Sorted Sets
1 array('foo', 'bar')
1 shuffle(array('foo', 'bar'))
1 ksort(array(3 => 'foo', 1 => 'bar'))
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
The basic datatypes of everylanguage exist in Redis
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Process data in Redisinstead of PHP
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
A Few Features
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Atomic Operations1 SETNX name Alice2 GET name3 > Alice4 SETNX name Bob5 GET name6 > Alice
1 INCR foo2 GET foo3 > 14 INCRBY foo 35 GET foo6 > 4
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
EXPIRE / EXPIREAT / PERSIST
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Pipelining
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
SORT - SQL in your NoSQL1 SORT key2 SORT key LIMIT 0 10 DESC3 SORT key ALPHA4 SORT page:6:visitors BY user_*->rank GET
user_* DESC
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Append-Only File & Snapshots
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Pub/Sub1 SUBSCRIBE foo12 PSUBSCRIBE foo*3 PUBLISH foo0 message
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Master/Slave Replication
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Transactions: MULTI / EXEC /DISCARD
Optimistic Locking with WATCH
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Upcoming Features
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Lua Scripting: EVAL / EVALSHA
This is great.
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
EVAL1 EVAL <body> <num_keys_in_args> [<arg1> <arg2> ... <arg_N>]
1 GET A2 EVAL "return redis.call('get', KEYS[1])" 1 A
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
EVAL
Example: Atomic Conditional Decrement, Client-Side
This may return -ERR, or will block.
1 WATCH foo2 $val = GET foo3 $newVal = max(0, $val - 1); // decrement if foo > 0 client-side4 MULTI5 SET foo $newVal6 EXEC
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
EVAL
Example: Atomic Conditional Decrement, Server-Side
This is instant and can not fail.
01 EVAL "local value = tonumber(redis.call('get', KEYS[1]))02 if value == nil03 then04 return {err="Value at key is not integer"}05 end06 if value > tonumber(ARGV[1])07 then08 value = value - 109 redis.call('set', KEYS[1], value)10 end11 return value" 1 foo 0
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Redis Cluster
Almost as delayed asDuke Nukem Forever
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Speed
Redis is FAST.CPU is unlikely to be the
bottleneck.
What do you do whenyou run out of RAM?
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Wait for Redis Cluster?
If you can. Cluster looks great.
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Use sharding, most clientlibraries can do it.
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Use Edis, a protocol compatiblestorage-oriented server.http://inaka.github.com/edis/index.html
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Use it without persistence, store to disk/DB
and load hot data in memory.A good example in these slides from Wooga
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Ok, great. But..
What is it good for?
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
WEB SCALE
THE CLOUD
NOSQL
ELASTIC
HORIZONTAL SCALING
VERTICAL TOO
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
WEB SCALE
THE CLOUD
NOSQL
ELASTIC
HORIZONTAL SCALING
VERTICAL TOO
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
BULLSHIT
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Get real"If you do actually have to scale,then your database isn't going to
magically do it for you."
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Really.
What is it good for?
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Storing Sessions
Caching Data
Other typical Memcached uses
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Caching: Newspaper SiteExpensive readsFast-changing pagesDynamic tracking of what's read for users
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Caching: Newspaper SiteCache generic results:
Store user read-state in sets
Combine and render for each user
1 SETEX <pageid>:content 600 <data>
1 SADD <pageid>:views <userid>
1 GET <pageid>:content2 SISMEMBER <pageid>:views <userid>
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Logging, games and otherwrite-heavy usages
1 LPUSH logs "Log message"2 // keep the last 1000 entries3 LTRIM logs 0 999
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Highscore tables
Note: players will only be listed once since it is a set.
1 ZADD scores 4290 <playerid>2 ZADD scores 390 <playerid2>3 // ...4 ZREVRANK scores <playerid> // 05 ZREVRANGE scores 0 10 WITHSCORES
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Who's online now?1 ZADD visits <unix> <userid>2 ZADD visits 1327681399 523 ZADD visits 1327683245 184 ZRANGEBYSCORE visits <unix>-3600 <unix>5 ZREMRANGEBYSCORE visits 0 <unix>-3600
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Job QueuesWorkers:
Application:
e.g. the Resque lib from GitHub
1 BRPOP queue
1 LPUSH queue "job description"
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Behavior Tracking1 SETBIT click:<item>:<date> <userid> 12 GETBIT click:3:2012-01-28 55239
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Last but not least, it puts thefun back in Datafunbases
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Usage with PHP
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Extension: phpredis
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Extension: phpredis
Comes with a session handler, sharding, up to date.
http://github.com/nicolasff/phpredis
1 $redis = new Redis();2 $redis->connect('127.0.0.1', 6379);3 $redis->watch('x');4 $val = $redis->get('x');5 $newVal = max(0, $val - 1);6 $result = $redis->multi()7 ->set('x', $newVal)8 ->exec();9 echo $result !== false ? 'Success' : 'Race lost, try again';
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Extension: phpiredis
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Extension: phpiredis
Basic hiredis bindings, protocol parsing, low level.
http://github.com/seppo0010/phpiredis
1 $redis = phpiredis_connect('127.0.0.1', 6379);2 phpiredis_command($redis, 'WATCH x');3 $val = phpiredis_command($redis, 'GET x');4 $newVal = max(0, $val - 1);5 phpiredis_command($redis, 'MULTI');6 phpiredis_command($redis, 'SET x '.$newVal);7 $result = phpiredis_command($redis, 'EXEC');8 echo $result !== false ? 'Success' : 'Race lost, try again';
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Plain PHP: Predis
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Plain PHP: Predis
Very complete, sharding, master/slave auto-select,can use phpiredis for parsing.
http://github.com/nrk/predis
http://github.com/snc/SncRedisBundle (Symfony2integration)
01 $redis = new Predis\Client('tcp://10.0.0.1:6379');02 03 $options = array(04 'cas' => true, // enable Check-and-Set05 'watch' => 'x',06 'retry' => 10, // automatic retries07 );08 09 $result = $redis->multiExec($options, function($transaction) {10 $val = $transaction->get('x');11 $newVal = max(0, $val - 1);12 $transaction->multi();13 $transaction->set('x', $newVal);14 });15 16 echo $result !== false ? 'Success' : 'Race lost 10 times, giving up';
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
It's fast. It's fun. Try it.
try.redis-db.com
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Thank you.
Slides: http://slides.seld.be
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Find Out MoreRedis.ioTry RedisThe Little Redis BookRedis Cluster Specification
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be
Questions?
@seldaek
Feedback:
http://joind.in/4764
Jordi Boggiano Company nelm.ioTwitter @seldaek Blog seld.be