Redis ndc2013

Redis Tips

[email protected]

• OpenSource Contributor • REDIS • TWEMPROXY

Before Presentation

Agenda • Dangerous Commands •Memory Policies •Replication •RDB

Dangerous Commands •Keys *(pattern)

•FlushAll

Dangerous Commands •Why? –Single Thread. –Each command should spend small time.

Keys * • Don’t use keys command in production. –“Warning: consider KEYS as a command that should only be used in production environments with extreme care” is written in redis.io keys commands manual.

http://redis.io/commands/keys

Keys * di = dictGetSafeIterator(c->db->dict); allkeys = (pattern[0] == '*' && pattern[1] == '\0'); while((de = dictNext(di)) != NULL) { …… stringmatchlen(pattern,plen,key,sdslen(key),0) }

FlushAll Cache Item Count Time

Memcache 1,000,000 1~2ms

Redis 1,000,000 1000ms(1 second)

FlushAll • Redis’s FlushAll is slow and paused.

• But memcache’s flush is really fast.

–memcache doesn’t depend on Item count.

–Redis depends on Item count.

FlushAll • What is different from memcache’s flush.

• Memcache just sets time to flush.

• Redis deletes all items.

–It spends 1 second in 1M or 2M items

FlushAll-Redis long long emptyDb() { for (int j = 0; j < server.dbnum; j++) { dictEmpty(server.db[j].dict); dictEmpty(server.db[j].expires); } return removed; }

FlushAll-Redis

FlushAll-Memcache static void process_command(conn *c, char *command) { if (exptime > 0) settings.oldest_live = realtime(exptime) - 1; else /* exptime == 0 */ settings.oldest_live = current_time - 1; }

FlushAll-Memcache if (settings.oldest_live != 0 && settings.oldest_live <= current_time && it->time <= settings.oldest_live) { do_item_unlink(it, hv); do_item_remove(it); it = NULL; }

Redis Memory Policy • Volatile-lru • Allkeys-lru • Volatile-random • Allkeys-random • Volatile-ttl • noeviction

Redis Memory Policy • In 64bit, Memory Policy doesn’t work,

if you don’t set maxmemory parameter.

• In 32bit, maxmemory is set 4GB as default

Default Setting in 64 • There is no memory limitation.

• So If you insert bigger data than memory. It will be slow.

• But But But. Make sure Vm.overcommit_memory property.

Memory Policy • If you use Memory-Policy and

replication.

– It can cause some consistency problem.

Master Slave Delete propagation

Delete items because of its own memory policy

Replication •Support Chained Replication

•Common mistake

Replication •Support Chained Replication

Master

Slave Slave

Slave Slave

Replication • Common mistake. –Don’t forget “slaveof no one”

• Slave’s replicationCron keeps track of master’s health.

Replication

Master Slave replicationCron

Health check

Replication


Health check

Replication


When master reruns, Resync with Master

Replication


Slave will has no data after resyncing

If master has no data.

Initial Replication Step

RDB • Good and Bad.

• It causes all of problems.

RDB - GOOD • Can store data into persistent layer.

RDB - BAD • Bad Performance in Large memory.

• Twice memory problem.

• Denying write problem when storing RDB fails.

• If you just want Cache. Turn off RDB

RDB – Large Memory • Performance is relevant to Memory

Size.

17.1 GB 34.2 GB 68.4 GB

117 GB

RDB - BAD • Using bgsave in slave.

• Turning off RDB.

• Using save in master or slave.

RDB – Twice Memory • fork() and COW(copy on write) Issue

–In Write Heavy System:

RDB – Twice Memory



Real Case Study • Background

–Can’t write to Redis Server

–Sentinel doesn’t find Server’s failure.

Real Case Study • Reason

– If redis fails to save RDB, Redis basically denies write operations from client.

– “MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.”

Real Case Study if (server.stop_writes_on_bgsave_err && server.saveparamslen > 0 && server.lastbgsave_status == REDIS_ERR && c->cmd->flags & REDIS_CMD_WRITE) { flagTransaction(c); addReply(c, shared.bgsaveerr); return REDIS_OK; }

Real Case Study • Solution #1

• Solution #2

config set stop-writes-on-bgsave-error no

Turn off RDB Setting

2.6.12 부터 conf 에서 stop-writes-on-bgsave-error 설정이 가능해짐.

SELECT Command • Don’t use “SELECT” Command.

–You will not be able to use “SELECT” in Redis Cluster.

• Redis Proxy can make some troubles.

Response Time • Why memcache’s response time is

uniformed? –Difference of Memory Allocator

Memcache Redis - Slab Allocator - Just malloc()

Redis VS Memcache • The popular question

–Redis and Memcache: What is better?

Memcache Redis - Uniformed Response

Time - More stable

- Data Structure - Persistent

Sentinel

Sentinel • Sentinel is Failover Solution for Redis.

Master Slave

Sentinel Sentinel periodically checks Redis Master

Sentinel

Master Slave

Sentinel Send “slaveof no one” to slave node

Sentinel

Master Slave

Sentinel Notify to client about changing master

Client

Sentinel redis 127.0.0.1:2003> psubscribe * Reading messages... (press Ctrl-C to quit) 1) "pmessage" 2) "*" 3) "+switch-master" 4) "resque 127.0.0.1 1999 127.0.0.1 2002"

Sharding or Clustering

Two Approches.

Client Library VS

Server Proxy

Client Library

NHN Case Study: http://helloworld.naver.com/helloworld/294797

Craiglist Case Study: http://blog.zawodny.com/2011/02/26/redis-sharding-at-craigslist/

Client Library ZooKeeper

or health Checker

Redis Library

Craigslist

NHN Application Servers

ZooKeeper

Redis Cluster Manager

Redis Shard-1

Redis Shard-2

Redis Shard-3

NHN

Client Library • Benefits –No hob(faster than server proxy)

• Liabilities

–Consistency Problem(Timing Issue)

Server Proxy

Redis Cloud Services

Twemproxy https://github.com/twitter/twemproxy

Twemproxy • Simple and fast memcache/redis proxy

Client Client Client Client

Twemproxy

Redis Redis Redis Redis

Twemproxy If you use redis as a store, Twemproxy might not be good for your purpose

Server Proxy • Benefits –Client doesn’t need any other libraries

• Liabilities

–Hob.(less speed and proxy is also SPOF)

Q & A

Thank you!

Technology

Redis ndc2013