-
Redis based In-Memory Data Grid and Distributed Locks in
Java
Nikita Koksharov, Founder of Redisson
-
Redisson overview(Redis based In-Memory Data Grid)
Distributed Locks and Synchronizers
Distributed Objects
Distributed Services
Distributed Collections
Integration with frameworksLocal Cache
-
Distributed objects
1. Object holder2. Binary stream holder3. Geospatial holder4.
BitSet5. AtomicLong6. Atomic Double7. Publish Subscribe8. Bloom
filter
-
Distributed collections
1. Map (supports eviction)2. Multimap (supports eviction)3.
Set (supports eviction)4. SortedSet / ScoredSortedSet5. List6.
Queue / Deque7. Blocking Queue / Deque8. Priority Queue /
Deque9. Delayed Queue
-
Distributed locks and synchronizers
1. Lock2. FairLock3. MultiLock4. RedLock5. ReadWriteLock6.
Semaphore7. CountDownLatch
-
Distributed services
1. Remote Service2. Live Object Service3. Executor
Service4. Scheduler Service5. Map Reduce Service
-
Integration with frameworks
1. Spring framework2. Spring Cache3. Hibernate 2nd Level
Cache4. JCache API (JSR-107) implementation5. Tomcat Session
Manager6. Spring Session
-
Local cache support
1. Read operations up to 45x faster2. Integrated into
1. Map2. Map with expiring entries3. Spring Cache4.
Hibernate Cache
-
Map
ConcurrentMapmap=newConcurrentHashMap();
map.put(20,newMyObject("oldobj"));
map.putIfAbsent(20,newMyObject("newobj"));
map.containsKey(1);
-
Redisson Map
ConcurrentMapmap=redisson.getMap("someMap");
map.put(20,newMyObject("oldobj"));//wrapsHSETcommand
map.putIfAbsent(20,newMyObject("newobj"));//wrapsLUA-script
map.containsKey(1);
-
ConcurrentMap.putIfAbsentLua Script Implementation
"ifredis.call('hsetnx',KEYS[1],ARGV[1],ARGV[2])==1then"
"returnnil;"
"else"
"returnredis.call('hget',KEYS[1],ARGV[1]);"
"end;“
//KEYS[1]=mapname
//ARGV[1]=mapkey
//ARGV[2]=mapvalue
-
Redisson Map Eviction
//implementsjava.u\l.concurrent.ConcurrentMap
RMapCachemap=redisson.getMapCache("someMap");
//maxIdleTime=20minutes
map.put(20,"oldobj",20,TimeUnit.MINUTES);
//maxIdleTime=20minutes,\meToLive=10minutes
map.put(20,"oldobj",20,TimeUnit.MINUTES,10,TimeUnit.MINUTES);
-
Redisson Sharded Map
Redis Node 1
Redis cluster
Map Shard 1
Map Shard 2
Redis
Redis master/slave or single
Map Shard 1
Map Shard 2
Map Shard 3
Map Shard 4
Map Shard 5
…
Map Shard 5
Map Shard 5
Redis Node 2
Map Shard 3
Map Shard 4
Redis Node 3
Map Shard 5
Map Shard 6
…
-
Redisson Map with Local Cache
ConcurrentMapmap=
redisson.getLocalCachedMap("someMap");
-
Local cached Map entry Eviction policy
1. Last Recently Used2. Last Frequently Used3. Soft
Reference4. Weak Reference
-
Local cached Map entry Invalidation policy
1. On change2. On change + clears whole cache on
reconnect3. On change + clears changed entries
only on reconnect
-
Redisson references
RMapse^ngs=redisson.getMap(“se^ngs”);
RMapop\ons1=redisson.getMap(“se^ngs_server1”);
op\ons1.put(“name”,“s1”);
op\ons1.put(“cpu”,“80”);
//store“op\ons”mapinside“se^ngs”map
se^ngs.put(“server1”,op\ons1);
//read“op\ons”mapfrom“se^ngs”map
RMapop\ons=se^ngs.get(“server1”);
-
Redisson lock
//implementsjava.u\l.concurrent.locks.Lock
RLocklock=redisson.getLock("lock");
lock.lock();
//or
lock.lock(10,TimeUnit.MINUTES);
//…
lock.unlock();
-
Locking algorithm
1. Tries to acquire lock by invoking Lua-script2. If lock
couldn’t be acquired it subscribes to LockChannel3. Does in
loop:
1. Tries to acquire lock again2. If lock couldn’t be acquired
it waits for “unlock”event from
LockChannel published by current lock owner4. Unsubscribes from
LockChannel if lock has been acquired5. During unlock method
invocation it deletes lock state and
publishes “unlock”event to LockChannel
-
Redisson fair lock
//implementsjava.u\l.concurrent.locks.Lock
RLocklock=redisson.getFairLock("lock");
lock.lock();
//or
lock.lock(10,TimeUnit.MINUTES);
//…
lock.unlock();
-
Fair locking algorithm
1. Tries to acquire lock by invoking Lua-script2. If lock
couldn’t be acquired it subscribes to
LockChannel:redissonId:threadId and stores this id into threads
queue
3. Does in loop:1. Tries to acquire lock again2. If lock
couldn’t be acquired it waits for “unlock”event from
LockChannel:redissonId:threadId published by current lock
owner4. Unsubscribes from LockChannel:redissonId:threadId if lock
has been
acquired5. During unlock method invocation it deletes lock
state object and
publishes “unlock”event to the first
LockChannel:redissonId:threadId obtained from threads queue
-
Redisson multilock
RLocklock1=redisson1.getLock("lock1");
RLocklock2=redisson2.getLock("lock2");
RLocklock3=redisson3.getLock("lock3");
//implementsjava.u\l.concurrent.locks.Lock
RLocklock3=newRedissonMul\Lock(lock1,lock2,lock3);
lock.lock();
//…
lock.unlock();
-
Multilocking algorithm
1. Acquires lock one by one:1. Tries to acquire lock 2. If
lock on current step can’t be acquired
all previously acquired locks are unlocked and locks iterator is
reset to the first lock
-
Redisson Remote Service.Service registration
RRemoteServiceremoteService=redisson.getRemoteService();
ServiceImplservice=newServiceImpl();
//Canhandleonly1invoca\onconcurrently
remoteService.register(SomeService.class,service);
//Canhandleupto12invoca\onsconcurrently
remoteService.register(SomeService.class,service,12);
-
Redisson Remote Service.Service invocation
RRemoteServiceremoteService=redisson.getRemoteService();
SomeServiceservice=remoteService.get(SomeService.class);
Stringresult=service.doSomeStuff(1L,“secondParam”,newAnyParam());
-
Redisson Remote Service.Asynchronous calls
publicinterfaceService{
LongsomeMethod(Longparam1,Stringparam2);
MyObjectsomeMethod();
}
@RRemoteAsync(Service.class)
publicinterfaceServiceAsync{
RFuturesomeMethod(Longparam1,Stringparam2);
RFuturesomeMethod();
}
-
Tasks execution architectureRedisson Node
Redis (single, master/slave, sentinel, cluster)
Application Node
Redisson
Application Node
Redisson
Application Node
Redisson
workers
Redisson Node
workers
Redisson Node
workers
-
Tasks execution without Redisson Node
Redissonredisson=…
//customexecutorworkers
redisson.getExecutorService("myExecutor").registerWorkers(16);
//map-reduceworkers
redisson.getExecutorService(MAPREDUCE_NAME).registerWorkers(16);
-
Workers on application side
Redis (single, master/slave, sentinel, cluster)
Application Node
Redisson
Application Node
Redisson
Application Node
Redisson
workers workers workers
-
Task definition / Callable
publicclassCallableTaskimplementsCallable{
@RInject
privateRedissonClientredissonClient;
privateStringparam;
@Override
publicMyResultcall()throwsExcep\on{
//taskbody
}
}
-
Task definition / Runnable
public class RunnableTask implements Runnable {
@RInject
private RedissonClient redissonClient;
@Override
public void run() {
// task body
}
}
-
Submitting tasks for execution
//implementsjava.u&l.concurrent.ExecutorService
RExecutorServiceexecutorService=redisson.getExecutorService("myExecutor");
executorService.submit(newRunnableTask());
//orwithparameters
executorService.submit(newRunnableTask(41,"someParam"));
executorService.submit(newCallableTask());
//orwithparameters
executorService.submit(newCallableTask(53,"someParam"));
-
Submitting tasks for scheduled execution
//implementsjava.u&l.concurrent.ScheduledExecutorService
RScheduledExecutorServicees=redisson.getExecutorService("myExecutor");
es.schedule(newCallableTask(),10,TimeUnit.MINUTES);
//or
es.schedule(newRunnableTask(),5,TimeUnit.SECONDS);
es.scheduleAtFixedRate(newRunnableTask(),10,25,TimeUnit.HOURS);
//or
es.scheduleWithFixedDelay(newRunnableTask(),5,10,TimeUnit.HOURS);
-
Scheduling tasks with cron expressions
RScheduledExecutorServicees=redisson.getExecutorService("myExecutor");
//usesQuartzcronformat
es.schedule(newRunnableTask(),CronSchedule.of("100/5***?"));
//or
es.schedule(newRunnableTask(),CronSchedule.dailyAtHourAndMinute(10,5));
-
Map Reduce overview
Map task
Map phase* …
Map task
Map task
Map task
Reduce task
Reduce phase …
Reduce task
Reduce task
Reduce task
*splits to several tasks and run in parallel only for
RShardedSet and RShardedMap objects
-
Map ReduceData source example
RMapmap=redisson.getMap("wordsMap");
map.put(“page1:line1","Alicewasbeginningtogetvery\red");
map.put("page1:line2","ofsi^ngbyhersisteronthebankand");
map.put("page1:line3","ofhavingnothingtodoonceortwiceshe");
map.put("page1:line4","hadpeekedintothebookhersisterwasreading");
map.put("page1:line5","butithadnopicturesorconversa\onsinit");
map.put("page1:line6","andwhatistheuseofabook");
map.put("page1:line7","thoughtAlicewithoutpicturesorconversa\on");
-
Mapper definition
publicclassWordMapperimplementsRMapper{@Overridepublicvoidmap(Stringkey,Stringvalue,RCollectorcollector){String[]words=value.split("");for(Stringword:words){collector.emit(word,1);}}}
-
Reducer definition
publicclassWordReducerimplementsRReducer{@OverridepublicIntegerreduce(Stringword,Iteratoriter){intsum=0;while(iter.hasNext()){Integeri=(Integer)iter.next();sum+=i;}returnsum;}}
-
Collator definition
publicclassWordCollatorimplementsRCollator{@OverridepublicIntegercollate(Mapresult){inttotalWordsAmount=0;for(Integercount:result.values()){
totalWordsAmount+=count;}returntotalWordsAmount;}}
-
Running Map Reduce process
RMapmap=redisson.getMap("wordsMap");RMapReducemapReduce=map.mapReduce().mapper(newWordMapper()).reducer(newWordReducer()).\meout(60,TimeUnit.SECONDS);//countoccurrencesofwordsMapwordToNumber=mapReduce.execute();//counttotalwordsamountIntegertotalWordsAmount=mapReduce.execute(newWordCollator());
-
How to start
//1.Createconfigobject
Config=newConfig();
config.useClusterServers()
.addNodeAddress("myserver.com:7000","myserver.com:7001");
//2.CreateRedissoninstance
RedissonClientredisson=Redisson.create(config);
//3.Getobjectyouneed
Mapmap=redisson.getMap("myMap");
-
Connection modes
1. AWS Elasticache nodes2. Azure cache nodes3. Cluster nodes4.
Sentinel nodes5. Master and Slave nodes6. Single node
-
Data serialization
1. Jackson JSON2. CBOR3. MsgPack4. Snappy5. Kryo6. FST7.
LZ48. JDK Serialization9. Amazon Ion10. Avro11. Smile
-
Asynchronous command
executionRMapAsyncmap=redisson.getMap("someMap");
FutureputIfFuture=map.putIfAbsentAsync(20,"object");
FuturegetFuture=map.getAsync(20);
getFuture.addListener(newFutureListener(){
@Override
publicvoidopera\onComplete(Futurefuture)
throwsExcep\on{
//…
}
});
-
Reactive command execution
RedissonReacFveredisson=Redisson.createReacFve(config);
RMapReacFvemap=redisson.getMap("someMap");
PublisherputRes=map.put(20,"object");
Publishervalue=map.get(20);
-
Redisson vs Other IMDG or Cache Solution
Redisson + Redis
Java based IMDG or cache solution (hazelcast, ehcache …)
Distributed objects and services ü ü
Large memory amount handling ü have issues
Fully-managed service support ü û
-
Used by
-
Thanks关注开源数据库论坛
