Upload
zareh
View
49
Download
2
Embed Size (px)
DESCRIPTION
Infinispan , transactional key- value DataGrid and NoSQL database. 11. April 2013 Alexander Petrov. Alexander Petrov. Sr. Consultant at Inmeta Consulting Current project: Skattetaten Grid POC Previous projects involving grid technologies: - PowerPoint PPT Presentation
Citation preview
Infinispan, transactional key-value DataGrid and NoSQL database11. April 2013 Alexander Petrov
Alexander Petrov
• Sr. Consultant at Inmeta Consulting
• Current project: Skattetaten Grid POC
• Previous projects involving grid technologies:• Mattilsynet food authority system.• FrameSolution BPM framework used in Lovisa National Court
Authority(Norway), Mattilsynet Food Authority
• Other noteworthy projects• Coca Cola Basis ERP system – Coca Cola Bottler factories• mPower Mobilitec 300 million subscribers worldwide, and delivers over
500,000 pieces of content every day.
Usage scenarios
• Big data, Databases are slow. Memory is FAST!• Provides huge computing power.• Tax calculation • Financial organizations• Government organizations use it for communication and
data sharing between the different departments.• Scientific computations• MMORPG games
Agenda
• General terminology relevant to Distributed Caching
• Challenges related to introducing distributed caching to existing system
• Metrics and tuning
Distributed Caching - Concepts
• Cache JSR – 107• Java Data Grid JSR - 347• In memory Data Grid• Cluster• Distribution• Node – a member of a cluster• Transaction awareness• Colocation• Map / Reduce• Consistency
Real World Use Case
Typical J2EE backend
Data access
• Transaction scope• Locking\deadlocking• Flushing policies• Mixing the technologystack.• Performance
Legacy Cache
Our end goal
• Wow we did it!
Summary
• Our Custom cache is super fast, but its cache hit ratio is rather low.
• Our custom cache has a tendency of getting dirty as the updates to the shared data can not be propagated. At the same time the separation of the data regions is not full.
• Marshaling is a rather slow and heavy process.
• We are facing a technological cocktail and we need to keep integrity.
Replication
• Write through• Write Behind• Replication Queue
Invalidation
Distribution
More terminology
• Eviction• Least Recently Used• First In First Out• LIRS• Custom
• Expiration
• Invalidation
Caching topologies – Mirrored Cache
• Ref. Data vs Transactional• Reference data: Good.Max 30000 reads/sec 1k size• Transactional data: Good.Max 25000 writes/sec 1k size
.
Caching topologies – Replica Cache
• Reference data: Good.30000 reads/sec per server.Grow linearly by adding servers.
• Transactional data: Not sogood. Max 20000writes/second. Drops if you add 3rd server to2500.
Caching topologies – Partitioned Cache
• Ref. Data vs Transactional• Reference data: Good.Max 30000 reads/sec 1k size• Transactional data: Good.Max 25000 writes/sec 1k size
Caching topologies - Partitioned Replica
• Reference data(1kb):Good. 30000 reads/sec per server. Grow linearly by adding servers.
• Transactional data(1kb):Good. 20000 writes/sec per server.Grow linearly by adding servers.
How to define our topology
• What is the size of our cluster? Reads vs. Writes• Communication inside our grid
• UDP,TCP• Synchronous vs. Asynchronous.• What about the transaction isolation?
• Repeatable Reads vs. Read Committed• What is the nature of our application?• Read intensive data
• CMS systems • Write Intensive Data
• Document Management System
Level 1 Cache / Near Cache
• Level1 cache is Supported only for Distribution mode• Level 1 cache mighthave a performance Impact in certain systems
Cache stores and loaders
• Passivation
• Activation
• Hibernate
Transactions, Isolation and Locking
• Long running transactions need to be avoided.
• What is a long running transaction? How long is actually long.
• Read Committed vs Repeatable Reads
Classic Deadlock situation
begin Update(A) Update(B) Update(C) Update(B)
Begin Update(C) Update(B) Release(A) Lock(A)
TX1 (Wants update A,B,C)
TX2 (Wants to update C,B,A)
C is locked by TX2
A is locked by TX1
Repeatable Read
begin get(k) - - Get(k)
Begin Get(k) put(k, v2) commit
What is returned??
TX1
TX2
Cache statistics
Remoting statistics
Locking statistics
Marshaling data
• Java serialization
• Java externalization
• Impact on performance
• Generic domain.
Real World Use Case
Data access
• Transaction scope• Locking\deadlocking• Flushing policies• Mixing the technologystack.• Performance
Our end goal
• Wow we did it!
The End
• Thank you for your attention
Used sources
http://www.alachisoft.com/ncache/caching-topology.htmlhttp://www.infoq.com/news/2011/10/java-data-gridhttps://github.com/datagrids/spec/wikihttp://www.jboss.org/infinispan/documentationhttp://code.google.com/p/thrift-protobuf-compare/wiki/Benchmarking