Ehcache 3 JSR-107 on steroids

Louis Jacomet @ljacomet

Lead Software Engineer

Who is that guy?• Louis Jacomet / @ljacomet

• Principal Software Engineer at Software AG / Terracotta since 2013

• A developer closer to his forties that did not fully manage to dodge all things management

• Interests range from concurrency to API design, with learning new things as a driving factor

And who are you?

• Who knows nothing about caching?

• Who already uses caching in production?

• Who had caching related problems in production?

• Ehcache anyone?

Agenda

• Caching introduction

• Ehcache 3 features

• What’s next?

(Caching) theory

– Phil Karlton

“There are only two hard things in Computer Science:

cache invalidation and naming things.”

Cache coherence• P write A to X, P read X => returns A when no

writes happened in between, must be valid for single processor too

• P2 writes A to X, P1 reads X => returns A if “enough time” has elapsed

• P1 writes A to X, P2 writes B to X => no one can ever see B then A at X

What is a cache in an application?

• Data structure holding a temporary copy of some data

• Trade off between higher memory usage for reduced latency

• Targets:

• Data which is reused

• Data which is expensive to compute or retrieve

Desired cache features

• Capacity control / eviction

• Data freshness / expiry

• Data consistency / invalidation

• Fault tolerant

Where and when to use a cache?

It depends …

Measure, do not guess!

https://www.flickr.com/photos/wwarby/

A bit of history[014] - Easy Hibernate Cache 0.5

  by Greg Luck (http://freshmeat.net/users/gregrluck/)

  Saturday, November 29th 2003 12:54

Software Development :: Libraries :: Java Libraries

About: Easy Hibernate Cache is a fast and simple, pure Java, in-process

cache, which acts as a pluggable cache for Hibernate 2.1. It has a small

memory footprint, minimal dependencies, and full documentation.

License: The Apache License

A bit of history (2)• Terracotta founded in 2003, in San Francisco

• Open sourced DSO in 2006

• Acquires Ehcache in Aug. 2009

• Releases Ehcache 2.0 Mar. 2010

• Software AG acquires Terracotta in 2011

A bit of history (3)

• JSR-107 submitted in 2001

• Early draft in 2012

• JSR-107 Final Mar. 2014

• Work on Ehcache 3.0 begins…

Ehcache 3• Breaking change from the 2.x line

• Objectives:

• Modernize API

• Clean up feature bloat

• Support JSR-107 as a first class citizen

Basic API

vs.

Type safe caches• Class<K> and Class<V> provided at configuration

• Provides runtime safety

• Key and value types are known and enforced

• Generics fun in companion objects

• Expiry<? super K, ? super V> getExpiry();

Fluent API for configuration• Builders are immutable

• Any method calls returns a new Builder instance

• Never ignore the return value

• Promotes sharing

• Think template

XML configuration

• XSD based and modular

• JSR-107 and transactions are separate schemas

• Cache template replaces former default cache

• More flexible and powerful

Demo time

Storage model• Multiple storage tiers

• Heap: memory of the JVM, impacts Garbage Collection, most constrained tier

• Off heap: memory of the machine, in JVM process but outside of GC reach, requires serialization

• Disk: Slower but larger file based storage, requires serialization

Overflow model

Heap

Disk

Heap

Disk

Overflow model

Disk

Heap

Overflow model

Disk

Heap

Overflow model

Authoritative tier model

Heap

Disk

Disk

Heap

Authoritative tier model

Heap

Disk

Authoritative tier model

Heap

Disk

Authoritative tier model

Why the new model?• Predictable latency

• Every put pays the price of the lower / slowest tier

• No degradation when a higher tier gets full

• Preserves fast(er) access for hot set

• When a mapping is accessed it moves to the fastest tier available

Ehcache 3 options• Heap

• Heap + off heap

• Heap + disk

• Heap + off heap + disk

• Off heap

• Disk

Ehcache Serialization• Critical to benefit from multiple tiers

• Defaults to Java serialization

• With some optimisations

• Further optimised serializer packaged

• Open for extension

Serializer API

Why use a custom serializer• Space

• Java serialization is really verbose

• Long -> 81 bytes !!

• Performance

• Because less to read, write or copy means more throughput

Expiry

• Goodbye Element

• Expiry done through companion object

• More powerful than before

• Makes some trade-offs explicit

Expiry API

Eviction advisor• Hint that some entries are better than others

• Only a hint, will be overridden if all mappings are advised against eviction

• At a significant performance cost … so beware

• Guaranteed to be invoked once per mapping

• When remains unspecified

Eviction Advisor API

What about eviction policy?

• Gone …

• … for now

• Be active and explain why you need a specific policy on the mailing lists

Cache-through

• Unified API

• CacheLoaderWriter

• Transparent from the outside

• Applies internally, not explicitly as in Ehcache 2.x

CacheLoaderWriter API

What about read-through?

• Still possible

• Just have no-op or failing write* and delete* methods

• Forces the user to make a decision

• Why is your read-through cache accepting puts?

Write-behind

Application Cache

Database

queue

Write-behind & eviction

Application Cache

Database

queue

(K1, V1)

(K1, V1)

Write-behind internally• CacheLoaderWriter wrapper

• Batching means calling *All methods

• No native support for retries

• Can be easily done in your CacheLoaderWriter implementation

• Improved configuration

• Prevents weird setups that were legal in Ehcache 2.x

JSR-107 support• Fully compliant

• TCK passes but for two tests related to open issues

• Opinions on some defaults

• JSR-107 Ehcache behaves slightly differently than stock Ehcache

by ref or by value• JSR-107

• Defaults to by value

• Ehcache

• Default depends on tiering, includes mixed mode

• In practice, default depends on configuration source

cache-through and CAS ops

• JSR-107

• Cache loader ignored in CAS operations but writer not

• Ehcache

• Cache loader and writer used in CAS operations

• Ehcache mode configurable in JSR-107 support

Dropped features• Search

• What was the meaning of average age of hot set?

• Explicit locking

• Promoted the wrong patterns

• CacheManager singleton

• Can be achieved inside your application without relying on a global static variable

Did we drop feature X?

• Well … ask ;-)

• Work on a migration guide and complete feature matrix comparison still needs to happen

Ehcache 2.x landscape• Latest release: 2.10.2

• Frameworks integration

• Spring, Hibernate, Play, JHipster and many more

• Clustering available as OSS

• Terracotta 4.3.2 being the latest

• Includes off heap storage on the server for scale up

Ehcache 3.x landscape• Latest release: 3.0.1

• Frameworks integration

• All supporting JSR-107,

• no native as of today

• Clustering: not yet available, under active development, will have OSS version

Q & Ahttps://github.com/ehcache/ehcache3

Google groups: ehcache-users, ehcache-dev ehcache label on Stackoverflow

@ehcache