Edge performance with in memory nosql

Edge Performance - NoSqlIn-Memory NoSQL: Memcache, Redis, Aerospike

Liviu CosteaSoftware DeveloperPlugAndTravel.com

About me

Technical guy

Email: email.lcostea@gmail.com

Twitter: clm160

Stack Overflow: user:4138058

Doing software for the travel industry

CTO @ PlugAndTravel.com

Do you need an In-Memory Server?

Scalability at PlugAndTravel?

Moving from one to more servers has challenges:

❏ Session❏ Cached items❏ Static items

Stop using Sticky sessions!

The destination is autoscaling

Autoscaling solutions

❏ Database - could really work, but..

❏ In memory key value stores:❏ Memcached❏ Redis❏ Aerospike

Questions to have answers for

❏ So now you believe an In-Memory NoSql server might be a solution to performance / scalability

First Question

❏ Is it expensive?❏ Linode: 1 cent per GB RAM/h❏ AWS: 1.1 cents per GB RAM/h (~0.7)❏ Azure: 1.2 cents per GB RAM/h

Second Question

❏ RAM is volatile, so I can lose everything?!❏ You can have:

❏ Backups❏ Disk persistence❏ Replica-sets / replication factor

Third Question

❏ Can it scale?❏ Yes, server side clusters will automatically rebalance

themselves after adding more machines

Meet the main characters

❏ Each of these products were created to solve a problem

Memcached - the veteran

❏ Simple KV store to use as a cache❏ Easy ‘immutable’ API❏ Limits: key max 250 bytes, value max of 1MB❏ No clustering, no high availability❏ Not built to add/remove servers live

Redis - the mature

“Redis is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker…” (redis.io)

On AWS & Azure you have it as a platform

Very popular, big community: Twitter, Airbnb, Flickr, StackOverflow, GitHub, PlugAndTravel and many others

Going with Redis

❏ single threaded server❏ data structures with many usages❏ it has scripting in Lua (stored procedures like)❏ a high-availability solution (replica set & monitoring)❏ server side cluster (newest addition)❏ persistence and backups

Redis - Data structures

❏ Strings (everything is a string)

❏ Lists (double linked lists)

❏ Sets (unique lists)

❏ Hashes (like an object)

❏ Sorted sets (by a value at creation)

Efficiently modeling of data is your main concern!

Redis Demohttps://github.com/lcostea/Redis-node-tutorial

Redis - pay attention to

❏ Beware of the long running processes

❏ Keys or Flush commands can bring your master down

❏ Nice to have: memory inspection command(s)

❏ Timeouts will occur, better be ready

Aerospike - the new guy

“Whether you are building new category defining apps or reimagining legacy apps to take advantage of more data for better outcomes, Aerospike is the only database that delivers both speed and scale” (aerospike.com)

Community and Enterprise editions

AWS and Azure Markets, vagrant box

Gaining popularity: Kayak, AppNexus, AdForm, PlugAndTravel and others

Going with Aerospike

❏ Less data structures than Redis

❏ Scripting support with Lua

❏ Storage models: RAM, RAM + Disk, SSD optimized ->

❏ Synchronous replication and async cross datacenter replication❏ Replication factor in cluster

❏ Namespaces, sets, bins and indexes

Aerospike - Storage models

A namespace can have the storage:❏ In memory (without any persistence)❏ In memory, with persistence; data is backed up on disk❏ SSD storage, with indexes in memoryEach namespace is configured according to its storage requirements.

Aerospike Demohttps://github.com/lcostea/Aerospike-Console-App

Aerospike - pay attention to

❏ Replication is synchronously, so pay attention to cross data center (AZs on AWS not recommended)

❏ Cold / fast restart❏ Row size is max 1M (while on disk)❏ Bin (column names) can not be > 14 bytes (not for storing

values)❏ Your indexes should fit in RAM

Small review

❏ In-memory will mean High Performance and Scalability❏ It is the path to AutoScale❏ Memcached: Immutable cache❏ Redis: Remote dictionary server❏ Aerospike: providing hybrid approaches

Questions

In-Memory NoSQL

email.lcostea@gmail.com@clm160