©  2014  GridGain  Systems,  Inc.  

DMITRIY  SETRAKYAN  Founder  &  EVP  Engineering  @dsetrakyan  

GridGain  In-­‐Memory  Data  Fabric:  UlCmate  Speed  and  Scale  for  TransacCons  and  AnalyCcs  

www.gridgain.com   #gridgain  

©  2014  GridGain  Systems,  Inc.  

Agenda  

•  EvoluCon  of  In-­‐Memory  CompuCng  •  GridGain  In-­‐Memory  Data  Fabric  •  Distributed  Cluster  &  Compute  

–  Coding  Example  

•  Distributed  Data  Grid  –  Coding  Examples  

•  Distributed  Streaming  &  CEP  •  Plug-­‐n-­‐Play  Hadoop  Accelerator  

©  2014  GridGain  Systems,  Inc.  

What  is  In-­‐Memory  CompuFng  

•  High  Performance  &  Low  Latencies  •  Faster  than  Disk  and  Flash  •  Cost  EffecCve  •  Distributed  or  Not  •  Caching,  Streaming,  ComputaCons  •  Data  Querying  –  SQL  or  Unstructured  •  VolaCle  and  Persistent  •  OLAP  and  OLTP  Use  Cases  

©  2014  GridGain  Systems,  Inc.  

EvoluFon  of  In-­‐Memory  CompuFng  

Caching  

Distributed  Caching  

In-­‐Memory  Data  Grids  IMDBs  

Database  IM  opCons   Hadoop  

accelerators  

Streaming  

BI  accelerators  

Clustering & Compute Grid

Data Grid Streaming

Hadoop Acceleration

©  2014  GridGain  Systems,  Inc.  

ExisFng  Market  is  Fragmented  

Company   Product   Proprietary/  Open  Source   CharacterizaFon  

Oracle In-Memory Option for Oracle Database Proprietary Cost Option

Oracle Times Ten Proprietary Point Solution IMDB

Oracle Coherence Proprietary Point Solution IMDG

SAP Hana Proprietary Point Solution - IMDB

Microsoft SQL Server 2014 Proprietary Feature Upgrade

DataBricks Apache Spark Open Source Point Solution - Hadoop

VoltDB VoltDB Open Source Point Solution – IMDB

Aerospike Aerospike Open Source Point Solution – NoSQL DB

IBM DB2 with BLU Acceleration Proprietary Feature Upgrade

Software AG Terracotta Open Source Point Solution - IMDG

Hazelcast Hazelcast Open Source Point Solution - IMDG

GridGain  In-­‐Memory  Data  Fabric:    Strategic  Approach  to  IMC  

•  Supports all Apps

•  Open Source – Apache 2.0 •  Simple Java APIs •  1 JAR Dependency •  High Performance & Scale •  Automatic Fault Tolerance •  Management/Monitoring •  Runs on Commodity Hardware

•  Supports existing & new data sources

•  No need to rip & replace

Clustering & Compute Grid

Data Grid Streaming

Hadoop Acceleration

©  2014  GridGain  Systems,  Inc.  

©  2014  GridGain  Systems,  Inc.  

Clustering  &  Compute  •  Direct  API  for  MapReduce  •  Direct  API  for  Fork/Join  •  Zero  Deployment  •  Cron-­‐like  Task  Scheduling  •  State  Checkpoints  •  Early  and  Late  Load  Balancing  •  AutomaCc  Failover  •  Full  Cluster  Management  •  Pluggable  SPI  Design  

©  2014  GridGain  Systems,  Inc.  

AutomaFc  Cluster  Discovery  

©  2014  GridGain  Systems,  Inc.  

Closure  ExecuFon  

©  2014  GridGain  Systems,  Inc.  

Closure  ExecuFon  

©  2014  GridGain  Systems,  Inc.  

In-­‐Memory  Caching  and  Data  Grid  

•  Distributed  In-­‐Memory  Key-­‐Value  Store  •  Replicated  and  ParCConed  •  TBs  of  data,  of  any  type  •  On-­‐Heap  and  Off-­‐Heap  Storage  •  Backup  Replicas  /  AutomaCc  Failover    •  Distributed  ACID  TransacCons    •  SQL  queries  and  JDBC  driver  •  CollocaCon  of  Compute  and  Data  

©  2014  GridGain  Systems,  Inc.  

Cache  OperaFons  

©  2014  GridGain  Systems,  Inc.  

Cache  TransacFon  

©  2014  GridGain  Systems,  Inc.  

Distributed  Java  Data  Structures  

•  Distributed  Map  (cache)  •  Distributed  Set  •  Distributed  Queue  •  CountDownLatch  •  AtomicLong  •  AtomicSequence  •  AtomicReference  •  Distributed  ExecutorService  

©  2014  GridGain  Systems,  Inc.  

Client-­‐Server  vs  Affinity  ColocaFon  

Client-­‐Server   Affinity  ColocaCon  

©  2014  GridGain  Systems,  Inc.  

In-­‐Memory  Streaming  &  CEP  

•  Streaming  Data  Never  Ends  •  Branching  Pipelines  •  CEP  Sliding  Windows  •  Pluggable  RouCng  •  Real  Time  Analysis  •  At  Least  Once  Guarantee  

©  2014  GridGain  Systems,  Inc.  

Plug-­‐n-­‐Play  Hadoop  Accelerator  

•  Up  to  100x  AcceleraCon  •  In-­‐Memory  NaCve  MapReduce  

–  In-­‐Process  Data  ColocaCon  –  Eager  Push  Scheduling  

•  GGFS  In-­‐Memory  File  System  –  Pure  In-­‐Memory  –  Write-­‐Through  to  HDFS  –  Read-­‐Through  from  HDFS    

•  Sync  and  Async  Persistence  

©  2014  GridGain  Systems,  Inc.  

In-­‐Memory  NaFve  MapReduce  

•  In-­‐Memory  NaCve  MapReduce  –  Zero  Code  Change  –  Use  exisCng  MR  code  –  Use  exisCng  Hive  queries  

•  No  Name  Node  •  No  Network  Noise  •  In-­‐Process  Data  ColocaCon  •  Eager  Push  Scheduling  

©  2014  GridGain  Systems,  Inc.  

DevOps  Management  and  Monitoring  

©  2014  GridGain  Systems,  Inc.  

THANK  YOU  

www.gridgain.com   #gridgain  @dsetrakyan