Thomas RisbergVMware

trisberg@vmware.com

Spring Data

1Tuesday, April 17, 12

• currently works for VMware’s Cloud Foundry team on framework integration

• member of Spring Data team

• committer on Spring Framework

• former Oracle DBA

• former Java J2EE developer

email: trisberg@vmware.comtwitter: @trisberg

•expert in NO SQL

2Tuesday, April 17, 12

3Tuesday, April 17, 12

New demands on data access ...

• massive amounts of data

• structured and unstructured data

• real-time analysis

image courtesy of Bitcurrent

• social networking features

• inexpensive horizontal scaling

• deploying apps in the cloud

• ...

4Tuesday, April 17, 12

New types of data stores ...

• Redis• Riak • Cassandra

• HBase

• MongoDB• CouchDB • Neo4j

Column/

Hadoop

5Tuesday, April 17, 12

Spring Framework built-in data access support

• Transaction abstractions, Data access exceptions

• JDBC - JdbcTemplate

• ORM - Hibernate, JPA support

• OXM - Object to XML mapping

• Cache support (Spring 3.1)

6Tuesday, April 17, 12

Spring Data• Bring classic Spring benefits to all databases

✓ Productivity

✓ Programming model consistency

• Conventions based generic repository support

• Mapping between Java domain objects and data store

• Support for a wide range of new databases7Tuesday, April 17, 12

• JPA Repository

• JDBC Extensions

• Redis (Key Value)

• MongoDB (Document)

• Neo4j (Graph)

• Gemfire (Data Grid)

• Apache Hadoop (Big Data)

• Riak (Key Value - in development)

• Cassandra (Column Store - planned)

Spring Data sub-projects

8Tuesday, April 17, 12

Spring Data Building Blocks

• Low level data access APIs

✓ MongoTemplate, RedisTemplate ...

• Object Mapping (Java to Datastore)

• Generic Repository support

• Cross Store Persistence Programming model

9Tuesday, April 17, 12

Finding Spring Data

• GitHub: https://github.com/SpringSource

• Web page:

http://www.springsource.org/spring-data

• Forum:

http://forum.springsource.org/forumdisplay.php?f=80

10Tuesday, April 17, 12

Three databases for today’s talk

• Relational database

• Document database

• Graph database

11Tuesday, April 17, 12

Three persistence strategies for today’s talk

• Conventions based persistence -- Repositories

• Lower-level Template approach

• Cross-store persistence using JPA and a non-relational database

12Tuesday, April 17, 12

Repository

http://www.sxc.hu/photo/1020163

13Tuesday, April 17, 12

RepositoryMediates between the domain and data mapping layers using a collection-like interface for accessing domain objects.http://martinfowler.com/eaaCatalog/repository.html

http://msdn.microsoft.com/en-us/library/ff649690.aspx

14Tuesday, April 17, 12

Spring Data Repository

• Generic repository implementation

• Basic CRUD (create, read, update and delete) methods

• Generating code for dynamic find methods defined in repository interface like findByName, findByAgeBetween etc.

• Pagination and sorting support

• Currently JPA, Mongo and Neo4j implementations

15Tuesday, April 17, 12

CrudRepository<T, ID extends Serializable>

Repository<T, ID extends Serializable>

PagingAndSortingRepository<T, ID extends Serializable>

16Tuesday, April 17, 12

Query Methods

17Tuesday, April 17, 12

<?xml version="1.0" encoding="UTF-8"?><persistence version="1.0" xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"> <persistence-unit name="test"> <properties> <property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect"/> <property name="hibernate.hbm2ddl.auto" value="update"/> <property name="hibernate.ejb.naming_strategy" value="org.hibernate.cfg.ImprovedNamingStrategy"/> </properties> </persistence-unit></persistence>

<persistence>

18Tuesday, April 17, 12

<jpa:repositories base-package="org.springframework.data.demo.repository" />

<tx:annotation-driven transaction-manager="transactionManager"/>

<context:component-scan base-package="org.springframework.data.demo"> <context:exclude-filter expression="org.springframework.stereotype.Controller" type="annotation"/></context:component-scan>

<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager"> <property name="entityManagerFactory" ref="entityManagerFactory" /></bean>

<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"> <property name="dataSource" ref="dataSource" /> <property name="persistenceUnitName" value="test"/> <property name="jpaVendorAdapter"> <bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter"> <property name="generateDdl" value="true" /> </bean> </property></bean>

<configuration>

19Tuesday, April 17, 12

Built F

or

<beans profile="default"> <bean class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" id="dataSource"> <property name="driverClassName" value="com.mysql.jdbc.Driver"/> <property name="url" value="jdbc:mysql://localhost/test"/> <property name="username" value="root"/> <property name="password" value=""/> <property name="testOnBorrow" value="true"/> </bean></beans>

<beans profile="cloud"> <cloud:data-source id="dataSource"/></beans>

20Tuesday, April 17, 12

App using JPA Repository

CODEhttps://github.com/trisberg/jpa-bookshelf-repository

21Tuesday, April 17, 12

{ "name" : "Jon Brisbin", "_links" : [ { "rel" : "books", "href" : "http://localhost:8080/data/author/2/books" }, { "rel" : "self", "href" : "http://localhost:8080/data/author/2" } ]}

{ "_links" : [ { "rel" : "Book", "href" : "http://localhost:8080/data/author/2/books/1234567890" } ]}

{ "title" : "Spring Data REST in Action", "price" : 22.55, "_links" : [ { "rel" : "author", "href" : "http://localhost:8080/data/author" }, { "rel" : "self", "href" : "http://localhost:8080/data/author/2" } ], "categories" : [ "Spring", "Java" ], "published" : "2012-04-24"}

https://github.com/SpringSource/spring-data-rest22Tuesday, April 17, 12

23Tuesday, April 17, 12

Horizontally Scalable

{ author: “steve”, date: new Date(),

text: “About MongoDB...”, tags: [“tech”, “database”]}

Document Oriented

Application

High Performance

24Tuesday, April 17, 12

Documents

> p = {author: “roger”, date: new Date(), text: “about mongoDB...”, tags: [“tech”, “databases”]}

> db.posts.save(p)

25Tuesday, April 17, 12

Querying

> db.posts.find()

> { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "roger", date : "Sat Jul 24 2010 19:47:11", text : "About MongoDB...", tags : [ "tech", "databases" ] }

26Tuesday, April 17, 12

Secondary Indexes

// 1 means ascending, -1 means descending > db.posts.ensureIndex({author: 1}) > db.posts.find({author: 'roger'}) > { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "roger", ... }

27Tuesday, April 17, 12

Conditional Query Operators

// find posts with any tags > db.posts.find( {tags: {$exists: true }} )

// find posts matching a regular expression> db.posts.find( {author: /^rog*/i } )

// count posts by author> db.posts.find( {author: ‘roger’} ).count()

$all, $exists, $mod, $ne, $in, $nin, $nor, $or, $size, $type, $lt, $lte, $gt, $gte

28Tuesday, April 17, 12

Atomic Updates$set, $unset, $inc, $push, $pushAll, $pull, $pullAll, $bit

> comment = { author: “fred”, date: new Date(), text: “Best Movie Ever”}

> db.posts.update( { _id: “...” }, $push: {comments: comment} );

29Tuesday, April 17, 12

Nested Documents { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "roger", date : "Sat Apr 24 2011 19:47:11", text : "About MongoDB...", tags : [ "tech", "databases" ], comments : [ { author : "Fred", date : "Sat Apr 25 2010 20:51:03 GMT-0700", text : "Best Post Ever!" } ]}

30Tuesday, April 17, 12

Advanced indexing// Index nested documents> db.posts.ensureIndex( “comments.author”: 1)> db.posts.find({‘comments.author’:’Fred’})

// Index on tags (multi-key index)> db.posts.ensureIndex( tags: 1)> db.posts.find( { tags: ‘tech’ } )

// geospatial index> db.posts.ensureIndex( “author.location”: “2d” )> db.posts.find( “author.location”: { $near : [22,42] } )

31Tuesday, April 17, 12

Rich Documents{ _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), line_items : [ { sku: ‘tt-123’, name: ‘Coltrane: Impressions’ }, { sku: ‘tt-457’, name: ‘Davis: Kind of Blue’ } ], address : { name: ‘Banker’, street: ‘111 Main’, zip: 10010 }, payment: { cc: 4567, exp: Date(2011, 7, 7) },

subtotal: 2355}

32Tuesday, April 17, 12

Getting started with

$ wget http://fastdl.mongodb.org/osx/mongodb-osx-x86_64-2.0.4.tgz$ tar xvzf mongodb-osx-x86_64-2.0.4.tgz$ cd mongodb-osx-x86_64-2.0.4$ ./bin/mongod --dbpath /Users/trisberg/Data/mongodb/Sun Apr 15 11:33:11 [initandlisten] MongoDB starting : pid=34692 port=27017 dbpath=/Users/trisberg/Data/mongodb/ 64-bit host=Montserrat.localSun Apr 15 11:33:11 [initandlisten] db version v2.0.4, pdfile version 4.5Sun Apr 15 11:33:11 [initandlisten] git version: 329f3c47fe8136c03392c8f0e548506cb21f8ebfSun Apr 15 11:33:11 [initandlisten] build info: Darwin erh2.10gen.cc 9.8.0 Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_I386 i386 BOOST_LIB_VERSION=1_40Sun Apr 15 11:33:11 [initandlisten] options: { dbpath: "/Users/trisberg/Data/mongodb/" }Sun Apr 15 11:33:11 [initandlisten] journal dir=/Users/trisberg/Data/mongodb/journalSun Apr 15 11:33:11 [initandlisten] recover : no journal files present, no recovery neededSun Apr 15 11:33:12 [websvr] admin web console waiting for connections on port 28017Sun Apr 15 11:33:12 [initandlisten] waiting for connections on port 27017

33Tuesday, April 17, 12

$ vmc create-service mongodb mongo-books$ vmc tunnel mongo-books mongoPassword: *****Binding Service [mongo-books]: OKStopping Application: OKStaging Application: OK Starting Application: OK Getting tunnel connection info: OK

Service connection info: username : 97da4cc2-8919-42f9-8bdb-11f0e565c2b6 password : 61b04646-4f9f-4804-8360-20662c2a2c95 name : db

Starting tunnel to mongo-books on port 10000.Launching 'mongo --host localhost --port 10000 -u 97da4cc2-8919-42f9-8bdb-11f0e565c2b6 -p 61b04646-4f9f-4804-8360-20662c2a2c95 db'

MongoDB shell version: 2.0.1connecting to: localhost:10000/db> show collectionsauthorbooksystem.indexessystem.users> db.author.find(){ "_id" : ObjectId("4f8af65d942bb7cb40b29764"), "_class" : "org.springframework.data.demo.domain.Author", "name" : "Craig Walls" }{ "_id" : ObjectId("4f8af65d942bb7cb40b29765"), "_class" : "org.springframework.data.demo.domain.Author", "name" : "Kristina Chodorow" }...

&

34Tuesday, April 17, 12

Spring Data MongoDB✓ MongoTemplate

• MongoConverter interface for mapping Mongo documents

• MappingMongoConverter for POJO mapping support, leveraging Spring TypeConverters

• Annotation based mapping (@Document, @Id, @DbRef)

• Connection affinity callbacks

• Exception translation

• Support for GridFS and Map Reduce

✓ MongoRepository• Similar support as JPA Repositories

35Tuesday, April 17, 12

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"> <constructor-arg name="mongoDbFactory" ref="mongoDbFactory"/> </bean>

<context:component-scan base-package="org.springframework.data.demo"> <context:exclude-filter expression="org.springframework.stereotype.Controller" type="annotation"/> </context:component-scan>

<configuration>

36Tuesday, April 17, 12

Built F

or

<beans profile="default"> <mongo:db-factory host="localhost" port="27017" dbname="db"/></beans>

<beans profile="cloud"> <cloud:mongo-db-factory id="mongoDbFactory" write-concern="NORMAL"/></beans>

37Tuesday, April 17, 12

Document store using MongoTemplate

LIVE CODEhttps://github.com/trisberg/mongo-bookshelf-template

38Tuesday, April 17, 12

• DB is a collection of graph nodes, relationships

• Nodes and relationships have properties

• Querying is done via a traversal API

• Indexes on node/relationship properties

• Written in Java, can be embedded

• Transactional (ACID)

• Master-Slave replication

• Standalone using REST API

39Tuesday, April 17, 12

Neo4j Data Model

40Tuesday, April 17, 12

• Support for property graphs (nodes connected via relationships, each with arbitrary properties)

• Transparent mapping of annotated POJO entities (using AspectJ

• Neo4jTemplate with convenient API, exception translation and optional transaction management

• Supports the Cypher and Gremlin query languages

• Dynamic type projections (duck typing)

• Spring Data Repositories Support

• Cross-store support for partial JPA – Graph Entities

Features

41Tuesday, April 17, 12

• JPA data and “NOSQL” data can share a data model• Separate the persistence provider by using annotations

– could be the entire Entity– or, some of the fields of an Entity

• We call this cross-store persistence– One transaction manager to coordinate the “NOSQL” store

with the JPA relational database– AspectJ support to manage the “NOSQL” entities and fields

• holds on to changed values in “change sets” until the transaction commits for non-transactional data stores

Cross-store

42Tuesday, April 17, 12

A cross-store scenario ...You have a traditional web app using JPA to persist data to

a relational database

43Tuesday, April 17, 12

JPA Data ModelRestaurant

@Entitypublic class Restaurant {

@Id @GeneratedValue private Long id; private String name; private String city; private String state; private String zipCode;

UserAccount@Entity@Table(name = "user_account")public class UserAccount { @Id @GeneratedValue private Long id; private String userName; private String firstName; private String lastName; @Temporal(TemporalType.TIMESTAMP) private Date birthDate; @ManyToMany(cascade = CascadeType.ALL) private Set<Restaurant> favorites;

44Tuesday, April 17, 12

Cross-store Data Model

Recommendation

@RelationshipEntitypublic class Recommendation { @StartNode private UserAccount user; @EndNode private Restaurant restaurant; private int stars; private String comment;

Restaurant@Entity@NodeEntity(partial = true)public class Restaurant { @Id @GeneratedValue private Long id; private String name; private String city; private String state; private String zipCode;

UserAccount@Entity@Table(name = "user_account")@NodeEntity(partial = true)public class UserAccount { @Id @GeneratedValue private Long id; private String userName; private String firstName; private String lastName; @Temporal(TemporalType.TIMESTAMP) private Date birthDate; @ManyToMany(cascade = CascadeType.ALL) private Set<Restaurant> favorites;

@GraphProperty String nickname; @RelatedTo(type = "friends", elementClass = UserAccount.class) Set<UserAccount> friends; @RelatedToVia(type = "recommends", elementClass = Recommendation.class) Iterable<Recommendation> recommendations;

45Tuesday, April 17, 12

Cross-store persistence with JPA/Neo4j

DEMOhttps://github.com/SpringSource/spring-data-neo4j/tree/

2.0.1.RELEASE/spring-data-neo4j-examples/myrestaurants-social46Tuesday, April 17, 12

Questions?

http://www.sxc.hu/photo/860327

47Tuesday, April 17, 12