Database Pro Power Days 2010 - Graph data in the cloud using .NET

Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]> 1Achim Friedland <[email protected]>

19./20. Oktober 2010

Nürnbergwww.databasepro-powerdays.de

Graph Data in the cloudusing .NET

1Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]>

Photo: Large Magellanic Cloud, ESO

sones

Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]> 2

Photo: funky64, flickr


For 35 years information has been well-defined data within some tables

jailed in closed database silos.


Photo: shamballah, flickr


The relational model and SQL have become much too limited for open

linked data and cloud requirements.== graph data




Applications can not access, under-stand and process unknown relational data easily.

DB 1

Application 1

DB 2

Application 2

? ?


Photo: Gephi, flickr


1. Graph-Databases


The Property-Graph Model



FriendsAliceID = 1

Age = 21

BobID = 2

Age = 23since = 2009/09/21

reason = classmates

Edge

Vertex

Edge-Properties

Vertex-Properties



7

1

Person

20101014…

Alice

21

Red, Green

1 Infinite Loop

Cupertino

Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]>

CarolID = 3

Age = 20

ID

TYPE

REVISION

Name

Age

Boyfriend

Friends

FavColors

Address.Street

Address.TownClose to Object- and

Document-Databases

BobID = 2

Age = 23


Direct linking without external

indices




8

BobID = 2

Age = 23

20101014…

Alice

21

Red, Green

1 Infinite Loop

Cupertino

ID

rdf:type

sones:revId

foaf:name

foaf:age

person

Set<person>

List<string>

XML

gn:streeet

gn:town

Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]>

CarolID = 3

Age = 20

using RDF-like semantics

ID

TYPE

REVISION

Name

Age

Boyfriend

Friends

FavColors

Address

Address.Street

Address.Town

+ Unambiguous identifiers

+ Named relations

+ Close to RDF molecules

http://test.com/vertices/1

http://test.com/#person


Graph-Databases in a cloud

Achim Friedland <[email protected]> 9


Database Pro Power Days , 10/20/2010

GraphDB

REST

Hypermedia • Representation must be „link-aware“e.g. XML+XLINK, ATOM, RDFa…

• Representation should be self-describing

• Vertices and edges are resources• Access via e.g. http://test.com/vertices/[$id]• Common CRUD operations (GET, POST, PUT…)

+ Atomicity

+ Statelessness

+ Idempotence

+ Parallelism


Photo: bombeador, flickr


2. The Object-relationalImpedance Mismatch

Graph

Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]> 1111Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]>

Inflexible relational schemata

• Expensive ALTER TABLE operations

• Entity-Attribute-Value Model ↔ RDF

• No semi-/unstructured dataXML, JSON, … hierarchies, graphs, … binary data

• No Multi-Attribute ValuesList<String>, Set<Integer>, Set<Person>

• No simple way for versioned data



Relational Anti-Patterns:

• Relations via foreign key constraintsNo explicit concept for relationsNo index-free adjacency

• Querying relational data via JOINs is hardJust storing a graph was never a challenge ;)

• No recursive JOINsInefficient query processing(Except: Oracle’s “CONNECTED BY”)





SQL and Cloud-Readiness?

• No explicit scaling or partitioning within the relational model

• No JOINs between different databases and/or vendors

• No well interaction with state-of-the-art web technologiese.g. HTTP/REST, Hypermedia, Semantic Web




3. Benefits of Graph-Databases



The explicit graph data model provides a higher level of abstraction

and a better understanding of the domain model.




Index-free adjacency provides an improved scalability, data-locality

and a superior graph traversal performance.

( Independent of the size of the graph )



Photo: squacco, flickr


Consistency criteria and indices for simple attributes up to complex

subgraph structures.


Photo: Khem, flickr

Traversing linked information, finding shortest-paths, do semantic

partitions.


Photo: NASA, flickr


Recommendation and discovery

of potentially interessting linked information.

personal social item-related


Photo: Birger Hoppe, flickr

Good integration into state-of-the-art programming concepts and web

technologies.

Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]> 21Achim Friedland <[email protected]> 21


Database Pro Power Days , 10/20/2010

Graph-Databases, REST and RDF symantics are a solid foundation for

cloud services




4. Graph data in the cloudusing .NET / Mono


• URL• License• Language• Goals• Concurrency• Repl./Scaling• Persistency• Cloud

http://www.sones.deAGPLv3C# 4.0Management of linked dataMVCCp2p (alpha)Proprietary file systemConnector for Microsoft Azure

sones GraphDB

Photo: sones



Photo: sones


sones Architecture

GraphDSREST, WebShell, C# API

GraphDBGQL, Graph Traversals, Indices

GraphFSObject Management, (De-)Serialization

Host File System / Microsoft Azure


Photo: sones


Shared nothing

User

GraphDS 1

GraphDS 2

GraphDB 2

GraphDB 1

GraphFS 2

GraphFS 1

Azureit depends…


Photo: sones

User Friend BobID = 2

Alter = 23since = 2009/09/21


sones Property-Hypergraph

User Friend CarolID = 3

Alter = 20since = 2010/04/11

SET<User> Friends

SetMaxNumber = 12

AliceID = 1

Alter = 21

Hyperedge-Properties

Hyperedge

Edge

Virtual -Edge


Photo: sones

• Properties may include code as dataThink of stored procedures; C#: Func<…>, ExpressionTrees

• Allows hyperedge calculations be done among the set of their edges(GetMinWeight, SetMaxNumber, …)

sones Property-Hypergraph

27NoSQL Frankfurt , 9/28/2010Achim Friedland <[email protected]>


Photo: Shayne Kaye, flickr

sones Graph Query Language


FROM User SELECT User.Friends.Friends.Name

• “SQL for graphs” providing a user-friendly DSL for ad-hoc graph queries and graph discovery

• Functions and aggregates are type-safe and can be extended by your own plug-ins, e.g.

• SELECT COUNT(User.Friends)

• SELECT User.Friends.Random(2)

• SELECT User.Friends.Name.Substring(2,5)



sones Graph Query Language


// sones gql example

CREATE VERTEX UserADD ATTRIBUTES (String Name, SET<User> Friends)INDICES (Name)MANDATORY (Name)

INSERT INTO User VALUES (Name = "Alice", Age = 21)INSERT INTO User VALUES (Name = "Bob", Age = 23)

LINK User(Name = ‘Alice') VIA Friends TO User(Name = ‘Bob')LINK User(Name = ‘Bob') VIA Friends TO User(Name = ‘Alice‘)



C# API


// C# API type creation

var _Person = _GraphDB.TypeManager.CreateVertex(„Person“).AddString(„Name“, mandatory: true, indexed: true).AddLoop(„Friends”, hyperEdge: true).execute();

Type _PersonT = _GraphDB.TypeManager.GenerateType(_Person);



C# API


// C# API vertex initialization

Person _Alice = _GraphDB.TypeManager.ActivateVertex(_Person, new VertexUUID(1));

_Alice.Name = „Alice“;

dynamic _Alice2 = _Alice;_Alice.Age = 21;_Alice.bdayparty = (Action) (() => { _Alice.Age++; });

_Alice.bdayparty();



C# API


// sones C# API example

var _Friends = new GraphAttribute(„Friends“, Type: „foaf:knows“);

var _Bob = _GraphDB.TypeManager.ActivateVertex(_Person, new VertexUUID(2));

_Alice.Link(_Friends, _Bob);_Bob.Link(_Friends, _Alice);



C# API


// Graph Traversals

Public T TraverseVertex<T> (IVertex myStartVertex,TraversalOperation TraversalOperation =

TraversalOperation.BreathFirst,Func<IVertex, IEdge, Boolean> myFollowThisEdge = null,Func<IVertex, Boolean> myMatchEvaluator = null,Action<IVertex> myMatchAction = null,Func<TraversalState, Boolean> myStopEvaluator = null,Func<IEnumerable<IVertex>, T> myWhenFinished = null)

{// Traverse the graph

}

Database Pro Power Days , 10/20/2010Achim Friedland <[email protected]> 34Achim Friedland <[email protected]>

For more information…

[email protected]://www.twitter.com/ahzf

http://www.twitter.com/graphdbs

sones

Technology

Database Pro Power Days 2010 - Graph data in the cloud using .NET