Upload
dataversity
View
1.743
Download
0
Embed Size (px)
Citation preview
Karen Lopez @datachick #HeartData
Heart of Data ModelingGraph Databases: Where does the modeling go?
Yes, Please do Tweet/Share today’s event
@datachick #heartdata
Karen López
Karen has 20+ years of data and information architecture experience on large, multi-project programs.
She is a frequent speaker on data modeling, data-driven methodologies and pattern data models.
She wants you to love your data…
She is loves new tech and gadgets
How new tech are you?...so let’s get to know you….
Use Q&A for
formal questions
Get them in now!
Use chat to discuss with each
other
We have a great community
Yes!
Slides
Recording
…next week…
Plan for Today
Why topic?
Graphy Stuff in Relational World
Another way?
Graph Resources
Modern Data
Architectures
will have hybrid
Technologies
WHERE ‘HYBRID’ = ‘SQL and NoSQL’
Clarifying terminology
Graphs Graph Databases Graph Processing
8
NoSQL, Not Only SQL
9
Relational Graph Columnar
Key ValueDocument Databases
Column Family
Graph & Hierarchy ConceptsOverview
Graph Nodes
Node
Node Node
Node
Node
Node
Node
Node
NodeNode
Node
Directed / Undirected
Node
Node
NodeNode
Node
Node
Node
NodeNode
Node
What is this structure?
Ragged Hierarchies
A hierarchy where there is variability in the number of levels across branches.
Node
Node
Node
Node
Node
Node
Node Node
Node
Node
Node
Node
14
Automobile
Engine
Fuel Line Valve Injector Fan
Fan Blade
Bearing
Bolt
Fanbelt
Entertainment System
Bolt
Radio
Satellite Radio
Media Player
Backup camera
Automobile
Engine
Injection System
Fuel Line
Valve
Injector
Fan
Fan Blade
Fan Bearing
Fanbelt
Entertainment System
Bolt
Radio
Satellite Radio
Media Player
Backup camera
Energy Graph
What Happens When…….?
Sometimes we take a group of “sibling widgets” and make them a widget just for them. Think “subassembly”. Then we have to think of this new group as a widget.
15
How we Model Graph in RelationalLots of tricks and tips happening here.
Recursive Relationship
Self Join
Recursive Association
Dog Ear / Mouse Ear
Bill of Materials
???
Data Model – Hierarchy Recursive
Data Model – Hierarchy Recursive
19
What happens when:
We add a new level?Take one away?Promote someone?
Relational Performance Tricks
Special data types
Adjacency Lists
Path Enumerations
Closure Tables
Nested Sets
20
But remember this?
21
Now we have M:N
22
Employee
Reporting Structure
Hierarchy vs. Real World
23
Recursive Relationships
24
Recursives IRL
Reporting structures
Components
Facilities
Documents
Networks
25
SQL Server HierarchyID
26
But aren’t relational databases about relationships?
Labeled Property Graph
Nodes have properties(think key-value pairs)
Nodes have labels(think meta-data and categories)
Relationships are directed
Relationships have names
Relationships have a start and endnode
Relationships have properties
28
Nodepropertyproperty
Nodeproperty
NodeNode
Node
Label
LabelLabel
Label
http://neo4j.com/graphgist/8139605
Graph Databases Physical Architecture
31
TripleStores
Come from semantic technologies movement
A triple is a subject:predicate:object data structure
Individually triples are semantically poor
En masse they provide rich dataset to harvest knowledge and infer connections
Use RDF and XML--SPARQL for queries
Ginger dances with Fred
Fred likes ice cream
Karen loves data
32
Graph Databases – Neo4j
CREATE (matrix1:Movie { title : 'The Matrix', year : '1999-03-31' })CREATE (matrix2:Movie { title : 'The Matrix Reloaded', year : '2003-05-07' })CREATE (matrix3:Movie { title : 'The Matrix Revolutions', year : '2003-10-27' })CREATE (keanu:Actor { name:'Keanu Reeves' })CREATE (laurence:Actor { name:'Laurence Fishburne' })CREATE (carrieanne:Actor { name:'Carrie-Anne Moss' })CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix1)CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix2)CREATE (keanu)-[:ACTS_IN { role : 'Neo' }]->(matrix3)CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix1)CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix2)CREATE (laurence)-[:ACTS_IN { role : 'Morpheus' }]->(matrix3)CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix1)CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix2)CREATE (carrieanne)-[:ACTS_IN { role : 'Trinity' }]->(matrix3)
http://neo4j.com/docs/stable/cypherdoc-movie-database.html
Notice anyting?
Querying Neo4j with Cypher
MATCH (you {name:"You"})
MATCH (expert)-
[:WORKED_WITH]-
>(db:Database
{name:"Neo4j"}) MATCH
path = shortestPath( (you)-
[:FRIEND*..5]-(expert) )
RETURN db,expert,path
IBM Graph
Querying IBM Graph
var url = process.env.graphDBURL + '/gremlin';var query = "def g = graph.traversal(); g.V().has('code','" + req.body.orig +"').out('route').has('code', '" + req.body.dest + "')";var opts = { auth: { user: process.env.username, pass: process.env.password}, json: { gremlin: query } };
request.post(url, opts, function(error, resp, obj) {var result = (obj.result && obj.result.data && obj.result.data.length >
0) ? obj.result.data[0] : null;if (result) { // found a route from orig to dest console.log('route
exists from ' + req.body.orig + ' to ' + req.body.dest); } });
Another IBM Graph model
Titan
Tools and Graph Databases
•No native supportERwin
•No native supportER/Studio
•No native supportPowerDesigner
“the data model is the database”
“the database is the data model”
ODBC / JDBC connectively for querying.
So what about modeling for graph?
Marketing vs. Real Project
There is a model
The model isn’t the structure
The model would be used to design the graph(s)
Same modeling issues: NamingPropertiesRulesConsistencyGovernance
Data Modeling & Graph
No* Logical + Physical Data model
The graph is the data model...and the database
Whiteboard data modeling
Traditional data models still have a role
Requirements
Data Model
Database*
More
requirements
/ changes /
tuning /
whims
+ Non Model Stuff
Data Model
Driven
Data Model Driven
10+ Tips for Architects
1. Understand the use cases for graph technologies
2. Evaluate/profile your data requirements for suitability for graph databases and/or graph processing
3. ACID support varies across products. You’ll want to test your use cases.
4. Your query data stories will guide your decisions
5. Test your current development tools for support
10+ Tips for Architects
6. Test your database design/data modeling tools
7. Leverage your existing metadata/models
8. True hierarchies are VERY RARE in the real world.
9. Know the questions you have to ask about all the exceptions
10. Keep asking where the data integrity happens/is relevant
46
Resources
47
http://www.neo4j.org/learn/try 48
Fun with Graphs
Scotch Whiskeys
Belgian Beer
Bank Fraud Detection
Access Control Management
…and 100 more….
GraphGist Project - http://gist.neo4j.org/
Bluemix
http://www.ibm.com/analytics/us/en/technology/cloud-data-services/products/graph.html
Trees and Hierarchies in SQL for Smarties
51
Whitepaper
http://whitepapers.dataversity.net/content50141/
And it’s FREE! GraphDatabases.com
PostgreSQLRiakHbaseMongoDBNeo4JCouchDBRedis
“Every design decision should include cost,
benefit and risk”
- Karen Lopez
Thank you, you were great. Really, really great.
Karen Lopez @datachickwww.datamodel.com