Upload
mongodb
View
233
Download
0
Embed Size (px)
DESCRIPTION
Arthur Viegers, Senior Solutions Architect, MongoDB. The value of the fast growing class of NoSQL databases is the ability to handle high velocity and volumes of data while enabling greater agility with dynamic schemas. MongoDB gives you those benefits while also providing a rich querying capability and a document model for developer productivity. Arthur Viegers outlines the reasons for MongoDB's popularity in IoT applications and how you can leverage the core concepts of NoSQL to build robust and highly scalable IoT applications.
Citation preview
I o T E U R O P E A N C I T Y T O U R
S T U T T G A R T
MongoDB and The Internet of Things
Arthur ViegersSenior Solutions Architect, MongoDB
MongoDB IoT City Tour 2014
*
MongoDB
Document Database
Open-Source
General Purpose
*
Documents Are Core
Relational MongoDB{ first_name: "Paul", surname: "Miller", city: "London", location: [45.123,47.232], cars: [ { model: "Bentley", year: 1973, value: 100000, … }, { model: "Rolls Royce", year: 1965, value: 330000, … } ]}
*
Documents Are Core
Relational MongoDB
Modelling time series datain MongoDB
*
Rexroth NEXO Cordless Nutrunner
*
• Store event data
• Support Analytical Queries
• Find best compromise of:- Memory utilization- Write performance- Read/Analytical Query Performance
• Accomplish with realistic amount of hardware
Time series schema design goal
*
• Document per event
• Document per minute (average)
• Document per minute (second)
• Document per hour
Modelling time series data
*
Document per event
{ deviceId: "Test123", timestamp: ISODate("2014-07-03T22:07:38.000Z"), temperature: 21}
• Relational-centric approach
• Insert-driven workload
*
Document per minute (average)
{ deviceId: "Test123", timestamp: ISODate("2014-07-03T22:07:00.000Z"), temperature_num: 18, temperature_sum: 357}
• Pre-aggregate to compute average per minute more easily
• Update-driven workload
• Resolution at the minute level
*
Document per minute (by second)
{ deviceId: "Test123", timestamp: ISODate("2014-07-03T22:07:00.000Z"), temperature: { 0: 18, 1: 18, …, 58: 21, 59: 21 }}
• Store per-second data at the minute level
• Update-driven workload
• Pre-allocate structure to avoid document moves
*
Document per hour (by second)
{ deviceId: "Test123", timestamp: ISODate("2014-07-03T22:00:00.000Z"), temperature: { 0: 18, 1: 18, …, 3598: 20, 3599: 20 }}
• Store per-second data at the hourly level
• Update-driven workload
• Pre-allocate structure to avoid document moves
• Updating last second requires 3599 steps
*
Document per hour (by second){ deviceId: "Test123", timestamp: ISODate("2014-07-03T22:00:00.000Z"), temperature: { 0: { 0: 18, …, 59: 18 }, …, 59: { 0: 21, …, 59: 20 } }}
• Store per-second data at the hourly level with nesting
• Update-driven workload
• Pre-allocate structure to avoid document moves
• Updating last second requires 59 + 59 steps
*
Rexroth NEXO schema{ _id: ObjectID("52ecf3d6bf1e623a52000001"), assetId: "NEXO 109", hour: ISODate("2014-07-03T22:00:00.000Z"), status: "Online", type: "Nutrunner", serialNo : "100-210-ABC", ip: "127.0.0.1", positions: { 0: { 0: { x: "10", y:"40", zone: "itc-1", accuracy: "20” }, …, 59: { x: "15", y: "30", zone: "itc-1", accuracy: "25” } }, …, 59: { 0: { x: "22", y: "27", zone: "itc-1", accuracy: "22” }, …, 59: { x: "18", y: "23", zone: "itc-1", accuracy: "24” } } }}
When to scale
*
Disk I/O Limitations
*
Working Set Exceeds Physical Memory
Working SetIndexes Data
Working SetDataIndexes
How to scale
*
Scaling Up
*
Scaling Out
First Edition (1771)3 Volumes
Fifteenth Edition (2010)32 Volumes
MongoDB’s Approach
*
Shards and Shard Keys
Shard
Shard key range
*
• Cardinality
• Write distribution
• Query isolation
• Reliability
• Index locality
Shard Key Considerations
*
Shard Key Selection Rexroth NEXO
Cardinality Write Distribution
Query Isolation Reliability Index
Locality
_id Doc level One shard Scatter/gather
All users affected Good
hash(_id) Hash level All Shards Scatter/gather
All users affected Poor
assetId Many docs All Shards Targeted Some assets affected
Good
assetId, hour Doc level All Shards Targeted Some assets affected
Good
*
• MongoDB scales horizontally (sharding)
• Each shard is an independent database, and collectively, the shards make up a single logical database
• MMS makes it easy and reliable to run MongoDB at scale
• Sharding requires minimal effort from the application code: same interface as single mongod
Scaling Data - Summarized
Summary
*
• IoT processes are real-time
• Relational technologies can simply not compete on cost, performance, scalability, and manageability
• IoT data can come in any format, structured or unstructured, ranging from text and numbers to audio, picture and video
• Time series data is a natural fit
• IoT applications often require geographically distributed systems
Why is MongoDB a good fit for IoT?
Thank you!
I o T E U R O P E A N C I T Y T O U R
S T U T T G A R T