Topic 10: Taxonomy of Data and Storage

10: Taxonomy of Data and Storage

Zubair Nabi

[email protected]

April 20, 2013

Zubair Nabi 10: Taxonomy of Data and Storage April 20, 2013 1 / 27

Outline

1 Datasets

2 Storage

3 Beyond RDBMS

4 NoSQL Taxonomy

5 NewSQL


Outline

1 Datasets

2 Storage

3 Beyond RDBMS

4 NoSQL Taxonomy

5 NewSQL


Introduction

Data is everywhere and is the driving force behind our lives

The address book on your phone is data

So is the newspaper that you read every morning

Everything you see around you is a potential source of data whichmight be useful for a certain application

We use this data to share information and make a more informeddecision about different eventsDatasets can easily be classified on the basis of their structure

1 Structured2 Unstructured3 Semi-structured


Introduction








Introduction








Introduction








Introduction





We use this data to share information and make a more informeddecision about different events

Datasets can easily be classified on the basis of their structure1 Structured2 Unstructured3 Semi-structured


Introduction








Introduction








Structured Data

Formatted in a universally understandable and identifiable way

In most cases, structured data is formally specified by a schema

Your phone address phone is structured because it has a schemaconsisting of name, phone number, address, email address, etc.

Most traditional databases contain structured data revolving arounddata laid out across columns and rowsEach field also has an associated type

I Possible to search for items based on their data types


Structured Data







Structured Data







Structured Data




Most traditional databases contain structured data revolving arounddata laid out across columns and rows

Each field also has an associated typeI Possible to search for items based on their data types


Structured Data







Structured Data







Unstructured Data

Data without any conceptual definition or type

Can vary from raw text to binary data

Processing unstructured data requires parsing and tagging on the fly

In most cases, consists of simple log files


Unstructured Data






Unstructured Data






Unstructured Data






Semi-structured Data

Occupies the space between the structured and unstructured dataspectrum

For instance, while binary data has no structure, audio and video fileshave meta-data which has structure, such as author, time of creation,etc.

Can also be labelled as self-describing structure












Outline

1 Datasets

2 Storage

3 Beyond RDBMS

4 NoSQL Taxonomy

5 NewSQL


Database Management Systems (DBMS)

Used to store and manage data

Support for large amounts of data

Ensure concurrency, sharing, and locking

Security is useful too; to enable fine-grained access control

Ability to keep working in the face of failure






























Relational Database Management Systems (RDBMS)

The most popular and predominant storage system in use

Data in different files is connected by using a key field

Data is laid out in different tables, with a key field that identifies eachrow

The same key field is used to connect one table to another

For instance, a relation might have customer ID as key and her detailsas data; another table might have the same key but different data, sayher purchases; yet another table with the same key might have abreakdown of her preferences

Examples include Oracle Database, MS SQL Server, MySQL, IBMDB2, and Teradata










































Structured Query Language (SQL)

Non-procedural language used for data retrieval and manipulation inRDBMS

Adds a layer of abstraction over relational algebra, which enables setoperations, selections, etc.

Due to its declarative nature, users operate in terms of their expectedoutput while the underlying system decides the actual query executionplan

Instructions consist of a specific SQL statement and additionalparameters and operands

For instance, the SELECT operator retrieves certain records, INSERTadds a record, and so on






























RDBMS and Structured Data

As structured data follows a predefined schema, it naturally maps on toa relational database system

I The schema defines the type and structure of the data and its relations

Schema design is an arduous process and needs to be done beforethe database can be populated

Another consequence of a strict schema is that it is non-trivial toextend itFor instance, adding a new attribute to an existing row necessitatesadding a new column to the entire table

I Extremely suboptimal in tables with millions of rows




















Another consequence of a strict schema is that it is non-trivial toextend it

For instance, adding a new attribute to an existing row necessitatesadding a new column to the entire table

















RDBMS and Semi- and Un-structured Data

Unstructured data has no notion of schema while semi-structured dataonly has a weak one

Data within such datasets also has an associated typeI In fact, types are application-centric: It might be possible to interpret a

field as a float in one application and as a string in another

While it is possible, with human intervention, to glean structure fromunstructured data, it is an extremely expensive taskStructureless data generated by real-time sources can change thenumber of attributes and their types on the fly

I RDBMS would require the creation of a new table each time such achange takes place

Therefore, unstructured and semi-structured data does not fit therelational model



Unstructured data has no notion of schema while semi-structured dataonly has a weak oneData within such datasets also has an associated type

I In fact, types are application-centric: It might be possible to interpret afield as a float in one application and as a string in another















While it is possible, with human intervention, to glean structure fromunstructured data, it is an extremely expensive task

Structureless data generated by real-time sources can change thenumber of attributes and their types on the fly

























Outline

1 Datasets

2 Storage

3 Beyond RDBMS

4 NoSQL Taxonomy

5 NewSQL


Motivation

Different semantics:I RDBMS provide ACID semantics:

1 Atomic: The entire transaction either succeeds or fails

2 Consistent: Data within the database remains consistent after eachtransaction

3 Isolation: Transactions are sandboxed from each other4 Durable: Transactions are persistent across failures and restarts

I Overkill in case of most user-facing applicationsI Most applications are more interested in availability and willing to

sacrifice consistency leading to eventual consistencyI This basically available, soft state, eventually consistent (BASE) model

enables applications to function even in the face of partial failure

High Throughput: Most NoSQL databases sacrifice consistency foravailability leading to higher throughput (in some cases an order ofmagnitude)


Motivation


1 Atomic: The entire transaction either succeeds or fails2 Consistent: Data within the database remains consistent after each

transaction

3 Isolation: Transactions are sandboxed from each other4 Durable: Transactions are persistent across failures and restarts






Motivation



transaction3 Isolation: Transactions are sandboxed from each other

4 Durable: Transactions are persistent across failures and restartsI Overkill in case of most user-facing applicationsI Most applications are more interested in availability and willing to





Motivation



transaction3 Isolation: Transactions are sandboxed from each other4 Durable: Transactions are persistent across failures and restarts






Motivation




I Overkill in case of most user-facing applications

I Most applications are more interested in availability and willing tosacrifice consistency leading to eventual consistency

I This basically available, soft state, eventually consistent (BASE) modelenables applications to function even in the face of partial failure



Motivation





sacrifice consistency leading to eventual consistency

I This basically available, soft state, eventually consistent (BASE) modelenables applications to function even in the face of partial failure



Motivation









Motivation









Motivation (2)

Horizontal Scalability: To cater for more data, NoSQL stores can bescaled up by just adding more machines and the underlying systemautomatically re-distributes the data

Commodity Hardware: A large number of RDBMS require specializedand proprietary hardware for operation. In contrast, NoSQL databasesfunction over commodity off-the-shelf hardware

Programming Language Support: Over the years programminglanguages have started providing abstractions for database support(LINQ, etc.) while bypassing SQL. NoSQL databases provideabstractions that directly map onto the language abstractions leadingto tighter coupling


Motivation (2)





Motivation (2)





Motivation (3)

The Rise of Cloud Computing: Cloud Computing applications requirehorizontal scalability and low administration overhead. Bothrequirements are naturally satisfied by NoSQL stores


Outline

1 Datasets

2 Storage

3 Beyond RDBMS

4 NoSQL Taxonomy

5 NewSQL


Introduction

NoSQL databases can be classified on the basis of:

1 Data Model: How data is represented

2 Scalability: How scalable the system is

3 Query Model: What type of API it exposes

4 Persistence: How persistent the data is


Introduction







Introduction







Introduction







Classification by Data Model

Based on the data model, NoSQL databases can roughly be categorizedinto three categories:

1 Key/value Stores: A map/dictionary allowing put/get semantics perkey

2 Document Stores: Complex data structures to encapsulate documentkey/value pairs

3 Column-Oriented Stores: Data laid out by column














Key/value Stores

Data is stored within a large hash map

Simple get/put API

Favour scalability over consistency

Limit on the size of the key

Examples include Amazon’s Dynamo, LinkedIn’s Voldemort, Redis,and Memcached


Key/value Stores


Simple get/put API





Key/value Stores


Simple get/put API





Key/value Stores


Simple get/put API





Key/value Stores


Simple get/put API





Document Stores

Key/value semantics but based on documents

A document encapsulates data in a standard format, such as JSON,XML, PDF, etc.

Documents themselves can be heterogeneous

Documents can also be retrieved based on their content

Examples include Apache CouchDB and MongoDB


Document Stores







Document Stores







Document Stores







Document Stores







Column-Oriented Stores

Data is stored and processed by column

Useful for read-mostly and read-intensive data

Data within the same column is of the same type enablingopportunities for efficient compression

Columns are stored separately so they can be loaded in parallel

Examples include Google’s BigTable (Apache HBase is its open sourceclone) and Facebook’s Cassandra






























Outline

1 Datasets

2 Storage

3 Beyond RDBMS

4 NoSQL Taxonomy

5 NewSQL


Introduction

A hybrid of traditional RDBMS and NoSQL

I Scalability and performance of NoSQL and ACID guarantees of RDBMS

Use SQL as the primary language

Ability to scale out and run over commodity hardwareClassified into:

1 New Databases: Designed from scratch2 New MySQL Storage Engines: Keep MySQL as interface but replace

the storage engine3 Transparent Clustering: Add pluggable features to existing databases

to ensure scalability


Introduction

A hybrid of traditional RDBMS and NoSQLI Scalability and performance of NoSQL and ACID guarantees of RDBMS







Introduction








Introduction



Ability to scale out and run over commodity hardware

Classified into:1 New Databases: Designed from scratch2 New MySQL Storage Engines: Keep MySQL as interface but replace




Introduction




1 New Databases: Designed from scratch

2 New MySQL Storage Engines: Keep MySQL as interface but replacethe storage engine

3 Transparent Clustering: Add pluggable features to existing databasesto ensure scalability


Introduction





the storage engine

3 Transparent Clustering: Add pluggable features to existing databasesto ensure scalability


Introduction








New Databases

1 Query Distribution:I Each node holds a subset of the data

I Queries are split and shipped to nodes that own the dataI Examples include Google’s Spanner and NuoDB

2 Pull Data:I A central node (possibly replicated) holds all dataI A set of processing nodes receives queries and pulls in required data

from the central nodeI Examples include VMware’s SQLFire


New Databases

1 Query Distribution:I Each node holds a subset of the dataI Queries are split and shipped to nodes that own the data

I Examples include Google’s Spanner and NuoDB




New Databases

1 Query Distribution:I Each node holds a subset of the dataI Queries are split and shipped to nodes that own the dataI Examples include Google’s Spanner and NuoDB




New Databases


2 Pull Data:I A central node (possibly replicated) holds all data

I A set of processing nodes receives queries and pulls in required datafrom the central node

I Examples include VMware’s SQLFire


New Databases



from the central node

I Examples include VMware’s SQLFire


New Databases





References

1 NoSQL Databases: https://oak.cs.ucla.edu/cs144/handouts/nosqldbs.pdf

2 NewSQL – The New Way to Handle Big Data: http://www.linuxforu.com/2012/01/newsql-handle-big-data/


https://oak.cs.ucla.edu/cs144/handouts/nosqldbs.pdf

https://oak.cs.ucla.edu/cs144/handouts/nosqldbs.pdf

http://www.linuxforu.com/2012/01/newsql-handle-big-data/

http://www.linuxforu.com/2012/01/newsql-handle-big-data/

Technology

Topic 10: Taxonomy of Data and Storage