18
H-Store: A Specialized Architecture for High- throughput OLTP Applications Evan Jones (MIT) Andrew Pavlo (Brown) 13 th Intl. Workshop on High Performance Transaction Syste October 26, 2009

H-Store: A Specialized Architecture for High-throughput OLTP Applications

  • Upload
    gur

  • View
    18

  • Download
    0

Embed Size (px)

DESCRIPTION

H-Store: A Specialized Architecture for High-throughput OLTP Applications. Evan Jones (MIT) Andrew Pavlo (Brown) 13 th Intl. Workshop on High Performance Transaction Systems October 26, 2009. Intel Xeon E5540 (Nehalem/Core i7). - PowerPoint PPT Presentation

Citation preview

Page 1: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

H-Store: A Specialized Architecture for High-throughputOLTP Applications

Evan Jones (MIT)Andrew Pavlo (Brown)13th Intl. Workshop on High Performance Transaction SystemsOctober 26, 2009

Page 2: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Intel Xeon E5540 (Nehalem/Core i7)

October 26, 2009

Source: Intel 64 and IA-32 Architectures Optimization Reference Manual

Page 3: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Distributed Clusters

October 26, 2009

Page 4: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Scaling OLTP on Multi-Core?

Use a distributed shared-nothing design

October 26, 2009

Page 5: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

How to Make a Faster OLTP DBMS Main memory storage

Replication for durability

Explicitly partition the data

Specialized concurrency control Stored procedures only Single partition: execute one transaction at time Multiple partitions: supported but slow

October 26, 2009

Page 6: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Source: S. Harizopoulos, D. J. Abadi, S. Madden, M. Stonebraker, “OLTP Under the Looking Glass”, SIGMOD 2008.

OLTP: Where does the time go?

October 26, 2009

Page 7: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Users Rely on Partitioning

October 26, 2009

Source: R. Shoup, D. Pritchett, “The eBay Architecture,” SD Forum, Nov. 2006.

Page 8: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

What about multi-core? Traditional approach:

One database process Thread per connection Shared-memory, locks and latches

H-Store approach: Thread per partition Distributed transactions

October 26, 2009

Page 9: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Example Microbenchmark One table per client

Table(id INTEGER, counter INTEGER)

Each client executes the following query:UPDATE TableSET counter = counter + 1WHERE id = 0;

Add clients to find maximum throughput Data on RAM disk

October 26, 2009

Page 10: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Experimental Configuration

October 26, 2009

Threads Partitions

Page 11: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Partitions versus Threads

October 26, 2009

Ideal

Partitions

Threads

Page 12: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Scalability AnalysisPartitions scale better than threads.

Threads: contention for shared resources [1]

Partitions: memory bottleneck causes sublinear scaling

H-Store: Not just for distributed shared-nothing clusters

October 26, 2009

[1] R. Johnson et al., "Shore-MT: A Scalable Storage Manager for the Multicore Era," EDBT 2009.

Page 13: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Multi-core Design Problem How to automatically create a data placement

scheme to improve multi-core throughput?

Data Partitioning: Maximize the number of single-partition transactions.

Data Placement: Maximize the number of single-node transactions.

October 26, 2009

Page 14: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Database Partitioning Select partitioning keys and construct schema tree.

October 26, 2009

DISTRICTDISTRICT

CUSTOMERCUSTOMER

ORDER_ITEMORDER_ITEM

ITEMITEM

STOCKSTOCK

WAREHOUSEWAREHOUSE

ORDERSORDERS

DISTRICTDISTRICT

CUSTOMERCUSTOMER

ORDER_ITEMORDER_ITEM

STOCKSTOCK

ORDERSORDERS ITEMITEM

Replicated

WAREHOUSEWAREHOUSE

TPC-C Schema Schema Tree

Page 15: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

ITEMITEMITEMjITEMjITEMITEMITEMITEMITEMITEM

Database Partitioning Combine table fragments into partitions.

October 26, 2009

P2

P4

DISTRICTDISTRICT

CUSTOMERCUSTOMER

ORDER_ITEMORDER_ITEM

STOCKSTOCK

ORDERSORDERS

Replicated

WAREHOUSEWAREHOUSE

P1

P1

P1

P1

P1

P1

P2

P2

P2

P2

P2

P2

P3

P3

P3

P3

P3

P3

P4

P4

P4

P4

P4

P4

P5

P5

P5

P5

P5

P5

P5

P3

P1

ITEMITEMITEMITEM

ITEMITEM ITEMITEM

ITEMITEM

Partitions

ITEMITEM

Schema Tree

Page 16: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

Partition AffinityCore1 Core2

HT2

HT1

Core1 Core2

HT2

HT1

Data Placement Assign partitions to cores on each node.

October 26, 2009

P1ITEMITEM

P2ITEMITEM

P5ITEMITEM

Partitions Cluster Node

P4ITEMITEM

P3ITEMITEM

Node 1 Node n

P1

P2P3P4P5

Page 17: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

H-Store’s Future New Name. New Company.

Six full-time developers.

Open-source project (GPL)

Beta by end of 2009 Multiple deployments in financial service areas.

October 26, 2009

Page 18: H-Store:  A Specialized Architecture for High-throughput OLTP Applications

More Information H-Store Info + Papers:

http://db.cs.yale.edu/hstore/

VoltDB Project Information: http://www.voltdb.org/

October 26, 2009