RetroDB(We have seen it all)

Donald KossmannSystems Group, ETH Zurich

We got it all right…

why is nobody listening?

Why is nobody listening?• Web (e.g. Amazon, Facebook, Google) – reinventing the wheel is cooler than listening– do not worry about them

• Enterprise (e.g., Amadeus, Credit Suisse, …)– they do listen– but, new problem: No more silos! (aka Big Data)– RDBMS not a good match for that new problem– we need to repackage!

• (I do not know about Scientific applications)

Repackaging DB Technology

Blob store as a service (HDFS++)

Repackaging DB Technology

Blob store as a service (HDFS++)

OLTP

Repackaging DB Technology

Blob store as a service (HDFS++)

OLTP

OLAP

Streaming

Repackaging DB Technology

HDFSOLTP

OLAP

Streaming

Graph

Search… ML

Repackaging DB Technology

• Data in Blob Store, Processing in Compute Nodes• Great advantages– scales storage and processing individually– no need to worry about “multi-tenancy” & silos– fault-tolerance for free– commodity building blocks (KVS, 2PC, SI, SQL, …)– it is cool because Google does it

• Great disadvantages– poor data locality (data shipping)– poor semantics (sharing increases noise)

What we need to do!• Optimize Shared Memory DBMS – split work between tiers: e.g., push down scans– shared scans in storage tier– new ways to implement ACID in client/server system– (many more optimizations)

• Get semantics right– it is one big soup of data– but everybody wants to look at it in different ways

• And build a really good HDFS++– across the storage hierarchy (DRAM, SSD, NVRAM,

disk)

What we need NOT do!

• 300 gazillion TPS in a single box– great, but who needs that?– what to do with the data once it is in there?

• Think about caching– if you have locality, make it explicit

• Worry about eventual consistency, NoSQL, … or dismiss anything else we have done!