Embed Size (px)
Citation preview
Felix [email protected]
Context: Database-as-Service& Backend-as-a-Serivce
Infrastructure-as-a-Service
Platform-as-a-Service
…Database-as-a-ServiceXeround
Amazon RDS
SQL Azure
Cloudant
MongoHQ
Parse
Orestes
Google F1
DynamoDB
Managed NoSQLDatabases
ManagedRDBMS
Backend-as-a-Service"
Storage APIs
Cloud-Deploymentof DBMSs
Cloud-onlyDBaaS-Systems
BigQuery
EMR
Analytics-as-a-Service
ClientDatabase Application
Web Server
Web Server
3-Tier Architecture
Single Page Application
Data (e.g. JSON)
RedundanceApplications reimplementbackend functionality
Database DependenceStrong coupling betweenapplication and database system
Performance & ScalabilityError-prone application-specificscaling policies
Difficult DevelopmentBoth client and server have tobe implemented
ClientDatabase Application
Web Server
Web Server
3-Tier Architecture
Single Page Application
Data (e.g. JSON)
RedundanceApplications reimplementbackend functionality
Database DependenceStrong coupling betweenapplication and database system
Performance & ScalabilityError-prone application-specificscaling policies
Difficult DevelopmentBoth client and server have tobe implemented
With every 100ms of additional page load time, revenue decreases by 1%.
Study by Amazon
When increasing load time of search results by 500ms, traffic decreases by 20%.
Study by Google
Rich ClientOrestesDB-Cluster
RESTServer
Transpa-rentCaching Low Latency
Automated Choice of Database System
(PolyglotPersistence)
Cloud
Extensible High-Level Database and Backend Abstractions:• User-Management• Access Control• Schema• Transactions
RESTServer
Unified REST API
Orestes Middleware Architecture:◦ Stateless REST servers exposing the unified
REST API
High-Level Backend Abstractions:◦ Authentication, Authorization, Data Validation,
Stored Procedures, Standard Schemas, Push Updates, Files
High-Level Database Abstractions:◦ Polyglot persistence◦ Schema Management◦ Transparent web-caching ◦ Cache-aware optimistic transactions
Orestes Middleware Architecture:◦ Stateless REST servers exposing the unified
REST API
High-Level Backend Abstractions:◦ Authentication, Authorization, Data Validation,
Stored Procedures, Standard Schemas, Push Updates, Files
High-Level Database Abstractions:◦ Polyglot persistence◦ Schema Management◦ Transparent web-caching ◦ Cache-aware optimistic transactions
Thesis:Modern applications can be developed on a unified REST API with superior performance-and functional properties provided on top ofexisting database systems
Platform-specific interfaces map to unified REST API
REST API leverages existing HTTP infrastructure◦ Load-Balancer (stateless communication REST constraint)
◦ Web-Caches (caching REST constraint)
Register
User Browser
var usr=new User(name, pw);usr.register();
SDK
POST db/_native.User/JSON Object
Orestes
Defined in REST specification
## Get objectReturns the specified object...
@bucket : String The bucket name@oid : String The unique object identifier
GET /db/:bucket/:oid info.orestes.server.resource.db.DbBucketOid : OrestesObject
## ...
CRUD.rest
queryschematransactiontransaction-crudtransaction-queryconfigurationpartial-updateregistration-login
Unified REST API Specification
Orestes-Server
public OrestesObject load(ObjectInfo objInfo)
Data Module forspecific DB
db/bucket/oid Resource
New database systemscan easily be plugged in
Modules for: queries, CRUD, transactions, etc.
HTTP/Networking handledindependent from DB
Defined in REST specification
## Get objectReturns the specified object...
@bucket : String The bucket name@oid : String The unique object identifier
GET /db/:bucket/:oid info.orestes.server.resource.db.DbBucketOid : OrestesObject
## ...
CRUD.rest
queryschematransactiontransaction-crudtransaction-queryconfigurationpartial-updateregistration-login
Unified REST API Specification
Orestes-Server
public OrestesObject load(ObjectInfo objInfo)
Data Module forspecific DB
db/bucket/oid ResourceExample
Create a new object
> POST /db/:bucket
> JSON-Object
< 201 Created
Defined in REST specification
## Get objectReturns the specified object...
@bucket : String The bucket name@oid : String The unique object identifier
GET /db/:bucket/:oid info.orestes.server.resource.db.DbBucketOid : OrestesObject
## ...
CRUD.rest
queryschematransactiontransaction-crudtransaction-queryconfigurationpartial-updateregistration-login
Unified REST API Specification
Orestes-Server
public OrestesObject load(ObjectInfo objInfo)
Data Module forspecific DB
db/bucket/oid ResourceExample
Load object
> GET /db/:bucket/:oid
< JSON-Object
Defined in REST specification
## Get objectReturns the specified object...
@bucket : String The bucket name@oid : String The unique object identifier
GET /db/:bucket/:oid info.orestes.server.resource.db.DbBucketOid : OrestesObject
## ...
CRUD.rest
queryschematransactiontransaction-crudtransaction-queryconfigurationpartial-updateregistration-login
Unified REST API Specification
Orestes-Server
public OrestesObject load(ObjectInfo objInfo)
Data Module forspecific DB
db/bucket/oid ResourceExample
Perform Paginated Query
> GET /db/:bucket?query
&start=0&count=-1
< List of matching IDs
Request-Response model limitsthroughput◦ Proposal: Multipart Caching
◦ Multiplexing is core of HTTP/2.0
Timestamps have [s]-precision◦ Proposal: ISO-8601 dates
Networking infrastructure bugs◦ Solved: Non-persistent methods
◦ Solved: TCP-Deadlock caused byDelayed ACK and Nagle interaction
Put /db/…If-Unmodified-Since: Wed, 17 Sep 2014 09:46:56 GMT
Time 1 2 3
200ms ± 5ms
6x
Infrastructure-as-a-Service Cloud
REST-Server
Scalable Database System
HTTP
REST-Server REST-Server
DB protocols
Global Content Delivery Network
Browser
Mobile
Client
Browser
ISP
REST-API Transactions
Schema Management
Cache Consistency
Auto-Scaling Multi-Tenancy
Security and Access Control
Load Balancer
CacheCachescale,
invalidate
scale
Proxy CachesMobile
Client
Application
Server
Platform-as-a-Service Cloud
Access Control (Authorization)
Data Validation, Stored Procedures◦ OnUpdate, OnCreate, OnDelete, Validate Handler
◦ Same JS API as client
◦ Executed in Node.JS process in isolated Docker container
User Orestes
PUT db/posts/{id}User-TokenJSON Object
MongoDB
db.posts.update(…,_acl.allow:{$in:…_acl.deny:{$nin:…)
schema-level ACLcheck using Token
object-level ACLcheck through condition
Schema Management
Optimistic Transactions◦ Orestes-Servers perform version checks on write-sets (BOCC+)
◦ Applicable to all systems with consistent read option
Coherent web-caching
Annotation-based polyglot persistence
Coordinator
Server 1
Server 2
Schema
Field-TypeMapping
• Primitive • Collection • Reference• Embedded• JSON
2PC
Validate schema for everyoperation on objects
Client Expiration-based Caches
Invalidation-based Caches
Server
Bloom filter of stale objects for consistent caching
On connect: piggyback Cache SketchFor each request:Check in Cache Sketch if object isstale Cache Hit: fresh objects
Revalidation: stale objects
Write object
-Add object toCache Sketch forhighest previousTTL-Invalidate only ifnon-expired
Invalidate
Cache Hit Maximization
Invalidation Minimzation
application
Orestes servers
Redis MongoDB db4o
Schema annotations contain SLA
parse SLA & route data
manage materialisation
resolve mapping
Polyglot Persistence Mediator
application
Orestes servers
Redis MongoDB db4o
Schema annotations contain SLA
parse SLA & route data
manage materialisation
resolve mapping
Polyglot Persistence Mediator
Results:Article-Objects with Impression Count
Article
IDTitle…
Imp.
Imp.ID
MongoDB Redis Sorted Set
Speedup with PPM:• 50-1000% (depending on throughput)• 66% performance of Varnish
Client Machine
50 ...
Web
CacheOrestes
Server
Versant
DB
Amazon EC2 Ireland EC2 USA165 ms
Client Machine
Client Machine
30 000 Objekte
500 Anfragen/
Client
30 000 Objects500 Req./Clients10/1 Read/Write
Client Machine
50 ...
Web
CacheOrestes
Server
Versant
DB
Amazon EC2 Ireland EC2 USA165 ms
Client Machine
Client Machine
30 000 Objekte
500 Anfragen/
Client
30 000 Objects500 Req./Clients10/1 Read/Write
Backend-as-Service Abstractions
Database-as-Service Abstractions
Users & Authorization
Standard Schemas
Handlers & Stored Proc.
Schema Management
CoherentCaching
PolyglotPersistence
Unified REST API◦ „Middleware“ design to enhance existing database systems
◦ Decouples applications from concrete systems
◦ Adds critical functional and non-functional capabilities
Outlook
Unfied REST API:◦ Scalable Streaming Queries
Caching:◦ Adaptive Cache-Expirations for optimal Cache
Sketches
Polyglot Persistence:◦ Design of functional- and non-functional
annotations & mediator implementation
Transactions:◦ Database-independent validation, stochastic
analysis of Cache Sketch effectsCache
Cache
Cache
REST-Server
REST-Server
REST-Server
DB
Coordinator
validation
Begin Transaction
Bloom Filter
Reads
Writes
Commit: read- & write-set versions
Committed OR aborted + stale objects
1
2
3
4
5
Writes
(Hidden)
Writes (Public)
Client
prevent conflicting
validations
Read all
scale to high commit throughputs
and ensure availability
Nonfunctional RequirementsFunctional Requirements
Scan-Querys
Conditional Updates
Transactions
Filter-Querys
Joins
Analytics
Elasticity
Consistency
Read-Latency
Write-Latency
Write-Throughput
Scalability of Data Volume
Scalatbility of Request Load
Read-Availability
Write-Availability
Data model
Backend Systems
Constraints
Automized
Choice
Model Class
MongoDB Redis Db4o Cassandra HBase
Mediator
REST-Server
REST-Server
Event Server
[SELECT [* | _id]]FROM <bucket>[WHERE <condition> [AND <condition> ...]][ORDER BY, LIMIT, OFFSET]
1
2
Client
Write
Matching Objects (Streamed)
Match registered queries
Forward object3
Register
Client
Expiration-based Caches
Invalidation-based Caches
Server Cache Sketch
Stal
enes
s-M
inim
izat
ion
Mo
deIn
valid
atio
n-M
inim
izat
ion
Mo
de
10201040
10101010
Write Counting Bloom Filter
Expiration Priority Queue
RequestPath
Server/DB
Invalidations,Objects Report Expirations
and Writes
Needs Invalidation?
Client Cache Sketch
atconnect
Periodicevery t
seconds
attransaction
begin
CacheHits
10101010 Bloom filterNeeds Revalidation?
Browser Cache, Forward Proxy, ISP Caches
Content Delivery Networks, Reverse Proxies
1
2 3
4
OptimisticClient Caching
2StalenessBounding
1
3ConflictAvoidance
4InvalidationMinimization
Team: Felix Gessert, Florian Bücklers, Hannes Kuhlmann, Malte Lauenroth, Michael Schaarschmidt
19. August 2014
Internet
SeoxyDesktop
Mobile
Tablet
Orestes as a startup
