Embed Size (px)
Citation preview
Cloud Database Scalability – Designing and Building for Scale in Windows Azure SQL DatabaseScott KleinTechnical Evangelist
AZR311
Agenda
• Scalability• Architecture• Management• Best Practices
Goals
Explain the importance of database scalability and the challenges of scaling in the cloud.
Show the major components necessary to approach designing for scale at the database layer.
Demonstrate the different approaches for designing a scalable database layer.
Scalability
Database ScalabilityScale-upSingle database that houses all the data of an applicationHard to handle peak loadOK with exponential incremental cost
Scale-OutMultiple databases spread over multiple independent nodesCost effective, commodity class hardwareTypical patterns: Sharding and Horizontal Partitioning
Scalability Model For The Cloud
Cloud ApplicationsRequire Scale Beyond Scale-UpDemand the Best Economics
Best Price/PerformanceElasticity + Pay-as-you-go
Challenges
Defining the TenantEstablishing Tenant’s surrogate keyEstablish Sharding Strategy (Range / Point)Elastic Scalability (Splits/Merges/Tenant Moving)Application Lifecycle Management (Dev; Test; Deploy; Upgrades)Overcoming limitations of existing tools & available featuresTransient nature of connectivity
Database ScalabilityCanonical 3 tier app scales by adding and removing nodes
Extend the model to the DB Tier.Add and Remove SQL Azure nodes. Scale on demand to your traffic
SQL AzureLarger User
Traffic
FrontTierFrontTierFrontTierFrontTier
FrontTierFrontTierFrontTierMiddle
Tier
Why use database partitioningGo Beyond Single DB to Practically Unlimited Scale:
Harness 100s of SQL Azure nodes
Best Economics Elastic database tier that can repartition with your applications workload. No downtime required!
Simplified Multi-tenancy: Efficient management of tenant placement and re-placement
Simplified Development and Administration: Robust programming & connectivity model with native tooling
Single tenant per databaseMultiple-tenants per database Multiple databases per tenant
Common Scalability Scenarios A Few ExamplesWeb Scale DB Solutions
Multi-tenant Saas ISVs
Workloads with Spikes, Bursts, Peaks, etc…
NoSQL Applications
Architecture
Assumptions & Jargon• Reasons for scale-out are understood
• Platform is Azure & Data store is SQL Database
• Workload Group refers to a particular workload (a.k.a Functional or vertical partitioning)
• Shard refers to a single data store (Database) within a workload group. (a.k.a Horizontal partitioning)
• Elastic Scale-out is the ability to add or remove workload groups & shards in response to resource demands.
• Range sharding refers to each shard covers a range of sharding key values
• Point sharding refers to the case where a shard has no shard key
ArchitectureConfiguration DatabaseContains shard map and configuration meta data informationLow Volatility Critical during expansion / contraction of shards
Workload GroupParticular workload (a.k.a Functional or vertical partitioning
ShardsSQL Database instances or SQL Databases that contain part, or “slices” of data
Shards
Workload Group
Config Database
Architecture Cont.Tenant KeyUsed for all non-reference table records
Used by almost all queries (indexes)
Int, bigint, GUID
Reference TableInt, bigint, GUID
Reference TableLookup table
RepartitioningDynamic PartitioningSPLIT members to spread workloads over to more nodes
DROP members to shrink back to fewer nodes
Shards
Workload Group
Config Database
Reliable Routing
SalesDB
Orders_federationOrders_federationOrders_Fed
[5000, 7500) & [7500, 10000)
Built-in Data-Dependent Routing (DDR)Ensure apps can discover where the data is just-in-time
No “Shard Map” caching
Guaranteed member routing
Demo
Name
Management
Using Dacpac packages• Centralize and manage deployments• Contributes to reducing complexity of upgrades• Deploy packages in parallel• Track deployment progress down to shard level• Programmatically deploy packages• Elasticity demands ability to dynamically adjust # deployments• Address limitations & differences of dacpac
Parallel Execution of Queries
• Parallel execution error tolerance must be defined• As # shard increases, so does probability of errors increase
• Strive towards zero to minimal need for fan-out
• Fan-out query and DML capability must exist
Demo
Name
Best Practices
Best Practices And Design ConsiderationsDefining the Tenant
Most granular as possibleTenant Size The smaller the better Current Max size (150 GB)Zero to minimal cross-database requirementsSharding key typically equates to TenantIDExamples: User (very common) Region Company Cost Center of Company
Best Practices Cont.Establishing Tenant Surrogate KeyUsed for all non-reference table recordsUsed by almost all queries (indexes)Ideal Key Small Fixed size Large domainGUID is common data type Too large Cause of severe fragmentation Painful sharding boundary values Painful reference when troubleshootingBigInt is better Half-size of GUID
Related contentAZR221 – WASD and WA SQL VM Concepts for DBAsAZR323 – Building Real World 8.1 apps with Mobile Services Deep DiveProduct Demo Stations at the Cloud Booth
Find Me Later At the Cloud booth
Developer Network
Resources for Developers
http://msdn.microsoft.com/en-au/
Learning
Virtual Academy
http://www.microsoftvirtualacademy.com/
TechNet
Resources
Sessions on Demand
http://channel9.msdn.com/Events/TechEd/Australia/2013
Resources for IT Professionals
http://technet.microsoft.com/en-au/
Track Resources• Sign up for a Free Azure Trial:
http://www.windowsazure.com/en-us/pricing/free-trial/
• Activate your Azure MSDN Benefits: http://msdn.microsoft.com/dn369243
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
