Upload
garry-sparks
View
214
Download
0
Embed Size (px)
Citation preview
PowerPoint Presentation
Jason Baker, Chris Bond, James C. Corbett, JJ Furman, Andrey Khorlin, James Larson,JeanMichel Leon, Yawei Li, Alexander Lloyd, Vadim YushprakhMegastoreScalable Highly Available Storage for Interactive Systems
Presented By:
Hamid SeyedmoradiAyoub HamidiEhsan Mohamad Nezamian
Advanced Database Systems SRBIAU, Kurdistan Campus10May2012
MegastoreMotivationIntroductionNoSQL & RDBMSMegastorePaxos
SRBIAU, Kurdistan Campus Advanced Database Systems2
Megastorewow! more than 3 billion write and 20 billion read daily
key ContributionData Model and Storage SystemPaxos ReplicationReport on Experience
SRBIAU, Kurdistan Campus Advanced Database Systems3
AVAILABILITY & SCALEReplication For Availability, we implemented a synchronous, fault tolerant log replicator optimized for long distance links Partitioning and Locality for scale, we partitioned data into a vast space of small databases
SRBIAU, Kurdistan Campus Advanced Database Systems4
AVAILABILITY & SCALEReplicationStrategiesAsynchronous Master/SlaveSynchronous Master/SlaveOptimistic ReplicationWe decided to use Paxos
SRBIAU, Kurdistan Campus Advanced Database Systems5
Technology OptionsSRBIAU, Kurdistan Campus Advanced Database Systems
6
SRBIAU, Kurdistan Campus Advanced Database Systems
7
Technology OptionsAVAILABILITY & SCALEPartitioning and LocalityReplication
SRBIAU, Kurdistan Campus Advanced Database Systems8
DatacentersEntity Groups Partition the datastoreEach entity group is synchronously replicated across datacentersACID semantics within an entity group Looser consistency across entity groups Entity group data and replication metadata stored in scalable NoSQL datastores8AVAILABILITY & SCALEPartitioning and LocalityOperations:
SRBIAU, Kurdistan Campus Advanced Database Systems9
Entities (Units of data)Entity Group 1Most transactions are within a single entity group Cross Entity group transactions supported via Two Phase CommitAsynch communication between entity groups supported by QueuesGlobal Indexes span entity groups but have weaker consistencyEntity Group 2Local IndexLocal IndexSendqueuereceive9AVAILABILITY & SCALEPartitioning and LocalityEntity GroupsSelecting Entity Group BoundariesExampleEmailBlogsPhysical Layout
SRBIAU, Kurdistan Campus Advanced Database Systems10
MegastoreAPI Design PhilosophyData ModelPre-Joining with KeysSCATTERIndexesStoring ClauseRepeated Indexes.Inline IndexesMapping to Bigtable
SRBIAU, Kurdistan Campus Advanced Database Systems11
Megastore
SRBIAU, Kurdistan Campus Advanced Database Systems12
MegastoreTransactions and Concurrency ControlReadcurrent snapshot inconsistentTransaction Lifecycle1-Read 3-Commit 5-Clean up2-Application logic 4-ApplySRBIAU, Kurdistan Campus Advanced Database Systems13
MegastoreQueues
Two Phase Commit
SRBIAU, Kurdistan Campus Advanced Database Systems14
REPLICATIONBrief Summary of PaxosMegastores ApproachFast ReadsFast WritesReplica Types Witness ReplicaArchitectureSRBIAU, Kurdistan Campus Advanced Database Systems15
ArchitectureSRBIAU, Kurdistan Campus Advanced Database Systems16
Data Structures and AlgorithmsReplicated LogsSRBIAU, Kurdistan Campus Advanced Database Systems17
Data Structures and AlgorithmsReadsQuery LocalFind PositionLocal readMajority readCatchupValidateQuery DataSRBIAU, Kurdistan Campus Advanced Database Systems18
Data Structures and AlgorithmsSRBIAU, Kurdistan Campus Advanced Database Systems19
Data Structures and AlgorithmsWritesAccept LeaderPrepareAcceptInvalidateApplySRBIAU, Kurdistan Campus Advanced Database Systems20
FeedbackSRBIAU, Kurdistan Campus Advanced Database Systems21
ENDWith ThanksQuestion?SRBIAU, Kurdistan Campus Advanced Database Systems22