Cloud Design Patterns – part 2Masashi NarumotoAzureCAT patterns & practicesMicrosoft

RetryCircuit BreakerQueue based load levelingThrottlingLeader ElectionValet Key

Agenda

Retry problem – access to resource fails

Web siteUser Remote resource

Error

Retry solution

Web siteUser Remote resource

HTTP 500

HTTP 500

HTTP 200

When to retry?interval/count?

Service Retry capabilities Policy configuration Scope Telemetry features

AzureStorage Native in client Programmatic Client and individual operations TraceSource

SQL Database with Entity Framework

Native in client Programmatic Global per AppDomain None

SQL Database with ADO.NET

Topaz* Declarative and programmatic

Single statements or blocks of code Custom

Service Bus Native in client ProgrammaticNamespace Manager, Messaging Factory, and Client

ETW

Cache Native in client Programmatic Client TextWriterDocumentDB Native in service Non-configurable Global TraceSource

Search Topaz* (with custom detection strategy)

Declarative and programmatic Blocks of code Custom

Active Directory Topaz* (with custom detection strategy)

Declarative and programmatic Blocks of code Custom

Retry solution

Only retry transient errorsRetry strategy per contextAvoid cascading retryUse built-in retry logicTest/Log retry operations

Retry design considerations

https://github.com/mspnp/azure-guidance/blob/master/Retry-Service-Specific.md

Circuit Breaker problem - remote service causes cascading failure

Web siteUser Remote service

error

Retry consume memory, thread, network connection etc.

Circuit Breaker solution

Exception handling in Open stateTransition to Half-Open stateManual overwriteUnderstand the protected resourcesAccelerated Circuit BreakingNG for local resources

Circuit Breaker design considerations

Queue Based Load Leveling problem　Surge in requests exhaust resources

UserDatabase

XWeb Site

Queue Based Load Leveling solution

Database

Use queue as a buffer

Rate of requests Rate of processing

Web site

Reduce the rate that resource can handleIdentify # of queue/worker by testsUse reply queue when neededLatency will increase

Queue Based Load Leveling considerations

Throttling problem - surge in workload

Web siteUser

Storage

Multi-tenant

Throttling solution

Design the strategy early onPerform quicklyCan be used together with auto-scalingAggressive auto-scaling if demand grow very quickly

Throttling design considerations

Leader Election problem – control distribution

Storage Split/Shuffle

Mapper/Reducer

Leader Election solution

１． Try to acquire blob lease２． First one that acquires it is the leader３． Leader dispatches task to other instances４． Leader renews the lease

Make the election process resilientReplace leader when it goes downDistributed mutex could be SPOFConsider the leader may be removed by auto-scaling

Leader Election design considerations

Valet Key problem – protect storage

ApplicationUser StorageMalicious user

Valet Key Solution

Manage the key validity and controlBe aware you lose some controlValidate/Sanitize uploaded dataEnsure start time is earlier than currentGood for offloading from compute resource and reducing costs

Valet Key design considerations

✔

✔ ✔

✔

✔

✔

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Index Table problem Lack of 2ndary index

Query：Customers in Redmond?Customers(name=‘smith’)?

Index Table solution

Create another table with ‘town’ as index

Pros:- Query is fast since the new table has all dataCons:- Table size becomes large- Data consistency issue

Index Table solution

Pros:- Table size is small- Less consistency issuesCons:- 2 step lookup makes query slow

Create index only table

Index Table solution

Partially normalized table

Pros:- Query is fast- Table size is smallCons:- Depends heavily on Data access pattern

Index Table solution

Combined key ‘Town’_’Name’

Maintaining tables brings overhead Total table size will increase Data consistency issuesIndex skew problem

Index Table considerations

Static Content Hosting problem – storing static content　

cspkg

HTMLJava Script, CSS, Images

Request

DevOpsUserWeb site

Static Content Hosting solution　

cspkgHTML w/ link to content

request

DevOpsUser

cspkgJava Script, CSS, ImagesStatic content

Web site

CDN/Blob

Deployment and version controlVersion checkCache control/compressionStatic web siteEnable CORS for script

Static Content Hosting design considerations