8/14/2019 Life of an App Engine Request
1/53
8/14/2019 Life of an App Engine Request
2/53
From Spark Plug to Drive Train:Life of an App Engine Request
Alon Levi05/27/09Post your questions for this talk on Google Moderator:code.google.com/events/io/questions
8/14/2019 Life of an App Engine Request
3/53
Designing for Scale and Reliability
App Engine: Design Motivations
Life of a Request
Request for static contentRequest for dynamic content
Requests that use APIs
App Engine: Design Motivations, Recap
App Engine: The Numbers
Agenda
8/14/2019 Life of an App Engine Request
4/53
Designing for Scale and Reliability
8/14/2019 Life of an App Engine Request
5/53
Google App Engine
8/14/2019 Life of an App Engine Request
6/53
LiveJournal circa 2007From Brad Fitzpatrick's USENIX '07 talk:"LiveJournal: Behind the Scenes"
8/14/2019 Life of an App Engine Request
7/53
From Brad Fitzpatrick's USENIX '07 talk:"LiveJournal: Behind the Scenes"
LiveJournal circa 2007
Frontends StorageApplicationServers
Memcache
Static File Servers
8/14/2019 Life of an App Engine Request
8/53
Basic LAMP
Linux, Apache, MySQL,Programming Language
Scalable?Shared machine for databaseand webserver
Reliable?Single point of failure (SPOF)
8/14/2019 Life of an App Engine Request
9/53
Database running on aseparate server
RequirementsAnother machine plusadditional management
Scalable?Up to one web serverReliable?
Two single points of failure
Dedicated Database
8/14/2019 Life of an App Engine Request
10/53
Benefits:
Grow traffic beyond the capacity of onewebserverRequirements:More machinesSet up load balancing
Multiple Web Servers
8/14/2019 Life of an App Engine Request
11/53
Load Balancing: DNS Round Robin
Multiple Web Servers
Register list of IPs with DNS
Statistical load balancingDNS record is cached with Time To Live (TTL)
TTL may not be respected
8/14/2019 Life of an App Engine Request
12/53
Register list of IPs with DNS
Statistical load balancingDNS record is cached with Time To Live (TTL)
TTL may not be respected
Load Balancing: DNS Round Robin
Multiple Web Servers
Now wait for DNS changes to propagate :-(
8/14/2019 Life of an App Engine Request
13/53
Scalable?
Add more webservers as necessaryStill I/O bound on one databaseReliable?
Cannot redirect traffic quicklyDatabase still SPOF
Load Balancing: DNS Round Robin
Multiple Web Servers
8/14/2019 Life of an App Engine Request
14/53
Benefits:
Custom RoutingSpecializationApplication-level load balancing
Requirements:More machines
Configuration and code for reverse proxies
Reverse Proxy
8/14/2019 Life of an App Engine Request
15/53
Scalable?
Add more web serversSpecializationBound by
Routing capacity of reverse proxyOne database server
Reliable?Agile application-level routingSpecialized components are more robustMultiple reverse proxies requires network-level routing
DNS Round Robin (again)Fancy network routing hardware
Database is still SPOF
Reverse Proxy
8/14/2019 Life of an App Engine Request
16/53
Master-Slave Database
Benefits:
Better read throughputInvisible to application
Requirements:
Even more machinesChanges to MySQL
8/14/2019 Life of an App Engine Request
17/53
Master-Slave Database
Scalable?Scales read rate with # of servers
But not writes
What happens eventually?
8/14/2019 Life of an App Engine Request
18/53
Master-Slave Database
Reliable?Master is SPOF for writesMaster may die before replication
8/14/2019 Life of an App Engine Request
19/53
Partitioned Database
Benefits:Increase in both readand write throughput
Requirements:Even more machinesLots of managementRe-architect data model
Rewrite queries
8/14/2019 Life of an App Engine Request
20/53
The App Engine Stack
8/14/2019 Life of an App Engine Request
21/53
App Engine:Design Motivations
8/14/2019 Life of an App Engine Request
22/53
Design Motivations
Build on Existing Google Technology
Provide an Integrated Environment
Encourage Small Per-Request Footprints
Encourage Fast Requests
Maintain Isolation Between Applications
Encourage Statelessness and Specialization
Require Partitioned Data Model
8/14/2019 Life of an App Engine Request
23/53
Life of a Request:Request for Static Content
8/14/2019 Life of an App Engine Request
24/53
Request for Static Content on Google Network
Routed to the nearest Google datacenterTravels over Google's network
Same infrastructure other Google products useLots of advantages for free
8/14/2019 Life of an App Engine Request
25/53
Request for Static Content
Google App Engine Front Ends
Load balancingRouting
Frontends route static requests to specialized servinginfrastructure
Routing at the Front End
8/14/2019 Life of an App Engine Request
26/53
Request for Static Content
Google Static Content Serving
Built on shared Google Infrastructure
Static files are physically separate from code files
How are static files defined?
Static Content Servers
8/14/2019 Life of an App Engine Request
27/53
What content is static?
Request for Static Content
...
8/14/2019 Life of an App Engine Request
28/53
Request For Static Content
Back to the Front End and out to the user
Specialized infrastructure
App runtimes don't serve static content
Response to the user
8/14/2019 Life of an App Engine Request
29/53
Life of a Request:Request for Dynamic Content
8/14/2019 Life of an App Engine Request
30/53
Request for Dynamic Content: New Components
App Servers
Serve dynamic requestsWhere your code runs
App MasterSchedules applicationsInforms Front Ends
App Servers and App Master
8/14/2019 Life of an App Engine Request
31/53
Request for Dynamic Content: Appservers
Many applications
Many concurrent requestsSmaller app footprint + fast requests = more apps
Enforce IsolationKeeps apps safe from each other
Enforce statelessnessAllows for scheduling flexibilityService API requests
What do they do?
8/14/2019 Life of an App Engine Request
32/53
Request For Dynamic Content
Front Ends route dynamicrequests to App Servers
Routing at the Frontend
8/14/2019 Life of an App Engine Request
33/53
1. Checks for cached runtimeIf it exists, no initialization
2. Execute request3. Cache the runtime
System is designed to maximize caching
Slow first request, faster subsequent requestsOptimistically cache data in your runtime!
Request for Dynamic Content
App Server
8/14/2019 Life of an App Engine Request
34/53
Life of a Request:Requests accessing APIs
8/14/2019 Life of an App Engine Request
35/53
App Server
1. App issues API call2. App Server accepts3. App Server blocks runtime4. App Server issues call5. Returns the response
Use APIs to do things youdon't want to do in yourruntime, such as...
API Requests
8/14/2019 Life of an App Engine Request
36/53
APIs
8/14/2019 Life of an App Engine Request
37/53
Distributed in-memory cachememcacheg
Also written by Brad Fitzpatrickadds: set_multi, get_multi, add_multi
Optimistically cache for optimizationVery stable, robust and specialized
Memcacheg
A more persistent in-memory cache
8/14/2019 Life of an App Engine Request
38/53
The App Engine Datastore
Persistent storage
8/14/2019 Life of an App Engine Request
39/53
The App Engine Datastore
Your data is already partitionedUse Entity Groups
Explicit Indexes make for fast readsBut slower writes
Replicated and fault tolerantOn commit: 3 machinesGeographically distributed
Bonus: Keep globally unique IDs for free
Persistent storage
8/14/2019 Life of an App Engine Request
40/53
GMail
GoogleAccounts
Picasaweb
Other APIs
Gadget API
On roadmap
Google Talk
8/14/2019 Life of an App Engine Request
41/53
App Engine:
Design Motivations, Recap
8/14/2019 Life of an App Engine Request
42/53
creative commons licensed photograph: http://www.flickr.com/photos/cote/54408562/
Build on Existing Google Technology
8/14/2019 Life of an App Engine Request
43/53
Provide an Integrated Environment
Why?
Manage all apps togetherWhat it means for you:
Follow best practicesSome restrictionsUse our tools
Benefits:Use our toolsAdmin ConsoleAll of your logs in one place
No machines to configure or manageEasy deployment
8/14/2019 Life of an App Engine Request
44/53
Encourage Small Per-Request Footprints
Why?
Better utilization of App ServersFairness
What it means for your app:Less Memory UsageLimited CPU
Benefits:Better use of resources
8/14/2019 Life of an App Engine Request
45/53
Encourage Fast Requests
Why?
Better utilization of appserversFairness between applicationsRouting and scheduling agility
What it means for your app:Runtime cachingRequest deadlines
Benefits:Optimistically share state between requestsBetter throughput
Fault toleranceBetter use of resources
8/14/2019 Life of an App Engine Request
46/53
Maintain Isolation Between Apps
Why?
SafetyPredictability
What it means for your app:Certain system calls unavailable
Benefits:
SecurityPerformance
8/14/2019 Life of an App Engine Request
47/53
Encourage Statelessness and Specialization
Why?
App Server performanceLoad balancingFault tolerance
What this means for you app:Use API calls
Benefits:Automatic load balancingFault toleranceLess code for you to write
Better use of resources
8/14/2019 Life of an App Engine Request
48/53
Require Partitioned Data Model
Why?
The DatastoreWhat this means for your app:
Data model + IndexesReads are fast, writes are slower
Benefits:
Design your schema onceNo need to re-architect for scalability
More efficient use of cpu and memory
8/14/2019 Life of an App Engine Request
49/53
Google App Engine:
The Numbers
8/14/2019 Life of an App Engine Request
50/53
Google App Engine
Currently, over80K applications
Serving over140M pageviews per day
Written by over200K developers
8/14/2019 Life of an App Engine Request
51/53
Open For Questions
The White House's "Open For Questions" application
accepted 100K questions and 3.6M votes in under48 hours
8/14/2019 Life of an App Engine Request
52/53
Questions?
Post yourquestions for this talk on Google Moderator:code.google.com/events/io/questions
8/14/2019 Life of an App Engine Request
53/53