Upload
andreas-grabner
View
142
Download
2
Embed Size (px)
DESCRIPTION
Software Performance Metrics that you should look at throughout your Build Pipeline and not just when your app crashes in productiong. Find performance and scalability problems as soon as executing your first Unit Test. Simply focus on metrics such as #SQLs, #LogMessages, #Objects on Heap, ...
Citation preview
1
Checking Performance Along your Build Pipeline
@grabnerandihttp://apmblog.compuware.com
http://bit.ly/dttrial
2
Today we are here for …
Commit Stage• Compile• Execute Unit Test• Code Analysis• Build installers
Automated Acceptance
Testing
Automated Capacity Testing
Manual testing• Key showcases• Exploratory testing Release
Cont. Delivery / Deploy
3
Who is doing it? How many successful deployments can they do?
300 Deployments / Year
50-60 Deployments / Day
10+ Deployments / Day
Every 11.6 seconds
4
More on Amazons Story
75% fewer outages since 2006
90% fewer outage minutes
~0.001% of deployments cause a problem
Instantaneous automatic rollback
Deploying every 11.6s
5 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
Recommended BookRecommended Book
https://itrevolution.wufoo.com/forms/phoenix-project-ebook-offer/
66 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
Status Quo: Unreliable Software Impacts BusinessStatus Quo: Unreliable Software Impacts Business
7
The “War Room” – back then
'Houston, we have a problem‘NASA Mission Control Center, Apollo 13, 1970
The “War Room” – back then
'Houston, we have a problem‘NASA Mission Control Center, Apollo 13, 1970
8
The “War Room” – NOW
Facebook – December 2012
The “War Room” – NOW
Facebook – December 2012
9 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE
Status Quo: Bugfixing in Production happensStatus Quo: Bugfixing in Production happens
~80% of problems
caused by ~20% patterns
YES we know this
80% Dev Time in Bug Fixing
$60B Defect Costs
BUT
10
4 Situations on
WHY this happens,
HOW to avoid it
4 Situations on
WHY this happens,
HOW to avoid it
11
12
“Blindly” (Re)use Existing
Components
13
Requirement: We need a report
14
Using Hibernate results in 4k+ SQL Statements to display 3 items!
Hibernate Executes 4k+ Statements
Individual Execution VERY
FAST
But Total SUM takes 6s
15
Requirement: We need a fancy UI
16
Using Telerik Controls Results in 9s for Data-Binding of UI Controls
#1: Slow Stored ProcedureDepending on Request
execution time of this SP varies between 1 and 7.5s
#2: 240! Similar SQL StatementsMost of these 240! Statements are
not prepared and just differ in things like Column Names
17
Metrics: # Total SQLs# SQLs / Web Request# Same SQLs / Request
Transferred Rows
Test: With realistic Data
Dev: “Learn” Frameworks
18
19
Implementation Flaws
20
Business Impact requires Action!
21
Solution: Cache to the RESCUE!!
22
Implementation and Rollout
Implemented InMemory Cache
Worked well in Load Testing
23
Result: Out of Memory Crashes!!
Still crashes
Problem fixed!Fixed Version Deployed
24
Metrics: Heap Size, # Objects Allocated,# Objects in Cache
Cache Hit Ratio
Test: With realistic Data
25
12 000 000 $
26
#No “Agile” Deployment
27
Ad on air
Availability dropped to 0%
Load Spike resulted in Unavailability
28
Alternative: “GoDaddy goes DevOps”
Response time improved 4x
1h before SuperBowl KickOff
1h after Game ended
29
Behind the Scenes
30
Metrics: AvailabilityPage Size, # Objects
# Hosts, # Connections
DevOps: “Feature” Switches
31
32
#Push without a Plan
33
Mobile Landing Page of Super Bowl Ad
434 Resources in total on that page:230 JPEGs, 75 PNGs, 50 GIFs, …
Total size of ~ 20MB
34
m.store.com redirects to www.store.com
ALL CSS and JS files are redirected to the www domain
This is a lot of time “wasted” especially on high latency mobile
connections
35
Metrics: Load Time, # Resources (Images, …),
# HTTP 3xx, 4xx, 5xx
Dev: Build for Mobile
Test: Test on Mobile
Ops: Monitor Mobile
36
37
#1: Define Measures# of Requests / User
# of Log Messages
# of Exceptions
# Objects Allocated
# Objects In Cache
Cache Hit Ratio
# of Images
# of SQLs
# SQLs per RequestAvailability
# HTTP 3xx, 4xx
Page Size
38
#2: Automate
Commit Stage• Compile• Execute Unit Test• Code Analysis• Build installers
Automated Acceptance
Testing
Automated Capacity Testing
Manual testing• Key showcases• Exploratory testing Release
Unit & Integration Tests
Functional Tests
Performance TestsProductionMonitoring
Functional Tests
How? Performance Focus in Test AutomationAnalyzing All Unit / Performance Tests
Analyze Perf Metrics
Identify Regressions
How? Performance Focus in Test Automation
How? Performance Focus in Test Automation
Cross Impact of KPIs
Identify Regressions
4242
#3: Share Results
43
#4: Integrate
44
If we do all that
CANCELLED
46
FREE Products & More Info
dynaTrace Free Trial30 Days Free Trial on http://bit.ly/dttrial After that totally free for local usage
Our Blog: http://apmblog.compuware.com
Follow Me: @grabnerandi