How The Container Store uses

AppDynamics in their development life cycle August Azzarello, The Container Store

Our Presenter

August Azzarello Software Quality Engineer The Container Store

Agenda

•  The Container Store

•  Challenges

•  Solutions

•  Best Practices

•  Benefits

•  Q&A

•  Leading retailer of storage and organization products •  Founded in 1978 – 77 stores nationwide •  FORTUNE magazine’s 100 Best Companies To Work For®

www.containerstore.com www.whatwestandfor.com

Challenges

•  APM solution used as reactive monitoring tool

•  Customer impacting issues already introduced

•  No insight into test environment, lack of reporting on automated testing

Solutions

•  Install AppDynamics in test environments

•  Build automated functional and performance test suites

•  Baseline performance and functional metrics, alert on deviations

•  Build BT and alert policies in test

Transaction Score Easily identify trends

Application Dashboard – Flow Map Perform targeted testing by identifying which applications talk to each other

Custom Dashboards Complete visibility into application environment

Compare Releases Compare important metrics between deployments

Metric Browser and Custom Reports Any metric easily becomes a custom report

•  Selenium (Ruby) for Functional Testing

•  Locust.io (Python) for Performance Testing

Tools Used for Automated Testing

BEST PRACTICES

Best Practices

•  Monitor everything –  If you can’t see it, you can’t test or monitor it –  Make health rules in test, promote to production –  BT Maintenance

•  Test continuously –  Continuous tests will help build baselines, better the ability to realize deviations on

performance and functional errors –  Dedicated automation environment

•  Performance test early in development life cycle –  The earlier performance problems are detected, the less impact they will have on the release

cycle •  Empower development & QA team members

–  AppDynamics can give the ability for QA members to give more in depth bug reports, forging strong team relationships and increasing efficiency

Key Metrics to Monitor in Test •  Performance Testing

–  Infrastructure •  CPU •  Disk •  Memory •  Network

•  Functional Testing –  Application Metrics

•  Errors/Exceptions •  HTTP Error Codes

•  Number of Calls –  Calls to DB(s) –  Calls to backend services –  Calls to remote services

Copyright © 2014 AppDynamics. All rights reserved. 15

Expedite Remediation of Performance Issues

Copyright © 2014 AppDynamics. All rights reserved. 16

•  Auto-discover & monitor end-to-end business transaction performance within minutes, with no overhead

•  Self-learning platform dynamically computes optimal app performance

•  Resolve application issues quickly for exceptional end-user experience

Production Readiness in Pre-Production

•  Test for desirable performance in pre-production using same tool

•  Deep transaction traces and snapshots to find code issues

•  Set-up automated notification rules for performance testing

Copyright © 2014 AppDynamics. All rights reserved. 17

Reduce Cost and Risk with Automated Change Analysis •  Compare performance snapshots before

and after a major code release (or individual bug fixes)

•  Understand the performance impact of application, server/database changes, and with infrastructure changes

•  Identify and resolve unplanned changes proactively

Copyright © 2014 AppDynamics. All rights reserved. 18

Benefits

•  Set performance expectations before production deployments – ~40% improvement since we started using AppDynamics in test

•  Fine tune alert and BT policies prior to production deployments

•  Identify testing requirements, and testing gaps

•  Decrease performance test result reporting from 5 hours to 20 minutes

Copyright © 2015 AppDynamics. All rights reserved. 19

Q&A

Copyright © 2015 AppDynamics. All rights reserved.

20

Thank You