How to Use DevOps & APM to Release Better Software Faster

1 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #APMLive

How to Use DevOps & APM to Release Better Software Faster

Brett Hofer

Global DevOps Practice Lead

Dynatrace

@brett_solarch

Hasan Yasar

Technical Manager

Carnegie Melon SEI

@SEINews

© 2015 Carnegie Mellon University


Hasan YasarTechnical Manager

Software Engineering InstituteCarnegie Mellon University

3How to Use DevOps & APM to Release Better Software Faster© 2015 Carnegie Mellon University

DevOps ?Why


Known SDLC / Waterfall process


With Agile


Business

Research Budget Document Water-

Jez Humble, https://youtu.be/L1w2_AY82WY

Dave West, http://sdtimes.com/analyst-watch-water-scrum-fall-is-the-reality-of-agile/


Business

Research Budget Document Water-Development

Scrum-

Jez Humble, https://youtu.be/L1w2_AY82WY



Business

Research Budget Document Water-Development

Scrum-

Integrate Test Release FallJez Humble, https://youtu.be/L1w2_AY82WY


QA & Operations


Shared Goals CollaborationBusiness Needs

DevOps


Teams Together DevOps brings


Project Roles Have Different Functions

Developers Operations

Create

Change

Deliver

Maintain

Monitor

Manage Environment


Developers Favor Dynamic Behaviors


Create

Change

Deliver

Maintain

Monitor

Manage Environment


Operations Favors Static Behaviors


Create

Change

Deliver

Maintain

Monitor

Manage Environment



Create

Change

Deliver

Maintain

Monitor

Manage Environment

Where is the real pain?


Software Projects Are Complex

Deployment

Maintenance

Security

Programming

Infrastructure

Scalabilty

Networks

Functional

Requirements

Performance

Testing

User Interface

Technical

Documentation

Updates

Code Review

Release

Review User

Documentation

Data Privacy

Intrusion

Detection

User

Requirements

Business Constraints

Legal Issues

Market Needs

Budgets / Timelines


Information Security Expertise

Deployment

Maintenance

Security

Programming

Infrastructure

Scalabilty

Networks

Functional

Requirements

Performance

Testing

User Interface

Technical

Documentation

Updates

Code Review

Release

Review User

Documentation

Data Privacy

Intrusion

Detection

User

Requirements

Business Constraints

Legal Issues

Market Needs

Budgets / Timelines

Business Analyst Expertise

Quality Assurance Expertise

IT Operations Expertise

Developer Expertise


DevOps Breaks Down Silos: Eliminate pains


The DevOps Lifecycle : Faster release with visible feedback loop!

Collaboration between project team roles

Infrastructure as Code: Scripted Infrastructure Configuration

Automation of Tasks / Processes / Workflows

Performance Monitoring Applications and Infrastructure


Collaborative DecisionsDevOps enables


Goal: Eliminate risk in through SLDC /siloed Team


The DevOps lifecycle is Full of Decision Points

Without team knowledge, developers continuallymake uninformed decisions, causing eventual risk or inefficiency



How many users?

Payment model?

Who is the Target Market?

Which regions?



What OS?

What ports?

VMs or containers?

Entry points?



What kind of user authentication?

REST vs SOAP?

How to architect for scalability?


Uninformed Decisions Lead Your Project Down Suboptimal Paths

Developers Only Dev + Ops


DevOps enables

Infrastructure Automation


A Traditional Scenario

A system requires multiple servers networked together

Each server is to be provisioned uniquely, providing a specific service


A Traditional Scenario

At deployment, the deployment environment (Test / QA / Production)is manually provisioned to match the expected infrastructure configuration

This often means manual installation of

packages, configurations, and networking components…

…and room for human error


Divergence on deployment process

Environments are independent, volatile, and easily manipulated.

Without care, they will diverge.


Move into automation : Infrastructure as Code (IaC)A Code that creates infrastructure

* A concretely defined description of the environment is good material for conversation

between team members.


IaC Apparent Benefits

Because the environment and application are versioned together, tagging

code is that much more meaningful.


IaC Promotes Quality Attributes

The process that creates and configures the infrastructure for your application is, itself, an application.

As an application, environment creation and configuration is now:

• Automatable

• Repeatable

• Versionable

• Reviewable

• Diffable

• Testable (it works)

• Human-readable

• Verifiable (it is right)


IaC Has Operational Benefits

• Lower Risk

• Guaranteed Documentation

• Stable Rollback

• Fast Onboarding

• Time Savings

• 100% Environment Parity


Common DevOps enabled tools for :IaCChef or Puppet - wrappers around your shell

provide hooks and convenience methods

abstract operating system-specifics, provide portability

Ansible or Fabric - to automate apps and IT infrastructure

Docker - linux containers

runs on Linux only (for now)

entire environments can be easily shared

Shell scripts – simply scripting platform-specific commands

Vagrant - assists in managing virtual machines and executing provisioners


And brings

Quality!

DevOps improves

Defects Detection


Automating Integration Fixes These Issues

Automation…

Removes inefficiencies due to human-driven process

Standardizes artifact submission process

Guarantees consistent results

Allows team to fail fast (and fix fast)

Reduces pain of integration


Continuous Integration is Even Better

Continuous Integration uses a build server to…

Integrate artifacts on every change

Give team with immediate notification of failure or success

Require issues be fixed before moving forward

Enforce standards (can fail based on quality as well as functionality)


Fail the Build When Software is Not Good Enough

Don’t just configure failure for compile/build errors!

Want 90% test coverage? Fail the build if code base is <90% covered

Want all DB queries < 2sec? Test them, and fail the build otherwise

Want to make sure code conforms to style guide? You guessed it…

CI is your best tool to enforce quality standards


Continuous Integration Requires Some Discipline…

For successful implementation of a CI process:

Developers must commit changes often

CI system should build every commit

Automate every step of the build process

Automate tests, and fail the build on test failure

CI system should report results immediately to everyone

CI system should instantly revert to previous release on failure

All environments should have 100% parity


…But Yields Significant Rewards

Lower costs

• Immediate detection of problems• Removal of human integration labor

Greater visibility for everyone

• Developers• Operations• Quality Assurance• Management• Customers

Increased confidence in your software


Global Vision

Project Management

Tools

Issue Tracker

Wiki


DevOps Interoperability / Traceability - General


DevOps - Interoperability / Traceability –Reference Implementation


Human actions/inputs to the software development

process


Actions performed by autonomous

systems


And enables

Faster Release

DevOps reduces

Deployment Failures


Continuous Deployment

Changes are deployed ASAP into production

Continuous Delivery

Changes are deployed immediately into a production-like environment, to ensure that they could be deployed into production


Shift Left Operational Concerns


Shift Left Operational Concerns


Gold-standard deployment

• Environment Parity by IaC

• Use the same process to deploy to development, test, and

production

• Automate

• Perform incremental changes

• Appropriate Testing


Continuous Delivery Enforces 100% Parity

Environment inconsistencies between environments will result in risks

✔ ✔ ✔ ✖


Incremental Changes, Continues Testing, Early Performance Monitoring

• Smaller changes to production can be less risky than large ones

• Decrease the number of potential defects that could cause an outage

• Monitor the performance of application as early as possible.


Test Enough That You Are Sure You Could Deploy Successfully

How much is enough?

What factors are important to you and your organization?

Security? Automate a large number of security-focused unit/integration tests

Scalability? Automate high-volume load testing

Design your CI/CD success criteria to enforce your goals


Continuous Delivery Is REALLY About Rigorous Testing

Certainty requires rigorous testing before and after

deployment



Brett Hofer

Global DevOps Practice Lead

Dynatrace

@brett_solarch

Release at the Speed of Business • Increase speed to market while improving quality and performance

• Injecting DevOps practices into the Application Lifecycle

• Application Performance Management Best Practices


• Bake your APM strategy right into the application lifecycle

• Dev and Ops teams – improve processes!• Ensures Business Success Metrics are always visible &

understood throughout the entire Application lifecycle

• APM integration done in parallel with dev - minimal to no coding impacts yet increasing speed & quality

Release “at the speed of business”



• Dev and Ops teams – communicate effectively!

• Collaborate on the health of each build

• Comprehensive Dashboard strategy

• Consistent measurements between Dev, Test and Production

• Identify, prioritize and shift business goals as far left as possible

• Everyone understands them early on

• Monitor build state, KPIs, SLAs and risks throughout life cycle




• Optimize and automate issue feedback

• Faster feedback in Dev and Test = faster prioritization to fix

• Finding issues earlier and faster means cheaper

• Earlier in lifecycle = cheaper

• Delivering functionality faster means risk to experiencing issues in production

• Imperative to have a comprehensive APM strategy early to ensure root cause and fix lifecycle is reliable and fast

• Decreased MTTR at every leg of delivery means faster and more reliable delivery of functionality.



• Reducing cycle times/optimizing MTTR throughout

• Use APM best practices for reliable, accelerated deployments

• Integrating APM intelligence into deployment workflows

• Integrating APM throughout automated deployments (including elastic VM, cloud and container environments)



Thank-you!

Brett Hofer

@brett_solarch

http://blog.dynatrace.com

Hasan Yasar

@SEINews

https://insights.sei.cmu.edu/devops/

http://blog.dynatrace.com/

http://blog.dynatrace.com/




Technology

How to Use DevOps & APM to Release Better Software Faster