Dev Ops – Enabling Agility in Business

Rahul Soni – Project Manager

Cognizant Technology Solutions Pvt. Ltd.

Abstract

A recent study by Everest Group has revealed:

“24% organizations believe DevOps will be the de-facto development methodology of the future.”

This agile world demands continuous integration and continuous delivery where customer willingness, culture and maturity become the key to success. Other factors such as people skillset, DevOps tooling and framework, estimation methodologies and infrastructure scaling also play an important part in DevOps implementations.

The paper elaborates best practices of DevOps for continuous delivery and continuous integration:

1. Open sourced tooling strategy and lifecycle automation2. Customized DevOps frameworks, platforms and release models 3. API and test data automation4. Monitoring risk involved with each release5. Communication and collaboration

Adopting these principles can enable organizations to implement high-performing DevOps solutions that result in:

200 times more frequently releases 2,555 times faster lead times 24 times faster recovery 22% effort reduction on unplanned work and rework  29% increased bandwidth to focus on new features, code delivery and deployment

Dev Ops is a disruptive concept that has materialized fairly recently yet it is adored by so many people. In a crux, Dev Ops enables better integration of Development and Operations team and thus in turn achieving continuous integration and continuous delivery.

Every enterprise today is under pressure to respond quickly to changing market conditions. They want things to be delivered fast and with cost effective means. That’s where the enterprise responsiveness – Agility comes into play. Agility requires building robust infrastructure and processes to help deliver innovative applications to customers more quickly. There are lot of factors to discern if Dev Ops can be implemented for any organization. Customer willingness, culture and maturity are the key ingredients to make this happen. Apart from this, there are other aspects that need to be accounted like people skillset, dev ops tooling and framework, estimation methodologies and infrastructure scaling.

Key Business Drivers

Customer willingness and maturity Agility in Release with minimal manual intervention Reduced Time to Market Faster Deployment and turnaround cycles Effective Environment Management

There are few areas where Dev Ops is preferred more by organizations these days mainly Build and Deployment Automation, Release Management, Test Automation Enablement and Infrastructure Provisioning supplied by an effective Environment Management.

Best Practices

1) Open Sourced tooling strategy and Lifecycle Automation

These days customer is mainly focused on securing open source solutions and moving to lifecycle automation. As a starting point, there is a need to capture customer feedback, customer impact, frequent performed tasks, and their duration, failure rate in a tool. This will help to prioritize and plan tooling strategy.

Open Source tools like Git, Subversion, Jenkins, Nexus, Artifactory, Chef, Puppet are becoming quite popular. Specific tool strategy should be prepared with pros and cons where certain set of tools are identified. Jenkins tool easily integrates with build, test and deployment automation tools and plays a key role in lifecycle automation thus in turn enabling continuous integration and continuous delivery.

A common tooling strategy has to be agreed between Dev, QA and Ops that includes-

Communication and collaboration tools Colloborative development/build tools Continuous Integration tools Continuous Testing tools Continuous deployment tools

Communication tool automation will enable continuous feedback loops to fix issues quickly.

Apart from this, there is a need to identify tools to report aggregated metrics on manual and automation process.

Organisations these days focus on cloud platforms like On Premises or Off Premises. Overall objective is to automate everything possible.

And finally a set of acceptance tests for each of Dev Ops tools and process should be created to see if all fit in together in one eco system.

2) Customized Dev Ops Platform and Release models

Dev Ops Platform needs to be developed that suits customer needs. To start with, a proof of concept should be created to understand the customer needs and recommendations provided for tool selections or decommissioning (SCM replaced by GitHub, Nexus by Artifactory are few examples). Platform should be flexible and robust enough so that it keeps evolving and customizing based on customer inputs.  

Building blocks for Platform creation - Assess Enterprise DevOps Strategy

» Identify your DevOps business drivers, challenges, and initiatives» Examine critical success factors and best practices from successful projects» Review critical DevOps domains that align to your business goals» Identify priorities for based on duration of projects

Identify the DevOps maturity of core development and IT operations processes Adopt and implement change. Focus on the people, process and technology for key Dev Ops areas Measure progress and plan next improvement

Key differentiators of Platform: Change impact based testing Continuous Development On Demand monitoring of infra and apps Shift left performance testing Integrated process automation

Infra as a code Data integration Test design automation

Cloud Based Dev Ops Platform

Organizations these days are looking to move from Waterfall to Dev Ops but just Dev Ops will not work. There is a huge capital investment required in hardware and software set up and this in turn slows the overall application development and deployment process. Dev Ops with Cloud is the need of the hour. Dev Ops process and automation should be extended to Public and Private cloud. It enables speed-to-delivery to meet the business needs faster and with lower costs for development, testing, deployment and operations.

The centralized nature of cloud computing provides DevOps automation with a standard and centralized platform for testing, deployment, and production. Using a cloud platform solves many issues with distributed complexity.

Most public and private cloud computing providers support DevOps on their platform, including continuous integration and continuous development tools. This helps to lowers the cost associated with on-premises DevOps automation technology, and provides centralized governance and control for a sound DevOps process.

Clouds based Dev Ops leverages usage-based accounting, which tracks the use of resources by application, developer, user, data, etc. Traditional systems do not provide this service. When leveraging cloud-based resources, it's much easier to track costs of development resources and make adjustments accordingly.

The important thing is to understand development requirements, business value, define dev ops process and linkage to cloud platform, people and skill sets available and how applications will operate in the cloud.

Combination of Cloud and Dev Ops is like Marriage made in Heaven.

Few Cloud based Dev Ops Platforms are market leaders – Amazon’s AWS IBM’s Bluemix Microsoft’s Azure

These days, DevOps is driving the interest and the growth of cloud. As per Right Scale’s state of the Cloud Report, it was found - Overall DevOps adoption rises to 66 percent, with enterprises reaching 71 percent.

Containerization

Containers are a way to package software in a format that can run isolated on a shared operating system. Unlike VMs, containers do not bundle a full operating system - only libraries and settings required to make the software work are needed. This makes for efficient,

lightweight, self-contained systems and guarantees that software will always run the same, regardless of where it’s deployed. It is designed to run on everything from physical computers to virtual machines.

Containerization has gained recent prominence with the open-source Docker. Docker is the world’s leading software container platform and supports Linux, Windows Server, and Linux-on-mainframe apps. It automates the repetitive tasks of setting up and configuring development environments so that developers can focus on what matters i.e. building great software. When an app is dockerized, the complexity is pushed into containers that are easily built, shared and run. Onboarding a co-worker no longer means hours spent installing software and explaining setup procedures. Dependencies are pulled as neatly packaged Docker images and anyone with Docker and an editor installed can build and debug the app in minutes.

Benefits of Docker – Reduce onboarding time by 65% Quickly build, test and run complex multi-container apps Build, test, debug and deploy Linux and Windows Server container apps written in any

programming language without risk of incompatibilities or version conflicts. All dependencies run in containers, eliminating “works on my machine” problems.No risk of

incompatibilities or version conflicts. Built in Container Orchestration (Swarm Clustering that’s easy to configure).

Looking at containerization benefits, Microsoft is building its own containerization technology – Drawbridge to support Windows Server and Azure Cloud.

Automated Release Management (i.e. Automated Apps deployment) is one of the critical need of the clients these days. Overall release model has been broken down into smaller releases and each of these releases are integrated with Agile methodologies supported by Dev Ops tooling.

API and Test data Automation

Need for on-demand, relevant and ‘not-stale’ Test Data is more pertinent now than ever, with the advent of Continuous Integration and Continuous Deployment age (CI/CD), where number of iterations of automated tests are expected to be run. The context gets deeper with applications that demand ‘State’ based data, which requires identification, coverage patterns, mining, conditioning and repurposing, etc. There are multiple clients demanding different set of solutions for test data management especially it becomes more relevant in Dev Ops as multiple deployments are run each week.

In a typical DevOps environment, some of the significant TDM challenges include: Refresh of Test data in each deployment cycle is a tedious activity, particularly when

multiple deployments happen every week Regulatory compliance and data privatization are key challenges to protect sensitive

information while deploying data Too many clones of the production environment increase storage costs and cause larger

maintenance windows/downtime

Test data creation in some cases is a manual, time consuming, and error-prone process. Lack of knowledgeable resources makes it harder to test packaged applications

Proposed Solutions - Approach 1 – Taking a subset of the production data that is representative. Industry standard tools like for Informatica ILM & CA Datamaker do this job effectively.

Approach 2 – Synthetic Test Data Generation. Advantages of a completely synthetic approach is to avoid dependency on live data beyond original profiling, and therefore it is very quick and easy to generate new test data. Thus this is a particularly suitable approach for agile requirements.

Synthetic Data Generation is done either via Industry Standard Tools like Informatica ILM and CA Datamaker or via an API automation framework.

» API automation framework is a TDM tool which offers very innovative solutions in test data generation and is useful in Agile DevOps world. The APIs give project team complete flexibility in building unique business domain specific test data generation solutions in a cost effective manner

» It automates many manual test data generation tasks. For example, test application data model changes between releases. It can detect the changes in the data model and notify the Admin(s) about their options in automatically updating all (or some) of the test data generation Scenarios. It is proactive and assists testers by automating repetitive tasks.

» Real time data gets generated in different test environments on demand basis.

3) Continuous Monitoring Risk

Another important point is to understand and measure DevOps performance and improve organization agility. Business value to customer is key here along with awareness of risk in next deployment, articulation of level of quality or compliance.

There is a need to continuous monitor risks during build/test/deploy phases and to ensure rollback support for code/database change, backward and forward compatibility, re-usage of test scripts for production, re-usage of production data in dev and test environments.

Dashboard/Metrics view can play a vital role in keeping stakeholders informed. Nagios is one of the popular tools for Continuous Monitoring the service in production.

4) Communication and Collaboration

Communication and Collaboration are stepping stones for success of Dev Ops Implementation. Dev Ops is a cultural change that an organization needs to embrace and not just limited to tools and technology.

In today’s world, there are lot of challenges in terms of communication while doing project deliveries. As teams are geographically distributed, there are bound to have communication gaps and accountability issues. Owing to lack of connect among distributed teams, adhoc

changes are applied to applications and environments which disturbs the stability and equilibrium of eco system. Operational delays further add woes to it. Apart from this, lot of time gets wasted in frequent mails/meetings. Limited automation is another hindrance that needs to be tackled.

Proposed Solutions –

A clear goal needs to be defined with robust roadmap. This needs to be on boarded by Agile/Dev Ops team. Discussions should be encouraged without any fear. As teams are geographically distributed, it is key to understand and manage the cultural diversity and ensure all are on same page.

There is a need of Real time feedback and close communication between Dev, QA, Business and Ops ensuring communication is smooth. Code repositories should be easily searchable and sharable. Making information accessible through information radiators is a must.The best way ahead would be to use Automated Collaboration tools available like Yammer, Slack, HipChat, Mingle that gives a unique platform to Dev, QA, Business and Ops for communication. They can instantly discuss lot of things without any gaps.

Dev Ops – What Future Awaits?

The future of DevOps will see continued growth of the cloud, tools automation, containers and company-wide collaboration and help cut costs. Dev and Ops team together will ensure that new features developed are integrated well with the application and customer services are maintained at highest standards and in turn provide Excellent Customer Experience.

Emerging trends in Dev Ops -

Containers (specifically Docker) which serve as an ideal deployment vehicle. Gartner expects that by the end of 2018, 50% of enterprises will be running containers in Production.

DevOps and Big Data to come together to create predictive analytics throughout delivery cycle

Business coming together with Development and Operations team to create BizDevOps

Conclusion

DevOps is a way ahead to create robust infrastructure supported by tools and deliver a high quality application. There are clear benefits in terms of faster, predictable time to market, improve system stability and reliability and cost reduction. Adopting these best practices can enable organizations to implement high-performing Dev Ops solutions that can result in:

200 times more frequently releases 2,555 times faster lead times 24 times faster recovery 22% effort reduction on unplanned work and rework 

29% increased bandwidth to focus on new features, code delivery and deployment

References https://opexsoftware.wordpress.com (Puppet State of DevOps Report)

www.electric-cloud.com

Gartner (http://www.gartner.com)

Everest (https:// www.everestgrp.com )

https://www.docker.com

Appendix

Author Biography Rahul Soni has been working as a Senior Delivery Manager with extensive

exposure in managing BFS QA projects, Automation, End to end Program

management and QA Transformation initiatives.

Progressive experience of 12+ years in IT with extensive exposure to work from

onshore and offshore.

PMP, CSM, Advanced ISTQB Test Manager and Prince 2 Practitioner Certified.

Qualification – B.E. (Computers), PGDBM

THANK YOU!