A U T O M A T A

INDEXWhat is Automata - A glance! 3The Business Need 3Automata Architecture 4Automata Workflow 4Automata - Features and Functionalities 7One Click Deployment 8Pipeline Creation In Automata 9Continuous Deployment of Microservices 10Security 10Build Approval 11Lock Service 11Single Interface Microservices Automation 12Verifying the successful environment 13Roll Back Feature 13Jenkins Integration 14Dockers Registry and Image Management 14Maintain the Versioning of Microservices (DIMG) 15Namespaces 15Deployment History 15Resource Optimization 16Orchestration and Containerization 17Centralized Configuration Management System (Git-hub, Config Values) 17Automata Scalability 17Self healing 17

Automata Infra 19Solution provided 19Features 19Automata Compliance Report 19GKE 20GKE Cluster Creation -- With the help of Automata Infra, we just have to fill basic configurations (cluster names, minimal nodes, machine type, disc size) to create GKE clusters. 21GKE Live Migration 22Redis 23MongoDB 24MySQL 25NAT Gateway 25Jenkins VM creation 25Image patching 26Automata Command Line Interface 26Automata Governance 29Image patching/ OS patching 29Monitoring and Auditing 29Reporting 30

What is Automata - A glance!

Cloud-Based, Continuous Delivery, Containerized and Customizable

Automata is a for continuous delivery platform framework that is focused on cloud operational excellence and time to market, without the hassle of setting up manual delivery pipelines. Automata is fully integrated with Google cloud and container engine, but with a wide range extensible features for other cloud platforms. Automata is a microservice deployment platform which provides custom autoscaling of GKE and Openshift, which has enabled the creation of a more reliable, highly resilient and fault tolerant microservices

The Business Need

The pre-Automa Epoch:

In the era of Cloud Computing, to sustain the increasingdemand from market, organisations have tocontemplate whether

▪ Microservices over Monolithic application▪ Expected low Deployment turn around time▪ Need for Continuous Deployment▪ Time to market

Need to Address the Deployment Challenges

Handling multiple microservices in shorter cycles has its own challenges as these these microservices were interconnected and were dependent on artifacts from various other microservices. So the general perception that Continuous Deployment is easy doesn't hold true.

In addition to creation of pipelines, there exists a need for the microservices to follow the general practice of workflow, source code control, version control of artifacts, while supporting authentication and orchestration, security, log auditing introspects microservices. Microservices that are deployed had to be compatible with kubernetes, openshift while offering features like scaling.

Need to Address the Deployment Challenges

Handling multiple microservices at shorter duration brought in its own challenges as these these microservices were interconnected and were dependent on artifacts from various other microservices. So the general myth that Continuous Deployment is easy doesn't hold its truth.

In addition to creation of pipelines, there always existed a need for themicroservices to follow the general practice of workflow, source code control,version control of artifacts, while supporting authentication and orchestration,security, log auditing introspects microservices. The deployed microservices hadto be compatible with kubernetes, openshift while offering features like scaling.

Need to Address the Agility and provide Flexibility

While most clients prefer the traditional 4-stage environment deployment (Dev,QA, Staging and Production), many others prefer to have additional setup likeRelease Dev, Release QA, Performance, UAT etc. There is a demand to provide thisflexibility in order to ensure high agility.

This led to creating a stand alone product, Automata, that can house and addressall these requirements under a single framework.

Automata is not just limited to microservices pipeline but also supportsunderlying infrastructure, necessary for creating infrastructure (Automata Infra)

Automata Workflow

Automata Workflow defines how Microservices deployment is automatedthrough different levels of environment .Our (Engineering) goal is to get rid ofmanual approval process and everything be governed by the automation.

The entire workflow can be divided into three major sections

▪ LLE Kubernetes Environment▪ HLE Kubernetes Environment▪ PROD Kubernetes Environment

Each of these sections function to reduce approvals and the function of eachsegment is described in detail.

1) LLE Kubernetes Environment - CICD - Jenkins

a) Pipeline for code deployment in LLE DEV environment

▪ Code pulled from GitHub repo (Release Branch e.g., R18.9)▪ Create a docker image and push it to GCR (Google cloud repository)▪ Deploy the image into the kubernetes cluster▪ Approval will be sent to QA for the deployment(Once QA approves, the latest

build which is deployed in DEV will be propagated to QA)▪ If any issue, rollback to an older version using kubernetes

b) Pipeline for code deployment in LLE QA environment

▪ Pull image from GCR and deploy it in QA. (Same image is created in DEV env)

▪ QA will test and certify

c) Pipeline for code deployment in LLE RELDEV environment

After QA certification, the code will be merged to master branch

▪ Code pulled from GitHub repo (Master branch)▪ Create a docker image and push it to GCR (Google cloud repository)▪ The same image is pushed to HLE.

2) HLE Kubernetes Environment

a) Pipeline for code deployment in HLE QA environment

▪ Pull image from GCR(HLE repo) and deploy it in HLE QA. (Same image created in RELDEV env)

▪ QA will test and certify.

b) Pipeline for code deployment in HLE PERF

environment

▪ Pull image from gcr(HLE repo) and deploy it in HLE PERF. (Same

image created in RELDEV env)▪ PERF will test and certify.

c) Pipeline for code deployment in HLE UAT environment

▪ Pull image from GCR(HLE repo) and deploy it in HLE QA. (Same image created in RELDEV env)

▪ UAT testing is performed and sign off will be given if all test cases pass.

3) PROD Kubernetes Environment

a) Pipeline for code deployment in HLE PROD environment

▪ Pull image from gcr(HLE repo) and deploy it in PROD PRE-PROD.(Same image created in RELDEV env)

▪ PERF(Load Performance test) and QA(Regression test and Smoke test ) testing will be done and sign off will be given.

b) LABEL SWITCH (BLUE-GREEN DEPLOYMENT)

Switch Preprodution Label with Production label to point new deployments to production and serve the traffic.Process:

• We have 2 ingress in production.

• Ingress-1 pointing to preproduction deployments and Ingress-2 pointing to prod deployments.

• Once we get a approval for label switch, we switch ingress-2 label pointing to preprod deployments(Which becomes production).

• Once production deployments are tested, it will automatically delete pre-prod deployments.

Automata - Features and Functionalities

Functionality, Reliability, Usability, Efficiency, Maintainability and Portability

The framework built for microservice orchestration, Automata provides custom Autoscaling of GKE and Openshift. In addition it provides innumerable features as outlined in the below section.

Automata’s Features

One Click Deployment

The painstaking process of manual job writing in Jenkins has been replaced by single, intelligent YAML file for end to end deployment. With the help of a single automata YAML file, the information can easily be uploaded from GitHub repo making it very simple and user-friendly.

Pipeline Creation In Automata

• A pipeline is a set of jobs that is needed to maintain continuous integration and continuous delivery of version of the code from development to production environments which reduces time to market.

• Automata is a one stop solution where we provide a YAML consisting of all the details of a microservice code to be deployed till production, irrespective of the language of code the microservice is written in or the cloud where it has to be deployed.

Continuous Deployment of Microservices

• The product improves rapidly through fast feature introduction and fast turn-around on feature changes.

• Upgrades introduce smaller units of change at any particular time this causes it to be less disruptive.

• End-user involvement and feedback during continuous development leads to usability improvements. New requirements can be added based on customer’s requirements, deploying micro services on client demand

• The optimization and substitution of components in Automata’s pipeline are built to be dynamic to suit the customer’s ever changing needs.

Security

IAM Feature in Automata

User Management and Identity access management (IAM) ensures security in Automata. This is a combination of business processes, policies and Role Assignment which allows organizations to provide secure access to confidential data. IAM enables improving User Experiences and simplifies Auditing and Reporting. Based on Policy Name and Environment ( ex-QA, Dev), relative authorization can be provided to different users.

Automata offers strong role based IAM (Identity Access Management) like Builder, Locker, Approver and Admin for better control and process flow.

Build Approval

The Build Approval automatically triggers approval process before triggering a build in any environment. Clicking on Build Approval allows us to see the microservicesbuild status through the Approval List. The “BUILD” button, empowers the user to trigger a new build in the pipeline with just the simple click of a button. It helps in controlling and testing the life cycle for application builds while parallelymonitoring the status of build as shown in below figure.

Lock Service

The Lock Service functionality prevents multiple people working on the same microservice at any given time by locking the environment for the entirety of the deployment period.

All changes to a pipeline during the deployment will be locked. Users with the LOCKER access role are the only one who are given the permission to lock pipelines. However this can be unlocked after successful deployment. (As shown below in screenshot)

Single Interface Microservices Automation

The Automata framework comes with a single Interface MicroserviceAutomation. With the help of a single click the user can trigger a new build in the pipeline. Once the particular environment is approved by the designated personnel, users with the Jira Ticket and necessary comments can trigger a build.

Verifying the successful environment

Tracking and Reporting has been made easier with DIMG (a unique identification number for each deployment) along with builder and approval details for each deployment. At each stage, a user can verify the deployment status for success/failure and take action accordingly.

Roll Back Feature

In case of a deployment failure, Automata provide you with an in-build ability to rollback to the last successful deployment. At present we have given the user to power to roll back to the last 5 working deployments.

Jenkins Integration

Automata auto-establishes the connection with Jenkins and generates the pipeline for deployment, every time a build is triggered.

Dockers Registry and Image Management

Docker makes it easy to create, deploy and run applications by using containers.The unique DIMG ID that automata generates using Dockers can be used for creating, tracking and rolling back of images in any environment.

Advantages of using Docker Containers in Automata▪ Platform independence: Build it once, run it anywhere.▪ Resource efficiency and density.▪ Effective isolation and resource sharing.▪ Speed: Start, create, replicate or destroy containers in seconds.▪ Immense and smooth scaling.▪ Operational simplicity.

Maintain the Versioning of Microservices (DIMG

The Dockers Image Version serves as key value in maintaining different microservices. The Image version propagates from one environment to another environment until it reaches the production environment. For example, from Devto QA to Staging till Production environment.

Namespaces

Namespaces are intended for use in environments with many users spread across multiple teams or projects. Automata allows you to create Namespaces that enable one Kubernetes cluster to be used by multiple users or a single user with multiple applications without concern for undesired interaction. Thus it helps to maintain the uniqueness of different Microservices and Cluster

Deployment History

Detail deployment history of every microservices in Automata is made available in Automata.

Resource Optimization

Billing features gives you an insight allocated resources and current usage CPU and memory metrics for each service. Based on the allocated versus used, you could scale-in or scale-out to meet your need.

The scaling would have an impact on cost and this is a good means to control overrunning of cost.

Orchestration and Containerization

Automata uses Google’s Kubernetes (a container based orchestration system) for automating the deployment, scaling and management of containerized applications.

Orchestration helps in aligning the business request with the applications, data, and infrastructure. It defines the policies and service levels through automated workflows. This creates an application-aligned infrastructure that can be scaled up or down based on the needs and inputs by the customer. Orchestration also provides centralized management of the resource pool, including Billing feature in Automata. For example, orchestration reduces the time and effort for deploying multiple instances of a single application

It aims to provide automating the deployment, autoscaling and self healing. It works with a range of container tools, including Docker. By integrating Kubernetes in Automata, the deployment of CI pipeline, results in amazing reproducibility & easy rollbacks for any successful commit that was made.

Centralized Configuration Management System (Git-hub, Config Values)

Automata Scalability

Automata is currently configured to auto-scale - that is based on CPU utilization. Automata template we define the requirements for the Automata Machine. This is direct cost saving as google bills for actual consumption of resources.

Self healing

Self-healing, in automata describes the ability to perceive faulty operations and correct it on the go without less to no human intervention. Self-healing makes necessary adjustments to restore self to normal operation.

A U T O M A T A

- I N F R A -

Automata Infra

Automata-Infra framework facilitates infrastructure for deploying microservices. It provides the ability to create and customize different components as per requirements, takes cares of necessary parameter values, prompts the user to enter correct data and maintains all activity data performed by the user.

Solution provided

Automata-infra enables operation team to infrastructure components easily as per their requirements. Version 1.0.0 provides various functionalities like admin permissions for the user account to access any environment, user signup-login, listing existing automata deployments, downloading sample configuration template, creating –updating deployment, deleting deployment by logging all activities. Also, Automata has a role based access control system which tracks each user activity like deployments, sign up, logins, logouts.

Features

Automata Compliance Report

To check the compliances of VM through Metadata. Automata Infra UI helps us to get the compliance and Non- Compliance list based on environment and zone as shown in below figure.

GKE

We can deploy Kubernetes clusters with help of GKE while managing the masters and Databases. We can create GKE by Automata Infra UI, with the minimal required input.

GKE creation can be done by two ways

GKE Cluster Creation -- With the help of Automata Infra, we just have

to fill basic configurations (cluster names, minimal nodes, machine type, disc size) to create GKE clusters.

YAML - Drag and drop the YAML which will bind other values to the formsForm Filling- Manual filling of necessary parameters

GKE Live Migration

With just one click we can migrate the lower version of cluster to higher version of cluster. Automata Infra UI provides us two options:

▪ Choose from Cluster▪ Choose to Cluster

With the help of both options we have provisioning of the pre-view of clusters

Automata Infra Without Preview

Automata Infra With Preview

Redis

In automata Infra we can create the image of Redis database.

MongoDB

With the help of Mongo DB we can migrate old data to the new server

The underlying database:

My SQL

NAT Gateway

Connecting two different Subnets

Jenkins VM creation

You can create Jenkins VMs with minimal user inputs using Automata Infra UI, which helps in Pipeline creation and Microservices deployment.

Image patching

Security updates which is given by Red Hat every month is added as a new image to be integrated on the OS level based on same or different regions.

This not only reduces the time but also minimizes the errors and ensures the same configuration is maintained all across.

Automata Command Line Interface

Automata provides you with CLI interface using which you can execute commands for all functionalities. At the same time, you could use CLI for logging in, getting help, system information , version etc

For example use automata-aws-cli version to get current version of Automata.Similarly use automata-aws-cli login to login into automataExecute “help” command at CLI to get help on Automata AWS CLI

Refer the below screenshots for understanding these commands and their output:

Automata CLI init

First, the command is used to initialize the Jenkins configuration --env: this commands implies different environments example- LLE HLE and Prod

2nd CommandCreates deployment by passing YAML file through commands--env: this commands implies different environments example- LLE HLE and Prod--svc implies a Microservices name like Profile service Browse Service to Deploy

3rd Command--config by using this command YAML files are passed

Automata Governance

Automata helps us in monitoring, measuring and managing the entire framework as outlined below

Image patching/ OS patching

Image patching process where we created another image from base Image, and this images helps to create VMs.

Monitoring and Auditing

Monitor your CPU and memory usage and TPS via Automata Graphs Feature:

Reporting

Automata pulls out any reports for the given environments.

Kloud9 was founded with the vision of bridging the gap between E-commerce and cloud. The E-commerce of any industry is limitingand poses a huge challenge in terms of the finances spent on physical data structures.At Kloud9, we know migrating to the cloud is the single most significant technology shift your company faces today. We are yourtrusted advisors in transformation and are determined to build a deep partnership along the way. Our cloud and retail experts willease your transition to the cloud.

+1 315 636 0950info@kloud9.nyc