CI/CD with Jenkins and Docker

Hello!Teerapat Khunpech

@engineerball

https://github.com/engineerball

https://engineerball.com

2

DevOps keys

http://web.premsco.com/wp-content/uploads/2014/09/Premsco-industrial-automation.jpg

What is Continuous integration

▷ A development methodology▷ Verified by builds

○ Unit test○ Functional test○ Integration test

▷ Every commit trigger a build

What is Continuous delivery

▷ Continuous delivery/deployment▷ Every commit that passed a build

could be deploy to production▷ Automation deploy

What is Jenkins?

▷ CI/CD application▷ Easy installation▷ Rich plugin▷ Distributed build

Jenkins Workflow

What is Docker?

Open Source engine for containers

Build, ship, run your application

within containers

Docker enables separation of

concerns

How can you use Jenkins & Docker together?

Run Jenkins Master & Slave in Docker Build, Test & Deploy Docker Image from Jenkins

Example

CI/CD with Docker

CI/CD Workflow

Tools

▷ SCM: Github▷ CI/CD: Jenkins 2.0▷ Platform: docker▷ Container Orchestration: docker swarm▷ Service Discovery Tool: Consul

Setup

Node0 set up

▷ Start the Consul container$ docker run -d -p 8500:8500 -h consul --name consul progrium/consul -server -bootstrapUnable to find image 'progrium/consul:latest' locallylatest: Pulling from progrium/consul3b4d28ce80e4: Pull completee5ab901dcf2d: Pull complete<snip>

Node1-3 set up

▷ Edit DOCKER_OPTS daemon (/etc/default/docker)

DOCKER_OPTS="-H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

--cluster-store=consul://10.128.0.2:8500/network --cluster-advertise=ens4:2375"

Install Docker Swarm Cluster

▷ Node0: Create Swarm Master # Create Swarm Master

$ export TOKEN=$(docker run --rm swarm create)

$ docker run -d -p 3375:2375 swarm manage token://$TOKEN

# Join Swarm Node

$ docker run -d swarm join --addr=10.128.0.3:2375 token://$TOKEN

$ docker run -d swarm join --addr=10.128.0.4:2375 token://$TOKEN

$ docker run -d swarm join --addr=10.128.0.5:2375 token://$TOKEN

Jenkins Setup

▷ Build Docker image inside a docker container

▷ SSH Node with access to docker engine

Jenkins Setup

▷ Node0: Install Jenkins Master$ docker -H unix:///var/run/docker.sock run -d -p 8080:8080 jenkins

Jenkins Setup

▷ Node0: Configure Jenkins Master○ Configure Global Security○ Install plugins

■ Github plugin■ CloudBees Docker Build and Publish

○ Create Jenkins Slave node (node1)

Example Voting App

▷ A Python webapp which lets you vote between two options

▷ A Redis queue which collects new votes

▷ A Java worker which consumes votes and stores them in…

▷ A Postgres database backed by a Docker volume

▷ A Node.js webapp which shows the results of the voting in real time

Voting App

Dockerfile# Using official python runtime base imageFROM python:2.7

# Set the application directoryWORKDIR /app

# Install our requirements.txtADD requirements.txt /app/requirements.txtRUN pip install -r requirements.txt

# Copy our code from the current folder to /app inside the containerADD . /app

# Make port 5000 available for links and/or publishEXPOSE 80

# Define our command to be run when launching the containerCMD ["python", "app.py"]

Result App

DockerfileFROM node:0.10

RUN mkdir /appWORKDIR /app

ADD package.json /app/package.jsonRUN npm install && npm lsRUN mv /app/node_modules /node_modules

ADD . /app

ENV PORT 80EXPOSE 80

CMD ["node", "server.js"]

Worker

DockerfileFROM java:7

RUN apt-get update -qq && apt-get install -y maven && apt-get clean

WORKDIR /code

ADD pom.xml /code/pom.xmlRUN ["mvn", "dependency:resolve"]RUN ["mvn", "verify"]

# Adding source, compile and package into a fat jarADD src /code/srcRUN ["mvn", "package"]

CMD ["/usr/lib/jvm/java-7-openjdk-amd64/bin/java", "-jar", "target/worker-jar-with-dependencies.jar"]

Github Webhook Setup

▷ Automatically build job when pushes are made to Github

The Build Jobs

▷ Create 3 Jenkins jobs○ build-voting-app○ build-worker○ build-result-app

The Build Jobs

The Build Jobs

The Build Jobs

The Deploy Job

▷ Create a Jenkins jobs○ Build section, Execute shell

export DOCKER_HOST=10.128.0.4:2375

export DTR="https://registry.hub.docker.com/u"

docker-compose -f vote-apps/docker-compose.yml stop voting-app result-app worker

docker-compose -f vote-apps/docker-compose.yml rm -f

docker-compose -f vote-apps/docker-compose.yml pull voting-app result-app worker

docker-compose -f vote-apps/docker-compose.yml up -d

The Deploy Job

docker-compose.ymlversion: "2"

services: voting-app: build: ./voting-app/. networks: - front-tier - back-tier

result-app: build: ./result-app/. networks: - front-tier - back-tier

worker: image: manomarks/worker networks: - back-tier

redis: image: redis:alpine container_name: redis networks: - back-tier

db: image: postgres:9.4 container_name: db volumes: - "db-data:/var/lib/postgresql/data" networks: - back-tier

volumes: db-data:

networks: front-tier: back-tier:

Deploying The App

▷ Clone SCM, edit README.md and push to SCM

Deploying The App

root@node2:~# docker ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES0100cf6b0c14 engineerball/result-app:47d362f35a115f2098c80a11e161eef626e1dbcb "node server.js" 2 hours ago Up 2 hours 0.0.0.0:5001->80/tcp voteapps_result-app_1296eee196e4e engineerball/voting-app:47d362f35a115f2098c80a11e161eef626e1dbcb "python app.py" 2 hours ago Up 2 hours 0.0.0.0:5000->80/tcp voteapps_voting-app_140f8d8aec4ca engineerball/worker:47d362f35a115f2098c80a11e161eef626e1dbcb "/usr/lib/jvm/java-7-" 2 hours ago Up 2 hours voteapps_worker_144da08df3d5c postgres:9.4 "/docker-entrypoint.s" 2 hours ago Up 2 hours 5432/tcp voteapps_db_1fa10362631cb redis "docker-entrypoint.sh" 2 hours ago Up 2 hours 6379/tcp voteapps_redis_1

The Voting App

The Voting Result

DEMO

In Conclusion

In Conclusion

▷ Automate is the key▷ Docker simplifies environment problems▷ Jenkins is ready for Docker and CD