Django Development with DockerCuong Tran - Docker Hanoi

About me

Cuong Tran

Community: Docker Hanoi Organizer

Job: Developer at Higgs Valley

Email: tranhuucuong91@gmail.com

Github: https://github.com/tranhuucuong91

Blog: http://tranhuucuong91.github.io/

Domain: Machine Learning, DevOps, Cloud

2

Agenda

1. Introduction Django Stack

2. Run Django Stack

3. Common activity

4. Problems and solutions (P&S)

5. Snippets

6. Q&A

3

1. Introduction Django Stack

4

Nginx

Django

PostgresRedis

1. Introduction Django Stack: Project Structure├── docker-compose.yml├── docker-env├── nginx│ ├── Dockerfile│ └── sites-enabled│ └── django_project├── production.yml├── README.md└── web ├── docker_django │ ├── apps │ ├── __init__.py │ ├── settings.py │ ├── urls.py │ └── wsgi.py ├── Dockerfile ├── manage.py ├── requirements.txt └── static ├── images │ └── django-pony.png └── main.css

5

1. Introduction Django Stack: docker-compose.ymlversion: "2"

services: nginx: #restart: always build: ./nginx/ ports: - "80:80" volumes: - /www/static volumes_from: - web depends_on: - web

6

web: #restart: always build: ./web expose: - "8000" depends_on: - postgres - redis volumes: - ./web:/usr/src/app - ./web/static:/usr/src/app/static env_file: docker-env environment: DEBUG: 'true' command: /usr/local/bin/gunicorn docker_django.wsgi:application -w 2 -b :8000 --access-logfile /var/log/gunicorn.log

1. Introduction Django Stack: docker-compose.ymlpostgres: #restart: always image: postgres:9.5 ports: - "5432:5432" volumes: - pgdata:/var/lib/postgresql/data/

7

redis: #restart: always image: redis:3.2 ports: - "6379:6379" volumes: - redisdata:/data

volumes: pgdata: redisdata:

2. Run Django Stack

# build stackdocker-compose build

# up stackdocker-compose up -d

# migrate datadocker-compose run --rm web /usr/local/bin/python manage.py migrate

8

3. Common Activity: Run container

● View web● Run command in container● Show logs● Connect database to container

9

3. Common Activity: Update git

● Edit and git commit● Git ignore files for dev, test, prod environments● Update git

10

3. Common Activity: Build container

● Rebuild docker images● Push to private registry● Update docker images in production

11

4. P&S: Handle `docker stop`

● When call docker stop, docker will send SIGTERM to main process inside the container. And after a grace period (default 10s), docker will send SIGKILL.

12Image Source: http://www.slideshare.net/LeszekGodlewski/advanced-linux-game-programming

4. P&S: Handle `docker stop`

● The main process need to handle the signal and graceful stop.

import sys, signal, time

def handler(signum = None, frame = None):

print('Signal handler called with signal', signum)

time.sleep(1) #here check if process is done

print('Wait done')

sys.exit(0)

for sig in [signal.SIGTERM, signal.SIGINT, signal.SIGHUP, signal.SIGQUIT]:

signal.signal(sig, handler)

while True:

time.sleep(6)

13

4. P&S: Startup order

Issue: Docker Compose:

● Will not wait until a container is "ready".● Only until it’s running.

Solution:

● Use a tool such as wait-for-it or dockerize.● (Or) Write your own wrapper script to perform a more application-specific

health check.

14Reference: https://docs.docker.com/compose/startup-order/

4. P&S: Startup orderwait-for-postgres.sh

#!/usr/bin/env bash

set -e

host="$1"shiftcmd="$@"

until psql -h "$host" -U "postgres" -c '\l'; do >&2 echo "Postgres is unavailable - sleeping" sleep 1done

>&2 echo "Postgres is up - executing command"exec $cmd

15

wait-for-it.sh

#!/usr/bin/env bash# Use this script to test if a given TCP host/port are available

cmdname=$(basename $0)

echoerr() { if [[ $QUIET -ne 1 ]]; then echo "$@" 1>&2; fi }

usage(){ cat << USAGE >&2Usage: $cmdname host:port [-s] [-t timeout] [-- command args] -h HOST | --host=HOST Host or IP under test -p PORT | --port=PORT TCP port under test...

4. P&S: Speed up `docker build`

Use .dockerignore

● Because docker will put current directory in build context.● To increase the build's performance, you can exclude files and directories

by adding a .dockerignore file.● Syntax like .gitignore.

16

4. P&S: Speed up `docker build`

Build cache

● The orders of command -> image layers -> build speed & image size & storage capacity

1. Install fixed things before. Ex: library, runtime2. Install changed things after. Ex: code

17

4. P&S: Limit Resourcescpu_shares: 73cpu_quota: 50000cpuset: 0,1

mem_limit: 1000000000memswap_limit: 2000000000

18

5. Snippets#--- Docker stats ---# stats all service in docker compose directorydocker stats `docker-compose ps | tail -n+3|awk '{print $1}'`

# stats all docker container are runningdocker stats `docker ps | awk '{print $NF}' | tail -n+2`

#--- Docker remove ---# remove all stopped containersdocker ps -a | awk '/Exit/ {print $1}' | xargs -r docker rm

# remove all untagged imagesdocker images | awk '/^<none>/ {print $3}' | xargs -r docker rmi

# remove old version imagesdocker images | grep ro.lan:5001/tranhuucuong91/nginx | tail -n +4 | awk '{print $3}' | xargs -r docker rmi

19

5. Snippets (cont.)

Save all the images on docker-compose.yml and deploy on machine not connected to the internet# Save Compressed ImagesIMAGES=`grep '^\s*image' docker-compose.yml | sed 's/image://' | sort | uniq`

docker save $IMAGES | gzip > images.tar.gz

# Load Compressed Imagesgunzip -c images.tar.gz | docker load

20

[1] https://github.com/tranhuucuong91/docker-training

[2] https://realpython.com/blog/python/django-development-with-docker-co

mpose-and-machine/

[3] https://github.com/pydanny/cookiecutter-django

[4] https://docs.docker.com/engine/reference/builder/

[5] https://docs.docker.com/compose/compose-file

References

21

Q&A

22