Everything as a code

Everything as a CodeНепривычно о привычном

@aatarasoff

No warranty guarantee

Matrix has you

Выйти за пределы...

...поняв, что всё есть код

Как вы представляете себе код?

@Configuration@EnableConfigurationProperties(OneServerProperties.class)public class OneServerConfiguration implements ApplicationContextAware { ApplicationContext applicationContext;

@Autowired OneServerProperties properties;

@Bean public HttpServer httpServer() throws IOException { HttpServer httpServer = new RelativePathHttpServer();

for (String beanName : context.getBeans(HttpController.class)) { httpServer.addRequestHandlers(context.getBean(beanName)); }

return httpServer; }}

Наверное как-то так

Что такое код?

● Коллекция инструкций

● Человекочитаемый формат

○ plain text

● Может быть понят и “проигран”

○ после компиляции

○ интерпретатором

Да я тоже пишу код!

Вооружи себя

Настройка рабочего окружения

Install apps Code Checkout Configure workspace First Blood

brew install

brew installpip install

customize*.properties

customize*.propertiestemplate

.gradle

customize*.propertiesinstall

certificates

first build

mass buildfirst deploy

ansible-playbook -i local-inv setup.yml --tags configure

Интерпретатор

Конфигурация

Алгоритм

#checkout repositories from git- include: checkout.yml

#configure your local environment- include: configure.yml

#add useful mappings to your hosts file- include: hosts.yml

#add gradle support- include: gradle.yml

#clean and build all projects- include: build.yml

#delpoy all services to dev- include: build.yml

Алгоритм

Входные параметры

- name: checkout services git: repo: "{{ git.root }}/{{ item.name }}.git" dest: "{{ work_dir }}/{{ item.name }}" update: yes with_items: - "{{ services }}" tags: - services

Массовый checkout/pull

- name: checkout services git: repo: "{{ git.root }}/{{ item.name }}.git" dest: "{{ work_dir }}/{{ item.name }}" update: yes with_items: - "{{ services }}" tags: - services

Переменные

- name: checkout services git: // with_items: - "{{ services }}"

services: - { name: one-instant-messenger , sourceDir: src } - { name: one-discussions, sourceDir: src } - { name: one-attachment, sourceDir: src, testDir: test, local: true }

Циклы

services: - { name: one-instant-messenger, sourceDir: src } - { name: one-discussions, sourceDir: src } - { name: one-attachment, sourceDir: src, testDir: test }

Коллекции и структуры данных

- name: create gradle build file template: src: build.gradle.j2 dest: "{{ work_dir }}/build.gradle" mode: 0644

- name: create gradle settings file template: src: settings.gradle.j2 dest: "{{ work_dir }}/settings.gradle" mode: 0644

Шаблоны

{% if services is not none %}{% for service in services %} if (project.name == '{{ service.name }}') {{% if 'sourceDir' in service %} main.java.srcDir '{{ service.sourceDir }}'{% endif %}{% if 'testDir' in service %} test.java.srcDir '{{ service.testDir }}'{% endif %} }

Условные операторы

{% if services is not none %}{% for service in services %} if (project.name == '{{ service.name }}') {{% if 'sourceDir' in service %} main.java.srcDir '{{ service.sourceDir }}'{% endif %}{% if 'testDir' in service %} test.java.srcDir '{{ service.testDir }}'{% endif %} }

Опять циклы

- stat: path={{ username }}.key register: certkey

- name: generate private key shell: openssl genrsa -out {{ username }}.key -aes256 4096 when: not certkey.stat.exists

Идемпотентность

Install apps Code Checkout Configure workspace Multiplie Use

Петля обратной связи

Что получили

● Автоконфигурация рабочего

пространства

○ повторяемая

○ немутабельная

● Можно дать скрипт новичку

● Можно и нужно пользоваться этим

каждый день

Код есть код

Искусство сборки

Compile code Unit tests Package Publishing

Compile code Unit tests Package

compiler

Publishing

javacresourceprocessing

Publishing

javacresources copyingdependency resolving

Publishing

jarnpm, deb, ...

Publishing

jarnpm, so, ...docker image

Publishing

maven, ivymaven

Publishing

Compile code Unit tests Package Publishing

maven, ivylocal or dev deploydocker registry

Системы сборки

● Ant + Ivy

● Maven

● Gradle

● Docker

● npm

● ...

FROM golang:1.7-alpineMAINTAINER aatarasoff@gmail.com

VOLUME /dataWORKDIR /data

RUN apk update && \ apk upgrade && \ apk add git bash

RUN go get github.com/aatarasoff/apistress && \ go install github.com/aatarasoff/apistress

CMD [ "apistress" ]

Dockerfile. Наследование

VOLUME /dataWORKDIR /data

RUN apk update && \ apk upgrade && \ apk add git bash

CMD [ "apistress" ]

Dockerfile. Инструкции

ARG maindir=/data

VOLUME $maindirWORKDIR $maindir

RUN apk update && apk upgrade && apk add git bash

CMD [ "apistress" ]

Dockerfile. Переменные

docker build … && docker push …

publishing { publications { mavenJava(MavenPublication) { artifactId 'spring-one-nio-autoconfigure'

from components.java

artifact sourceJar { classifier "sources" } } }}

Gradle. DSL

compile(ivyDependencies(projectDir, 'runtime'))

def ivyDependencies(ivyPath, conf) { def dep = [] def ivyModule = new XmlParser().parse(file("${ivyPath}/ivy.xml"))

ivyModule.dependencies.dependency.each dep.add([group: (null == it.@org ? 'ru.odnoklassniki' : it.@org), name: it.@name, version: it.@rev, configuration: (it.@conf =~ /->(\w+)/)[0][1]]) }

return dep}

Gradle. Ну просто код

<macrodef name="docker-build-image" description="Build docker image"> <attribute name=" buildcommand"

default="build -t @{repo}/@{image}:@{tag} ."/>

И даже в Ant-е есть жизнь

./gradlew build test ant-docker-build-image publish

Фреймворки для автоматизации

● Ant + Ivy

● Maven

● Gradle

● Docker

● npm

● ...

● Сборка это код

○ XML

○ DSL

○ Groovy

○ etc

● Системы сборки не только для

сборки

Ликвидация багов

Unit tests API tests Stress tests UI tests

BDD Specs

def "Return error code 400, if User-ID header is not presented"() { given: def request = post("/rent")

when: def result = this.mvc.perform(request)

then: result.andExpect(status().isBadRequest()) .andDo(document("rent-user-id-is-absent"))}

Дружелюбный BDD

def "Return error code 400, if User-ID header is not presented"() { given: def request = post("/rent")

then: result.andExpect( status().isBadRequest()) .andDo(document("rent-user-id-is-absent"))}

Простые проверки

JMeter, wrk, vegeta

JMeterproduction

{ "baseUrl": "http://host:9000/rent-service", "tests": [ { "rps": 10, "duration": 5, "target": { "method": "GET", "path": "/rent", "headers": [ ... ] }, "sla": { "latency": 1000, "successRate": 99.999 } }, ... ]}

config.json

cat config.json | docker run -i apistress -config=stdin

Где-то мы такое видели

Requests [total, rate] 50, 10.20Duration [total, attack, wait] 5.022872793s, 4.899943287s, 122.929506msLatencies [mean, 50, 95, 99, max] 143.772484ms, ..., 290.101831msBytes In [total, mean] 4842, 96.84Bytes Out [total, mean] 950, 19.00Success [ratio] 100.00%Status Codes [code:count] 200:50Error Set:

Test#1

> cat config.json | docker run -i apistress -config=stdin> echo $?0

Код возврата

Requests [total, rate] 200, 10.05Duration [total, attack, wait] 23.04784s, 19.899754743s, 3.148093811sLatencies [mean, 50, 95, 99, max] 3.023677499s, ..., 11.832287083sBytes In [total, mean] 6874, 34.37Bytes Out [total, mean] 1349, 6.75Success [ratio] 35.50%Status Codes [code:count] 0:129 200:71Error Set:Get http://host:9000/rent-service/rent: EOFGet http://host:9000/rent-service/rent: http: server closed idle connection...

Test#2

> cat config.json | docker run -i apistress -config=stdin> echo $?1

Код возврата

Selenium, Selenide

● Все тестовые сценарии в коде

● Можно встроить в процесс сборки

или доставки ПО

● Если хотите, то можно

генерировать отчёты для разбора

полётов

Кододокументация

Analyst, PM Developer Tester Docs

Word, PDF...

Word, PDF... Code + Tests

Word, PDF... Code + Tests Test cases

Analyst, PM Developer Tester Docs :(

Word, PDF... Code + Tests Test cases

Analyst, PM Developer Tester Docs :)

Markdown/Asciidoctor

Docs :)

= Hippo Rent ServiceThis is documentation for Open API of our hippo renting service

== Methods

=== Rent

==== Request specification===== Headers//тут опишем http-заголовки===== Example//а здесь будут примеры вызова

==== Response specification===== Response fields//здесь описание полей ответа===== Example//ну и пример того, что вообще ожидать в ответе

Вот такой документ

./gradlew ... asciidoc publishDocs

Ну, вы поняли да? :)

def "Rent a hippo"() { given: def request = post("/rent").header("User-ID", "aatarasoff")

then: result.andExpect(status().isOk()) .andDo(document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) ))}

…и снова тесты

def "Rent a hippo"() { given: def request = post("/rent").header("User-ID", "aatarasoff")

then: result.andExpect(status().isOk()) .andDo(document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")) ))}

А не документация ли это?

document( "rent-hippo", preprocessResponse(prettyPrint()), requestHeaders( headerWithName("User-ID").description("User unique identifier")), responseFields( fieldWithPath("hippoRemain").description("Hippo remain count"), fieldWithPath("parrot_fee").description("Fee in virtual parrots"), fieldWithPath("ins").description("Insurance number. Yeah, we sell it"), fieldWithPath("hash").description("Blockchain block hash")))

Имя сниппета

Тестируем заголовки

Тестируем поля ответа

generated-snippetsrent-hippo

curl-request.adochttp-request.adochttp-response.adochttpie-request.adocrequest-headers.adocresponse-fields.adoc

Получаем сниппеты

== Methods

=== Rent

==== Request specification===== Headersinclude::{snippets}/rent-hippo/request-headers.adoc[]===== Exampleinclude::{snippets}/rent-hippo/http-request.adoc[]

==== Response specification===== Response fieldsinclude::{snippets}/rent-hippo/response-fields.adoc[]===== Exampleinclude::{snippets}/rent-hippo/http-response.adoc[]

Вставляем их в документ

== Methods

=== Rent

==== Request specification===== Headersinclude::{snippets}/rent-hippo/request-headers.adoc[]===== Exampleinclude::{snippets}/rent-hippo/http-request.adoc[]

==== Response specification===== Response fieldsinclude::{snippets}/rent-hippo/response-fields.adoc[]===== Exampleinclude::{snippets}/rent-hippo/http-response.adoc[]

Что получили?

● Документация как код

○ лежит в репозитории с кодом

○ встроена в цикл сборки

○ рендерится в html, pdf и т.д.

○ почти всегда актуальна

● Синергия с тестами

Инфракод

Hardware

Containers

Application

Mesos/Kubernetes/Private cloud

Hardware + OS System Libs PaaS Application

Ansible

AnsiblePuppet/Chef

Ansible. Inventory

[datacenter]api-server-1api-server-2api-server-3

[datacenter:vars]magicvar = 42

Ansible. Playbook

- hosts: datacenter roles: - role: docker - role: rsyslog

ansible-playbook -i datacenter1 bootstrap.yml

Без комментариев

Ansible

AnsiblePuppet/Chef

Manifest

Docker compose

services: zk: image: zookeeper network_mode: bridge ports: - 2181:2181 environment: ZK_CONFIG: tickTime=2000,initLimit=10,clientPort=2181 ZK_ID: 1

Docker compose

Конфигурация сервиса

Mesos/Marathon{ "id": "/api/rent-service", "cpus": 1, "mem": 1024, "instances": 3, "container": { "docker": { "image": "rent-service:0.0.1", "portMappings": [ { "containerPort": 8080 } ] } }}

curl -X POST ... http://marathon/v2/apps?force=true

Вот и весь деплоймент

Конфигурация приложений

https://your_repo/rent-service-config/routes.yml

routes: payment: path: /payment-service/** serviceId: payment-service

Ещё конфигурация

● Zookeeper

● Consul

● Vault

● configo by Zeroturnaround

● ...

configo

docker run \ -e CONFIGO_SOURCE_0='{"type": "http", "format": "yaml", "url":

"https://my.server.com/common.yaml"}' \ rent-service

//внутри приложенияgetEnvironmentVariable("MY_ENV_VAR")

https://github.com/zeroturnaround/configo

● Инфрастуктура может быть легко

описана в виде кода

● Деплоймент и конфигурация

приложений в виде конфигов и

манифестов

Неубиваемый CI

Install Master Configure Slaves

Ansible

Jenkins Docker Cloud plugin

<——— Хост с докером

<——— Сколько контейнеров можно запустить

Автоконфигурация

<clouds> {% for group in ['build', 'test', 'production'] %} {% for node in groups[group + '-slaves'] %} <com.github.kostyasha.yad.DockerCloud plugin="yet-another-docker-plugin@0.1.0-rc31"> <name>{{ node }}</name>

... <templates> <com.github.kostyasha.yad.DockerSlaveTemplate> <id>mycloud-template</id> <dockerContainerLifecycle> <image>{{ group }}-jenkins-slave</image> ...

</templates> <connector> <serverUrl>tcp://{{ node.hostname }}:2375</serverUrl> <apiVersion>1.20</apiVersion> </connector> </com.github.kostyasha.yad.DockerCloud> {% endfor %} {% endfor %} </clouds>

Код доставки

pipeline-template.groovy

//checkout and definition stagenode('build') { // Mark the code checkout 'stage' stage 'Checkout' git credentialsId: 'jenkins-git', url: "${git_url}/${repo}.git"

// Mark build 'stage' stage 'Build'

sh ('./gradlew clean build final')}

//next steps128

//next steps129

//next steps130

//next steps131

//next steps132

//next steps133

//deploy artifact to testnode('test') { sh('ansible-galaxy install -r requirements.yml')

ansiblePlaybook( credentialsId: 'ansible', installation: 'ansible', playbook: 'deploy.yml', inventory: 'test' )}

jiraComment ( issueKey: issue_id, body: "Artifact has been deployed")

node('build') { def repos = fetchRepos(project) for (repo in repos) { build(repo) }}

def fetchRepos(String project) { def url = new URL("https://repo/projects/${project}/repos?limit=1000")

def conn = url.openConnection() conn.setRequestMethod("GET") def responseCode = conn.getResponseCode()

final slurper = new groovy.json.JsonSlurper()

def repos = slurper.parse(conn.getInputStream()).values

for (repo in repos) { if (repo.slug.contains('one-')) result << repo.slug }

return result}

Микросервисы

Create meta job

Ansible

Create meta job

AnsiblecURL

Create meta job

Create pipelines

jobs.each { job -> pipelineJob("${basePath}/${job}") { //define SCM

definition { cps { script(readFileFromWorkspace('pipeline-template.groovy')) sandbox() } } }}

JobDSL plugin

Create meta job

Create pipelines

ansible-playbook -i jenkins-for-my-team jenkins.yml

Это последний раз

● Пайплайн как код

● Неубиваемый CI

○ без бэкапов

○ всё хранится как код

○ разворачивается за X минут

Выводы

● Почти любой процесс можно

формализовать, представить в

виде кода и автоматизировать

● Мы все пишем код, хотя можем

думать, что это не так

Спасибо, что выбрали красную

@aatarasoff

Everything as a code

Technology

Everything should be as simple as possible, but no simpler

Infrastructure as Code Software Defined Everything for ...nyoug.org/wp-content/uploads/2017/09/nyoug1709_sde.pdf · Infrastructure as Code Software Defined Everything for Oracle DBAs

1 Core Ingredients of Computing A “universal” machine Programmability Binary Code (everything stored as “0” or “1”) Program and Data are interchangeable

Everything-as-a-service operating models

Everything is funny as long as it is happening to somebody

Runnable: Discover Everything Through Code

1. Everything is connected to everything else. 2. Everything must go somewhere. 3. There is no such thing as a free lunch

Everything as a Cookbook

Everything As A Service with IoT

Everything-as-code - a polyglot journey

ReactJS - webagesolutionsfiles.com · Reactjs Features • React is used for V in MVC. • React treats everything as component. • Code Maintenance is simple in component driven

Everything as Code

XaaS Benchmarking: Everything as a Service?

Everything you need for your as textiles technology

Supplier Code of Business Conduct - Honeywell...HONEYWELL’S SUPPLIER CODE OF BUSINESS CONDUCT Honeywell is committed to integrity and compliance in everything we do. As part of that

jsut code @ Future Everything 2012

Anything as a service makes everything possible

Everything-as-code. Polyglotte Software-Entwicklung in der Praxis

Code but not as we know it - Infrastructure as code

18. Lenka - Everything at once.notebook - Zakladní škola …€¦ · · 2013-11-26LYRIC Předmět: Anglický ... as _ as a wrong As _ as a road, ... 18. Lenka Everything