New Magento 2.2 Deployment New Magento 2… · Flexibility in what to do in build vs deploy phases, what is in git, ... Pipeline Deployment Deployment flow. Pipeline Deployment Configuration

© 2017 Magento, Inc.

New Magento 2.2 Deployment Capabilities & Patterns


Alan KentMagento Chief ArchitectMagento Commerce, USA


New features to help deployments– Build step no longer needs database access– Faster compilation and asset generation– Compact web assets – reduces bundle and extract time– Read-only directories in production

Improved support for multiple environments– Environment variables– Per environment configuration file (env.php)– Shared configuration file (config.php)– New “sensitive” settings (passwords, PII)

Coming in 2.2

https://12factor.net/

I. Codebase in version controlIII. Store config in environmentV. Build, release, runX. Dev/prod parity

Deployment Flow


A Simple Development/Deployment Lifecycle

Git pull

Develop

Test

Gitcommit/push

Compilation

PHP Code Generation

Database Upgrade

End MaintMode

Web Asset Generation

Git Pull

Maint Mode

Develop Deploy

Production ServerDeveloper Laptop


A Simple Development Deployment Lifecycle

Git pull

Develop

Test

Gitcommit/push

Compilation

PHP Code Generation

Database Upgrade

End MaintMode


Git Pull

Maint Mode

Develop Deploy

Can be slow

Needs DBCan be slow

Wan

t to

min

imiz

e

Can be slow

Production ServerDeveloper Laptop


Adding a Build/Package Phase to Build Pipeline

Git pull

Develop

Test

Gitcommit/push

Extract

Maint Mode

PHP Code Generation

Database Upgrade

End MaintMode

Switch Code

Git pull


Compilation

Test Automation

Develop Build & Bundle Deploy

Bundle (ZIP)

Faster*

Faster*DB not neededCompact

Can be

slow

Production ServerDeveloper Laptop Build Server

* Backported to 2.1 patches

© 2017 Magento, Inc. Production Server

Future: Zero Downtime Deployments (if no DB change)

Git pull

Develop

Test

Gitcommit/push

Extract

Maint Mode

PHP Code Generation

Database Upgrade

End MaintMode

Switch Code

Git pull


Compilation

Test Automation

DB Upgrade

Req’dY N

Switch Code

Develop Build & Bundle Deploy

Bundle (ZIP)

Faster

FasterNo DBCompact

Onl

y if

sche

ma

chan

ged

Normal

Developer Laptop Build Server


Summary of Performance Boosts

Code optimization of

compilation

(backported to 2.1.x patches)

Code optimization of

web asset generation

(backported to 2.1.x patches)

New compact deployments, reducing disk

I/O

Able to run on build server without DB

(not production server)


3 locales x2 faster; 15 locales x10 faster – due to less disk I/O

Compact Mode for Static Assets

$ magento setup:static-content:deploy --strategy compact ...

$ magento setup:upgrade --keep-generated ...

Compact mode reduces duplication when multiple themes or locales– Creates map.php & requirejs-map.js per area specifying real file locations– PHP & JS functions supplied to map URLs to shared resources

Flexibility in what to do in build vs deploy phases, what is in git, …


To move configuration between environments, 12 Factor App says “configuration is code” (under version control, pushed with code, …)

Configuration as Code

Sections of config.php• Enabled modules• Scopes (websites, stores, store views)

and Themes• Store configuration settings


Enabled Modules Section

Developer 1 Developer 2 Staging Production

config.php

module:enablemodule:disable

config.php config.php config.php

Developer Mode Production Mode

GitBuild


Enabled Modules Section


config.php

module:enablemodule:disable

config.php config.php config.php


GitBuild

magento module:enablemagento module:disable

Updates config.php, pushed with code


• Websites, stores, and store views defined via Admin– In database for foreign key validation

• List of available themes also in database• Needed by build phrase

– Export database content to file for deployment

Scopes and Themes


Scopes and Themes Sections


config.phpapp:config:dump

config.phpapp:config:import

config.php config.php


GitBuild

setup:upgrade setup:upgrade








GitBuild


/admin

Websites, stores, store views defined in Admin

1








GitBuild


magentoapp:config:dump

Adds scope and theme details to config.php

2








GitBuild

setup:upgrade setup:upgrademagento

app:config:import

Copies scope and theme details from config.php into database

3








GitBuild


magento setup:upgrade

Setup upgrade includes import and schema upgrade

4








GitBuild



System Configuration Settings Fallback

1• Configuration values in environment variables

2• Configuration values in environment-specific config file (env.php)

3• Configuration values in shared config file (config.php)

4• Configuration values in DB

5• Configuration default value (config.xml)

Settings in config.phpand env.php are “locked” - cannot be changed via Admin


System Configuration Settings


env.phpconfig.php

config:set

app:config:dump

env.phpconfig.php env.phpconfig.php

config:set --lock

env.phpconfig.php

config:sensitive:set


/admin

GitBuild




env.phpconfig.php

config:set

app:config:dump


config:set --lock

env.phpconfig.php



/admin

GitBuild

/adminmagento config:set

Settings changed via Admin OR scripted – updates DB

1




env.phpconfig.php

config:set

app:config:dump


config:set --lock

env.phpconfig.php



/admin

GitBuild

magentoapp:config:dump

Changed settings are written to config.php and env.php

2




env.phpconfig.php

config:set

app:config:dump


config:set --lock

env.phpconfig.php



/admin

GitBuild

config.php

Configuration settings pushed as code (“locked” in Admin)

3




env.phpconfig.php

config:set

app:config:dump


config:set --lock

env.phpconfig.php



/admin

GitBuild

magento config:set --lock

Locked values can be overridden in local env.php

4




env.phpconfig.php

config:set

app:config:dump


config:set --lock

env.phpconfig.php



/admin

GitBuild

magentoconfig:sensitive:set

Interactive mode to prompt for sensitive values for env.php

5

Other Recommendations


Git Repository

Marketplacecomposer require ...

Source Code Management Recommendations

composer.jsoncomposer.lock

vendor

Git Repository

Build Process

Alternative: publish to a private packagist repo

https://packagist.com/

Production

"config": {"preferred-install": {

"alankent/*": "source","*": "dist"

}}

composer install --prefer-sourceVS

Caches well


• Build process– Think about how to update tool chain, or customize per project– Git clone/pull project repo to get the code – needs repo keys– composer install – keep cache between runs for speed/traffic– Decide if test files should be in deployment bundle

• Test Automation – run functional tests on the production deployment build• Separate Integration Test & Production environments?

– Want environments close to each other– Minimize env.php (keep environments

as close as possible)

Build & Bundle Recommendations

Staging Production


Developer 1 Developer 2


Build


Developer 1 Developer 2


• Read-only file system (/var and /generated now separate)• Plan out your configuration settings per environment• Remember config.php and env.php are executable code• Have build phase create common bundle to share amongst environments• Swap files in – use symlink, change config file, blue/green servers• Consider serving stale content from cache during cache flush• Decide where to deploy

– Magento Cloud– Zerolag, Nexcess, Rackspace, Byte,

Simple Helix - more done for you– Docker / AWS / Azure - set it up yourself

Deployment Recommendations

Staging Production


Build

Conclusions


1. Use Admin to define websites, stores, store views in database2. Enable/disable modules via config.php3. Developer uses Admin or config:set to set store configuration settings4. Developer runs app:config:dump to create config.php and env.php

– Shared à config.php– Environment specific (sandbox vs live endpoints, …) à env.php– Sensitive (passwords, PII, …) setting names à config.php, values ->

env.php5. The config.php file is committed to git for sharing

Workflow Review – Developer Environment


1. Get updated copy of config.php along with rest of latest code2. Run app:config:import to update scopes and themes in database3. config:sensitive:set can be used to set any missing sensitive settings4. config:set will not override “locked” values already in config.php/env.php5. config:set --lock will add value to env.php even if “locked”6. Update database schema, if needed (with store in maintenance mode)7. Flush caches, go live

Workflow Review – Target Environments


• Command to detect if schema changed for optimized deployments• Declarative schemas to make schema changes easier• Further standardization of

– Development environment (DevBox still in beta)– Build pipeline commands– Test automation invocation (and other improvements)– Hosting partner specific deployment processes

Future

Alan Kent@akent99

Q&A

Pipeline DeploymentDeployment flow

Pipeline DeploymentConfiguration Management


Sample Developer and Production Environments


env.phpconfig.php env.phpconfig.php env.phpconfig.php env.phpconfig.php


GitBuild


Overview of StepsDevelop

Build & Bundle

Deploy to Test/Staging

Deploy to Prod

Identify per environment settings (including those used by extensions)Commit shared configuration (config.php) to git

Depends on production shared production mode settings(Must not depend on dev or specific prod environment settings)

Development Mode Settings

Production Mode Settings

Environment variables, env.php, (config.php), database settings


TODO: CONFUSING SLIDE – REDO?DEFINE LOCKING EARLIER



Build & Bundle


Deploy to Prod









Build & Bundle


Deploy to Prod







Developer controlledShared settings

Merchant controlledPer environment settings

Developer controlledPer environment settings

Documents

New Magento 2.2 Deployment New Magento 2… · Flexibility in what to do in build vs deploy phases, what is in git, ... Pipeline Deployment Deployment flow. Pipeline Deployment Configuration