View
43
Download
5
Category
Tags:
Preview:
DESCRIPTION
Citation preview
© 2011 Mayflower GmbH
One Click Deployment with Jenkins
Thorsten Rinne I 12th October 2011, Mainz, Germany
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Team Lead / Scrum MasterHead of Open Source Labs
@ThorstenRinne2
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Who are you?
3
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Developers?
4
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Administrators?
5
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
DevOps?
6
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Continuous Integration?
7
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
How much time do you need for a release?
8
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
More than a year?
9
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
6 - 12 months?
10
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
1 - 6 months?
11
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
2 - 4 weeks?
12
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
1 - 14 days?
13
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
1 - 24 hours?
14
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Less than 15 minutes?
15
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
15 Minutes!
16
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Why One Click Deployment?
17
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Everything is getting faster.
18
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Faster feedback from your customer / user
19
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Faster ROI (return on invest)
20
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Faster responseif the market changes
21
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 22Developers are lazy.
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Basic requirements
23
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Agile software development
24
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Scrum or Kanban
25
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Test Driven Development
26
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 27
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Continuous Integration
28
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Continuous Inspection
29
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Refactoring
30
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 31
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 31
ScrumKanban
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 31
ScrumKanban
ContinuousIntegration
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 31
ScrumKanban
ContinuousIntegration
ContinuousInspection
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Continuous Deployment
31
ScrumKanban
ContinuousIntegration
ContinuousInspection
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Things you don‘t want.
32
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Having your SVN repo in your webserver‘s docroot.
33
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Using SSH and vi and your root account on the live
server. Never.
34
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Who‘s still doing this? :-)
35
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 36
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
What do we need?
37
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
First: a central build server
38
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 39
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Continous Testing with PHPUnit
40
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Advantages
41
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Test results at one central page
42
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Software metrics at one central page
43
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Errors can be identified fast and then fixed
44
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Awareness
45
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 46
Awareness!
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Continuous Integration in PHP projects
47
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
PHPUnitDocBlox
48
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Continuous Inspection in PHP projects
49
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
PHPCSPHP Depend
PHPMDphpcpd
jslint
50
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
jslint?yes!
51
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Infrastructure
52
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 53
http://www.flickr.com/photos/nirak/644337888/
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
One Ubuntu package to rule them all.
54
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Development VMsJenkins CI ServerSelenium Server
Monitoring ServerTesting Server
55
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I
Only on production:Frontend Nodes 11.04
Backend Nodes 10.04 LTS
56
http://www.flickr.com/photos/nirak/644337888/
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 57
☁☁ ☁
Mayflower DMZ
Eucalyptus Cloud
amazon AWS
http://www.flickr.com/photos/nirak/644337888/
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 58
☁☁ ☁
Mayflower DMZ
Eucalyptus Cloud
amazon AWSDeveloper Laptopwith Ubuntu VM
(Git repo + completesoftware stack)
http://www.flickr.com/photos/nirak/644337888/
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 59
☁☁ ☁
Mayflower DMZ
Eucalyptus Cloud
amazon AWSGitorious ServerJenkins CI Server
Puppet MasterSelenium Server
http://www.flickr.com/photos/nirak/644337888/
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 60
☁☁ ☁
Mayflower DMZ
Eucalyptus Cloud
amazon AWS
2 Frontnodes3 Backendnodes
Development MonitorStaging Monitor
http://www.flickr.com/photos/nirak/644337888/
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 61
☁☁ ☁
Mayflower DMZ
Eucalyptus Cloud
amazon AWS2 Frontnodes2 Backendnodes
DrupalLive Monitor
http://www.flickr.com/photos/nirak/644337888/
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 62
Configuration Management done with
http://www.flickr.com/photos/nirak/644337888/
Building a Cloud-based social network I Mayflower GmbH I 11th October 2011 I 63
Configuration changes are on every VM in 5 Minutes.
#WIN
http://www.flickr.com/photos/nirak/644337888/
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 64
Frontnode 1Varnish nginx
PHP 5.3
Frontnode 2Varnish nginx
PHP 5.3
Backendnode 3RabbitMQ
Apache SolrPHP 5.3
Backendnode 2MySQL Slave
MogileFSMemcached
PHP 5.3
Backendnode 1MySQL Master
MogileFSMemcached
PHP 5.3
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Development workflow
65
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
1. git workflow
66
$ git add Foo.php$ git commit -m ″Bar″$ ant jslint$ ant phpunit ...$ git push
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
2. Post commit hook
67
Build triggered on Jenkins
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Jenkins build checks
68
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
1. PHPUnit Build:Unittests
Controller Tests (Zend_Test)
69
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
2. Debian Package Build
70
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
3. Deploy to staging4. Deploy staging to live
71
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
At midnight:DocBlox
Static code analysis
72
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
What is a build process?
73
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
ant syncant clean
ant db-updateparallelTasks:
ant phpunitant phpcbant deb
74
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
<target name="phpunit" depends="db-test" description="Run all tests"> <exec executable="phpunit" failonerror="true"> <arg line="--configuration ${basedir}/tests/phpunit-verbose.xml ${basedir}/tests/AllTests.php" /> </exec></target>
75
ant phpunit
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
parallelTasks:php_depend
phpmdphpunitphpcpdphpcsjslint
76
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 77
Build Debian Packageant autoloaderant phpunitant clean
ant compilejsant compilecss
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
<target name="deb" depends="autoloader,phpunit-coverage,clean,autoloader,compilejs,compilecss" description="Generate Debian package"> <exec executable="gpg"> <arg value="--armor" /> <arg value="--import" /> <arg value="scripts/build/deb/mapolis.mayflower.de.pgp" /> </exec> <buildnumber />
78
ant deb (I)
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
<exec executable="git" outputproperty="githash"> <arg value="describe" /> <arg value="--always" /> <arg value="--match" /> <arg value="" /> </exec> <touch file="debian/changelog" /> <move file="debian/changelog" tofile="debian/changelog.old" />
79
ant deb (II)
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
<exec executable="git" output="debian/changelog.new"> <arg value="log" /> <arg value="-n" /> <arg value="1" /> <arg value='--format=format:mapolis-web (${build.number}-${githash}) unstable; urgency=low%n%n * %s%n%n -- %an <%ae> %aD%n%n' /> </exec>
80
ant deb (III)
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
<concat destfile="debian/changelog"> <fileset file="debian/changelog.new" /> <fileset file="debian/changelog.old" /> </concat> <delete file="debian/changelog.new" /> <delete file="debian/changelog.old" />
81
ant deb (IV)
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
<exec executable="dpkg-buildpackage"> <arg value="-b" /> <arg value="-v${build.number}-${githash}" /> <arg value="-Zlzma" /> <arg value="-z9" /> <arg value="-k78532445" /> <arg value="-pgpg" /> <arg value="-sgpg" /> <arg value="-mMayflower Mapolis Team <mapolis@mayflower.de>" /> </exec>
82
ant deb (V)
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
<move todir="build/deb/"> <fileset dir="${basedir}/../"> <include name="mapolis*.deb" /> <include name="mapolis*.changes" /> </fileset> </move> <exec executable="dpkg-buildpackage"> <arg value="-Tclean" /> </exec> </target>
83
ant deb (VI)
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Deploy Stagingant clean
ant deb-signant db-staging-upscp to all hosts
84
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 85
Deploy Staging to livescp to all hosts
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Disadvantages and problems
86
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Build time15 min on harddisk
massive disk i/o4 min on ramdisk :-)
87
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Complexity beginning such a infrastructure
88
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Jenkins:Single point of failure!
89
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Cool stuff which is now possible
90
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 91
Deploy by customer. :-)
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I 92
Jenkins Chuck Norris Plugin
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
git bisect start $BAD $GOOD
93
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Minimum time from bugreport to deploy:
30 minutes
94
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
$ git branch hotfix-2011-08-22$ git checkout hotfix-2011-08-22$ git reset --hard "last stable sha-1"$ git cherry-pick "hotfix sha-1"$ echo build.number=23 > build.number$ ant deb$ scp build/deb/*.deb xyz.amazonaws.com:/opt/repository/$ git checkout master$ git push origin hotfix-2011-08-10
95
LiveDemo
Continuous Improvement in PHP Projects I Mayflower GmbH I 25th February I
Questions?
97
Thank you very much for your attention!
© 2011 Mayflower GmbH
Contact Thorsten Rinnethorsten.rinne@mayflower.de+49 89 242054-31
Mayflower GmbHMannhardtstr. 680538 MünchenGermany
Please rate me @ http://joind.in/talk/view/3904
Recommended