Automated RPM based Java Artefact Deployments

With Puppet for Red Hat Based Linux Systems

Code that is written and not deployedis money wasted.”

Jesse Robbins, Opscode

My Current Problem

Tomcat Application Deployment where:• Artefacts are supplied as a drop in WAR/JAR file built by Maven

• Environment specific configuration items are not externalized to the WAR/JAR e.g. hardcoded hostnames and databases in hibernate.xml

• Environment variables are not being used to define different environments e.g. java –Denv=dev

• Due to Rapid development there are many releases and files types and their uses are in flux e.g. changing formats in hibernate.xml

Anti-patterns

The way we currently deploy our Java applications is an anti-pattern (ConfigurationBirdsNest*)

Example deploy of mytomcatapp application:$ service tomcat6 stop

$ rm /var/lib/tomcat6/webapps/mytomcatapp.war

$ rm –rf /var/lib/tomcat6/webapps/mytomcatapp

$ rm –rf /var/cache/tomcat6/work/Catalina/localhost/mytomcatapp

$ service tomcat6 start

$ wget –O /var/lib/tomcat6/webapps/mytomcatapp.war

http://path/to/mytomcatapp.war

$ sleep 30

$ service tomcat6 stop

$ vi /var/lib/tomcat6/webapps/mytomcatapp/META-INF/context.xml

$ vi /var/lib/tomcat6/webapps/mytomcatapp/WEB-

INF/classes/application.properties

$ vi /var/lib/tomcat6/webapps/mytomcatapp/WEB-

INF/classes/hibernate.cfg.xml

$ vi /etc/tomcat6/Catalina/localhost/mytomcatapp.xml

$ service tomcat6 start

*http://code.google.com/p/devops-toolchain/wiki/ConfigurationBirdNest

Why are these anti-patterns?

It’s slow

It’s risky (typing errors anyone?)

It requires detailed knowledge of the application

Configuration files and their purpose are in flux and require constant updating of puppet (or insert configuration management tool of choice here) code

Design PatternsAdaptive Deployment* & Code Data Split**

*http://code.google.com/p/devops-toolchain/wiki/AdaptiveDeployment

** http://code.google.com/p/devops-toolchain/wiki/CodeDataSplit

A problem shared is a problem halved…

Communicating to our Developers why this causes us pain

Why we should either:• Externalize our configuration with overrides• Or use environment variables to separate them• Or tokenize them

Why War files don’t fully meet our needs in Ops• Version querying• Audibility• Speed• Fast rollback

Design PatternRPM Packaged Artifact* based delivery for Java

*http://code.google.com/p/devops-toolchain/wiki/PackagedArtifact

Proposal

Using RPM-Maven-Plugin* • Automated Red Hat Package creation of Java applications and deploy them to your software artefact repository

Leveraging Jenkins** (or insert CI tool of choice here) to automatically build them as artefacts alongside the war/jar’s

Using Jenkins ssh plugin*** to copy rpm to cobbler repo and perform a “cobbler reposync”

*http://mojo.codehaus.org/rpm-maven-plugin/

**http://jenkins-ci.org/

*** https://wiki.jenkins-ci.org/display/JENKINS/SSH+plugin &

https://wiki.jenkins-ci.org/display/JENKINS/Publish+Over+SSH+Plugin

Why?

Just by adding something like this to your mavenpom.xml build files:

<properties>

<app.home>/var/lib/tomcat6/webapps/mytomcatapp</app.home>

</properties>

...

<plugin>

<groupId>org.codehaus.mojo</groupId>

<artifactId>rpm-maven-plugin</artifactId>

<version>2.0.1</version>

<executions>

<execution>

<goals>

<goal>rpm</goal>

</goals>

</execution>

</executions>

<configuration>

<copyright>2011, Uncommon Sense Consulting</copyright>

<group>Development</group>

<description>Maven Recipe: RPM Package.</description>

<mappings>

<mapping>

<directory>${app.home}</directory>

<sources>

<source>

<location>target/mytomcatapp</location>

</source>

</mapping>

</mappings>

</configuration>

</plugin>

Why?

We can get a Red Hat Package

mytomcatapp-$version.rpm – contains

/var/lib/tomcat6/webapps/mytomcatapp

/var/lib/tomcat6/webapps/mytomcatapp/META-INF

/var/lib/tomcat6/webapps/mytomcatapp/META-INF/maven

/var/lib/tomcat6/webapps/mytomcatapp/WEB-INF

/var/lib/tomcat6/webapps/mytomcatapp/WEB-INF/classes

/var/lib/tomcat6/webapps/mytomcatapp/WEB-INF/lib

Why?

Need to install a fully functioning mytomcatapp in Development?

$ yum install mytomcatapp

Dependencies Resolved

================================================================================

Package Arch Version

================================================================================

Installing:

mytomcatapp noarch 1.2.3-1

Installing for dependencies:

oracle-instantclient11.2-jdbc x86_64 11.2.0.2.0-1

oracle-instantclient11.2-basic x86_64 11.2.0.2.0-1

tomcat6 x86_64 6.0.26-1

tomcat6-servlet x86_64 6.0.26-1

tomcat6-jsp-2.1-api x86_64 6.0.26-1

tomcat6-el x86_64 6.0.26-1

tomcat6-lib x86_64 6.0.26-1

Transaction Summary

================================================================================

Install 8 Package(s)

Upgrade 0 Package(s)

Total download size: 58 M

Is this ok [y/N]:

Puppet

Current Puppet Module

class mytomcatapp::install {

$packagelist = ["oracle-instantclient11.2-jdbc"]

package{ $packagelist: ensure => installed }

file{"/var/lib/tomcat6/webapps/mytomcatapp.war":

source => ["puppet:///modules/mytomcatapp/var/lib/tomcat6/webapps/${fqdn}.mytomcatapp.war",

"puppet:///modules/mytomcatapp/var/lib/tomcat6/webapps/${custom_env}.mytomcatapp.war",

"puppet:///modules/mytomcatapp/var/lib/tomcat6/webapps/default.mytomcatapp.war"],

owner => "tomcat",

group => "tomcat",

notify => Class["tomcat6::service"],

mode => 644,

}

}

...100 lines later...

class mytomcatapp::portal {

include mytomcatapp::common

include mytomcatapp::install

include mytomcatapp::config

include mytomcatapp::backup

include mytomcatapp::monitor

}

Impact on Puppet

New Puppet Module

class mytomcatapp::install {

$packagelist = [$mytomcatapp::common::package]

package{ $packagelist: ensure => installed }

}

class mytomcatapp::monitor {

#noop

}

class mytomcatapp::portal {

include mytomcatapp::common

include mytomcatapp::common::environment

include mytomcatapp::install

include mytomcatapp::monitor

}

Value Proposition

Simplicity in application deployment – lower overhead, puppet friendly runtime changes

Massive speed improvement in time to deploy and upgrades• Bare metal deploy estimate is 5 minutes down from 8 minutes• Application upgrade is 60 seconds down from 60 minutes

Auditability – upgrades are logged and version’s easily and programmatically checked:

$ rpm –qi mytomcatapp

Rollback is as simple as:

$ rpm –e mytomcatapp

$ yum install mytomcatapp-$previousversion

SimplicityPop Quiz! What’s easier?

This?

$ service tomcat6 stop

$ rm /var/lib/tomcat6/webapps/mytomcatapp.war

$ rm –rf /var/lib/tomcat6/webapps/mytomcatapp

$ rm –rf /var/cache/tomcat6/work/Catalina/localhost/mytomcatapp

$ service tomcat6 start

$ wget –O /var/lib/tomcat6/webapps/mytomcatapp.war

http://path/to/mytomcatapp.war

$ sleep 30

$ service tomcat6 stop

$ vi /var/lib/tomcat6/webapps/mytomcatapp/META-INF/context.xml

$ vi /var/lib/tomcat6/webapps/mytomcatapp/WEB-

INF/classes/application.properties

$ vi /var/lib/tomcat6/webapps/mytomcatapp/WEB-

INF/classes/hibernate.cfg.xml

$ vi /etc/tomcat6/Catalina/localhost/mytomcatapp.xml

$ service tomcat6 start

Or this?

$ yum upgrade mytomcatapp

Bonus points if you guess what’s faster!

Auditability

Question – What version of mytomcatapp is in X environment?

[actionjack@rasengan]$ rpm -qi mytomcatapp

Name : mytomcatapp Relocations: (not relocatable)

Version : 1.2.4 Vendor: Uncommon Sense

Release : 1 Build Date: Fri 07 Oct 2011 06:21:10 PM BST

Install Date: Fri 07 Oct 2011 06:31:10 PM Build Host: rasengan.uncommonsense.local

Group : Development Source RPM: mytomcatapp-1.2.4-1.src.rpm

Size : 17922480 License: 2011, Uncommon Sense

Signature : (none)

Packager : Martin Jackson <martin@uncommonsense-uk.com>

Summary : Unnamed - com.uncommonsense-uk:mytomcatapp:war:1.2.4

Description :

My Tomcat App.

Question – Has anybody changed anything?

[actionjack@rasengan]$ rpm --verify mytomcatapp

[actionjack@rasengan]$ echo I guess not...

Rolling Back

Rapid recovery needed?

[actionjack@rasengan]$ sudo yum upgrade mytomcatapp

[actionjack@rasengan]$ sudo tail /var/log/tomcat6/catalina.out

mytomcatappv1.3.1-alpha starting...

Err something isn’t quite right..

I feel sick...

I’m gonna puke.. � � � � �

[actionjack@rasengan]$ sudo rpm –e mytomcatapp

[actionjack@rasengan]$ sudo yum install mytomcatapp-1.2.4

[actionjack@rasengan]$ sudo /etc/init.d/tomcat6 restart

[actionjack@rasengan]$ sudo tail /var/log/tomcat6/catalina.out

mytomcatappv1.2.4-alpha starting...

Top of the morning to you!

Ready for business.

[actionjack@rasengan]$

Using RPM-Maven-Pluginto build a test mytomcatapp RPM.

Questions anda Call to Action

Lets make things suckjust a little bit less!