Project-Builder.org Presentation

© Bruno Cornec / HP - Document under/sous CC 3.0 by SA Date : 02/06/2010 Page 1

Project-Builder.org

Project Builder:

A GPL continuous packaging solution

Bruno Cornec

Open Source and Linux Profession EMEA Lead

mailto:[email protected]

http://www.hp.com/linux


Project-Builder.orgIntroducting myself

Software engineering and Unices since 1988

Mostly Configuration Management Systems (CMS), Build systems, quality tools, on multiple commercial Unix systems

Discover Open Source & Linux (OSL) & first contributions in 1993

Full time on OSL since 1995, first as HP reseller then @HP

Currently:

Technology Architect on OSL for the HP/Intel Solution Center, Grenoble

OSL HP Ambassador

EMEA OSL HP Profession Lead

Solutions Linux Conference board member

MondoRescue, Dploy.org, Project-Builder.org project lead

LinuxCOE, mrepo, tellico, rinse, fossology contributor

Mandriva, Fedora packager




Project-Builder.orgWhat is the problem ?

Users / Sysadmins environment:

< 1%

< 10%

>= 90%

Users: easyness (GUI, CLI)Admins: distribution compliance, smooth integration – reduce admin load




Project-Builder.org

Project

Flat Files

CMS

CVSSVNGIT, ...

What is the problem ?

Developers environment:

autotools ?Install proc ?Soft. Eng. ?Fear of distro ?

Sourceforgeftp, web




Project-Builder.org

Package

Flat Files

CMSCVSSVNGIT, ...

What is the problem ?

Packagers environment:

rpmlint ?lintian ?

Distroftp, web

Does s/he only exists ?On time ? Lag behind ?Distro coverage (ver, #)Beta avail ?




Project-Builder.org

Packaging should be a project concern as well as coding, testing, installing, .... especially for smaller projects

Package early, package always

Use packaging as your only way of delivery (not a dream)Overhead minimal (not much longer than providing a tgz), maximum benefit (consistancy, reproduceability, team work improvement, distribution integration, improved deployment)

Packaging is in fact a marketing activity based on a technical content. It's an easy way to extend your user base, and improve your community. Makes tests easy.

THE SOLUTION IS CONTINUOUS PACKAGING

What is the solution ?




Project-Builder.orgMulti platform life cycle

User Requirements

FunctionalSpecification

DesignSpecification

Coding

Unit tests

Integration tests

Validation




Project-Builder.org

I'm a sysadmin so I want to deal with packages only (deployment server integration)

I'm a developer so I want to ease my tests without screwing up my system and testing as a std user/admin

I'm a a packager so I want to streamline that work for new projects I'm interested in packaging

I'm the marketing department of my own project and use packages as a “competitive advantage”

I'm a software engineer and want no duplication of code nor information (metadata)

Founding no tool existing to help me doing all that, so I started writing one to scratch my own itch

My answer




Project-Builder.org

Project-builder main goal is to help you package continuously being agnostic:

CMS agnostic (no CMS but guys it's 21st century now, SVN, CVS, Mercurial, GIT, SVK....)

OS agnostic: Linux: RPM, deb, ebuild, slack based, ... 60+ distros at the moment – repositories for yum, urpmi, apt. Solaris pkg.

Build environment agnostic: local, VM (QEMU, KVM...), VE (rinse, mock, debootstrap...)

No project impact (preserves the md5sum of the delivered upstream sources). Can be completely external.

Goals




Project-Builder.orgMulti platform development cycle

.a

.o

.o.o.o .o.o.o

.o.o.o .o.o.o

.o.o.o .o.o.o

.o

SSPK

SLPK

.o

.a< 5%

90+%

< 10%




Project-Builder.org

Project-builder provides additional goodies:

Set of existing VMs/VEs or easy to recreate

Macro system with perl variables to avoid duplication

Skeleton generation to help starting

Manages delivery up to your ftp server (ssh based)

Manages announces on mailing lists

May use snapshot features of VMs/VEs

Integration of tests in the process

Manages patches when not upstream

Checks validity of packages built (lintian, rpmlint)

Easy creation of new versions for upstream management

Goodies




Project-Builder.org

upstream CMS repository

pbconf CMS repository

project-builder.org

local packaging& test system

1

2

upstream Web/ftp server

2

1 pbconf Web/ftp server

pbconf file/dirpbconf file/dir

upstream file/dir

2

2

3

4

5

ftp/ssh/nfs server

mailserver

CHROOT

ebuild

Architecture

1




Project-Builder.org

ProjectPackages

cms2build

build2vm

build2pkgpkg2ssh

newvm / setupvm

newver / newproj

The Big picture

build2sshDevelopers

Packagers




Project-Builder.org

cms2build: Create tar files for the project under your CMS. CMS supported are SVN and CVS parameters are packages to build if not using default listbuild2pkg: Create packages for your running distributionbuild2ssh: Send the tar files to a SSH hostpkg2ssh: Send the packages built to a SSH hostbuild2vm, build2ve: Create packages in VMs/VEs, launching them if needed and send those packages to a SSH host once builtlaunchvm: Launch one virtual machinescript2vm, script2ve: Launch a VM/VE if needed and executes a script on ittest2vm, test2ve: Test inside a VM/VEnewvm, newve: Create a new virtual machinesetupvm, setupve: Setup a virtual machine for pb usagenewver: Create a new version of the project derived from the current onenewproj: Create a new project and a template set of configuration files under pbconf

Commands




Project-Builder.org

Packages

Multi platform packaging cycle

Developers

Packagers

VMs/VEs

VM/VE




Project-Builder.org

More Tests on VEs (mock, debootstrap to be done, rinse OK + patches) LSB chroot support.

Support for libvirt, virsh

Other CMS (Bazaar, …) only when/if needed

Other VMs (VMWare, Xen, ...) only when/if needed

Multiple delivery means

Look at interactions with Buildbot

Add signature support

Create a reference documentation

Config-Model for configuration file management

To be done




Project-Builder.org

Web Resources

Project-Builder Trac / Wiki

http://trac.project-builder.org

Open Source at HP

http://opensource.hp.com

Other tools: SuSE/Fedora/Mandriva Build Systems, Buildbot, vcs-pkg.org

“The evolution of FLOSSand the Internet are

tightly coupled”



http://trac.project-builder.org/

http://opensource.hp.com/


Project-Builder.org

”Changes are never easy to make. There is comfort and safety in tradition, but change must come, no matter how painful or expensive it may be.”

Bill Hewlett

Contact

Thanks

[email protected]

(Linux Solution Consultant in the HP/Intel Solution Center)


Linus Torvalds, Richard Stallman, Eric Raymond, Nat Makarevitch, René Cougnenc, Eric Dumas, Rémy Card, Bdale Garbee, Hugo Rabson, Gallig Renaud, Andree Leidenfrost among others, for their work and devotion to the Open Source Software cause... and my family for his patience :-)






Project-Builder.org

V0.1-v0.4: script shells – linked to mondorescue – Usage of QEMU to generate cross distros packages

V0.5 (08/2007): rewritten in perl, keeping the principles, independent of the project – pbconf in the project – filtering support with macros – PoC state

V0.6: support multiple versions, perl conventions

V0.7: perl env var substitution in conf files for macros – first usage for mondorescue and itself including VM.

V0.8: functionaly similar to the original scripts (except ML, announces, ...). Multi CMS support. Trac BR. Removes AppConfig dep

V0.9: re-architectured for no project impact (pbconf outside of project, multi-conf file support, CMS independance, VM/VE, separate tar files for build info...)

History

