Embed Size (px)
DESCRIPTION
These are the slides of my talk at Foss.IN 2012 on Continuous Integration for Open Source Distribution ecosystems. I have covered how existing Continuous Integration efforts can be improved to ensure greate collaboration between open source projects, developers and distributions to make earlier r
Citation preview
Continuous Integration
For
Open Source Distributions
Sriram Narayanan
@sriramnrn
About me
Belenix Tester + Developer
Thoughtworker
Why we work on distros
● Innovate● more versatile installation● faster boot up times● experimental drivers
● Learn package management (Rolling updates!)
● Learn how a distro works (LFS,BLFS + more)
Pain points for a Distro Maintainer
● Repeat Tasks
● Dependency Tracking and Management
● Manual Testing
Pain points for a Distro Maintainer
● Repeat Tasks● Track upstream changes and fixes● Regular Distro builds● Changelog and Release Notes● Documentation Updates
● Dependency Tracking and Management
● Manual Testing
Pain points for a Distro Maintainer
● Repeat Tasks
● Dependency Tracking and Management● Dependencies Expressed by not Tested● Insufficient Integration Tests● Heavily dependent on manual testing
● Manual Testing
Pain points for a Distro Maintainer
● Repeat Tasks● Dependency Tracking and Management● Manual Tests
● Package Integration● Installation● Desktop Application Functionality● Service Functionality
● Testing at too many levels● Not just exploratory (More exploratory is good)
Continuous Integration - Basics
● “Integrate changes continuously”● Checkout source, make, make test, package● Usual Implementation
● Automated Agents that perform tasks● Farm of Agents depending upon scale and budget
– Great for packaging multiple apps in parallel– Parallel Tests
Continuous Integration - Benefits
● Detect issues earlier● Package software earlier● Automate various tasks
● Packaging● Testing● Integration● Generate ChangeLog + Release Notes skeleton● Pending Documentation Hints to Documentation
Team● Build the ISO!
Continuous Integration Today
● Isolated build environments● Based on chroot jails● Used by Fedora, Debian, Suse, et al
● Continuous Integration servers● Jenkins/Hudson● Koji● Others (distro specific)
Continuous Integration – What's missing
● Dependency awareness● A “Green” upstream should trigger downstream
builds● Complex Dependency graph builds and tests● Prevents Automated Consolidation Releases
● A Variety of Testing● “make test” is not enough!
Continuous Integration – What's missing
● Dependency Awareness
apr
httpd
subversion
Image Credits: http://cd-with-thoughtworks-go.blogspot.in/
Why “make test” is not enough
● Tests the app only on the compiled system● No tests post installation
● Packaging occurs after make test● Packaging errors are not detected
– Incorrect placement, ownership and permissions– Conflicting requirements– Missing files
● Integration Tests needed
./configure makemake make test package
Testing
Unit
Service Layer(Integration)
UserInterface
The Test Pyramid – Mike Cohn
http://martinfowler.com/bliki/TestPyramid.html
About Unit Tests
● Unit Tests● Requirements in executable form● Detect what exact functionality has broken
About Integration Tests
● Service Level Tests● Is ComponentA correctly calling ComponentB?● Is ComponentA fulfilling its public API contract?
About UI Level Tests
● Test End to End Functionality● e.g. Sahi based tests
What Tests tell you
● UI Tests - “Something in this feature broke”
● Integration Tests - “Component B returned an unexpected response to Component A”
● Unit Tests - “When we introduce functionality requirement #13 into Component B, then our code causes requirements #7 and #9 to break”.
Post – Installation Integration Tests
● CentOS package level tests● A good start!
● Some useful tests for distro maintainers– Is httpd able to serve files ? (selinux issues!)– Is SSH accepting connections ?– Did the network subsystem initialize correctly?– Did the installer set the MBR + active partition correctly?– Is the DNS server accepting queries ?
● Is the application behaving as required?
CI in Belenix
● Distro tests + Consolidation level smoke testskernel
httpd
postgres
ssh
WebStack
nginx
python
httpd
vi Userland
shell
awkIntegration
Tests
kernel
AutomatedInstall
Post InstallTests
Distro + package levelsmoke tests
Roadmap - CI and Belenix
● Package early● Consolidations – kernel, userland, subsystems
(storage, firewall, load balancer)● Custom distros – CoreOS, storage, firewall,
load balancer● Each with its own functional test suite!
● CI for open source projects that we bundle● Potential inter-distro inter-project collaboration
