Embed Size (px)
1Gerrit User Summit, Mountain View, 2014
Gerrit @ LibreOffice
David Ostrovsky
2Gerrit User Summit, Mountain View, 2014
Outline
Motivation to introduce GerritDecentralized CI InrastructureAdd Gerrit to CI InfrastructureGerrit plugin APIGerrit Buildbot plugin
3Gerrit User Summit, Mountain View, 2014
Introduction of Gerrit for LibreOffice: Motivation
4Gerrit User Summit, Mountain View, 2014
Patches Lost in Space: the old workflow
A patch is intended to:Be mailed to the dev-@mailinglistBeing picked up by a developer with commit rights and be
Checked for a valid licenseReviewed for GoodnessApplied, Build, Tested and PushedReported back to the mailing list that this is in now
Reality: fdo#40946
5Gerrit User Summit, Mountain View, 2014
http://upload.wikimedia.org/wikipedia/commons/7/71/US_Navy_100627-N-5319A-218_Sailors_and_Marines_leap_from_the_well_deck_of_the_amphibious_transport_dock_ship_USS_New_Orleans_%28LPD_18%29_after_participating_in_the_Crossing_the_Line_ceremony.jpg By US Navy and thus public domain
6Gerrit User Summit, Mountain View, 2014
Does it hurt?
Duplication of work
Frustration
Scaring away newcomers
Makes some kinds of work very hard or painful for everyone
8Gerrit User Summit, Mountain View, 2014
How does Gerrit help?
Automatic Patch UploadsIntegration in a wide range of communication channelsHuman ReviewBot Verification
Thus:Find errors early and before they cause too much pain
9Gerrit User Summit, Mountain View, 2014
Gerrit@LibreOffice
Gerrit version: 2.8.3one core projectmany other pojects (cpp unit, ...)80 committersreview workflow is optional on masterreview workflow is mandatory for release branches
11Gerrit User Summit, Mountain View, 2014
LibreOffice CI Infrastructure
12Gerrit User Summit, Mountain View, 2014
Decentralized CI Infrastructure
Independent tinderboxes that periodically check master and release branchesMultiple checks for the same commits on the same platforms possibleBuild and configuration logic is located locally in tinbuild2 scriptReport results to tinderbox master per fire and forgetBuildbot server presents results in a web pageIn failure case spam all committers from all changes since last successful build
13Gerrit User Summit, Mountain View, 2014
Decentralized CI Infrastructure
Tinderbox_1
Tinderbox_n
Tinderbox_2
Tinderbox_...RepositoryBuildbot ServerWeb page
14Gerrit User Summit, Mountain View, 2014
Decentralized CI Infrastructure
15Gerrit User Summit, Mountain View, 2014
http://upload.wikimedia.org/wikipedia/commons/1/18/US_Navy_091212-N-3404S-106_Navy_defensive_end_Jabaree_Tuani_%28%5E98%29_and_nose_guard_Chase_Burge_%28%5E93%29_stop_Army_slot_back_Patrick_Mealy_%28%5E5%29_during_the_first_quarter_of_the_110th_Army-Navy_college_football_game.jpg By US Navy and thus public domain
16Gerrit User Summit, Mountain View, 2014
Tinderboxes ...
Tinderboxes have different speeds.
Tinderboxes know nothing of each other.
Tinderboxes cannot communicate or coordinate.
The slowest and rarest tinderboxes (Windows) are the most valuable.
17Gerrit User Summit, Mountain View, 2014
http://upload.wikimedia.org/wikipedia/commons/d/d3/Broken_window_large.jpg CC-BY-SA3 by Tomas Castelazo
18Gerrit User Summit, Mountain View, 2014
LibreOffice CI Infrastructure with Gerrit
19Gerrit User Summit, Mountain View, 2014
Decentralized CI Infrastructure with gerrit 1/3
Queue manager is needed to distribute build jobs between tinderboxesOne patch set must be checked only once for each platformTinbuild2 must be teached to talk to queue managerResults from all tinderboxes are gathered and combined result is reported back to gerritA patch set is approved only when all check jobs were successful
20Gerrit User Summit, Mountain View, 2014
Decentralized CI Infrastructure with gerrit 2/3
Tinderbox_1
Tinderbox_n
Tinderbox_2
Tinderbox_...Queue manager gerrit
21Gerrit User Summit, Mountain View, 2014
Decentralized CI Infrastructure with gerrit 3/3
Tinderbox_1
Tinderbox_n
Tinderbox_2
Tinderbox_...
RepositoryTinderbox MasterWeb page
Queue managergerrit
Reusing existing infrastructureBuildbot teached to talk to queue managerDifferent strategies for gerrit patch verification vs. master check
Prefer gerrit verificationPrefer master check
22Gerrit User Summit, Mountain View, 2014
Queue manager's feature set
Consumption gerrit-stream eventsCommunication API/protocol for tinderboxesJob schedulerBuild log publication and visualizationQueue pipeline visualizationDocumentationNice to have: Gerrit UI integration/extension
dedicated channel for patch set verificationsduring review process specific configuration can be selected to be checked, i. e. with-java
23Gerrit User Summit, Mountain View, 2014
Gerrit Plugin Architecture
24Gerrit User Summit, Mountain View, 2014
Gerrit Plugin Architecture
Extension points for gerrit plugin:SSH commands under plugin's own namespace
Servlets under plugin's own namespace
UiAction (since 2.8)
JS API / Html fragments (since 2.9)
https://review.gerrit.net/plugins/plugin-name/page.html
ssh gerrit plugin-name some-really-cool-command --foo bar baz
25Gerrit User Summit, Mountain View, 2014
Gerrit Buildbot Plugin
26Gerrit User Summit, Mountain View, 2014
Build Trigger strategies
Patch set created eventPositive review from member of specific groupManually
The one actually used in LO set up
27Gerrit User Summit, Mountain View, 2014
Buildbot plugin in a nutshell
Schedule a build per SSH command or UI:
Poll the queue per SSH command:
Report result and publish log per SSH command:
Show queue content per SSH command or UI:
ssh gerrit buildbot get --project core --platform linux
ssh gerrit buildbot schedule --project core <sha1>
cat result.log.gz | ssh gerrit buildbot put --ticket 4711 --failed
ssh gerrit buildbot show --project core
28Gerrit User Summit, Mountain View, 2014
Gerrit Buildbot Plugin
29Gerrit User Summit, Mountain View, 2014
Documentation: per SSH
Documentation: per SSHssh logerrit buildbot put --helpbuildbot put [--] [--failed (-f)] [--help (-h)] [--log (-l) -|LOG] [--succeed (-s)] --ticket (-t) TICKET
-- : end of options --failed (-f) : specify this option if job failed --help (-h) : display this help text --log (-l) -|LOG : url of the job log page or - for standard input --succeed (-s) : specify this option if job was successfull --ticket (-t) TICKET : ticket of the job
30Gerrit User Summit, Mountain View, 2014
Documentation per web UI
Documentation: web page
31Gerrit User Summit, Mountain View, 2014
Using reviewer channel for patch set verification firehose
Results are presented in gerrit's comment tableNice to have: Show CI verification status table on gerrit's change screen: don't mix human reviews and bot verifications
32Gerrit User Summit, Mountain View, 2014
Dedicated communications channel for patch set verifications
Talk later today to implement this as a core feature or as a plugin
33Gerrit User Summit, Mountain View, 2014
Log file publication modes
Buildbot publish mode: local directory / servletDoesn't play nice with multi-master
Jenkins publish mode: delegate publishing to Jenkins instance
external–monitor–job
34Gerrit User Summit, Mountain View, 2014
Buildbot publish mode:
Local directory and servlet
35Gerrit User Summit, Mountain View, 2014
Jenkins publish mode 1/2
Submit a job per RPC
New SSH command contributed to Jenkins
36Gerrit User Summit, Mountain View, 2014
Jenkins publish mode 2/2
37Gerrit User Summit, Mountain View, 2014
Broken Buildbot/Jenkins bridge or why code review can be dangerous
39Gerrit User Summit, Mountain View, 2014
Future development
PersistencePresents results in own table on CS2Allow platforms to be configurableIdle timeoutEngaged time out
41Gerrit User Summit, Mountain View, 2014
All text and image content in this document is licensed under the Creative Commons Attribution-Share Alike 3.0 License (unless otherwise specified). "LibreOffice" and "The Document Foundation" are registered trademarks. Their respective logos and icons are subject to international copyright laws. The use of these therefore is subject to the trademark policy.
Questions ?
