Upload
agnes-bryan
View
217
Download
2
Tags:
Embed Size (px)
Citation preview
ContinuumA World Class Continuous Integration
Tool Stack
April 18, 2023
2
What is Continuous Integration?
• Continuous Integration is generally defined as:
– a software development practice where members of a team integrate their work frequently
– usually each person integrates at least daily - leading to multiple integrations per day.
– each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.
• What does this really mean?
– Instead of going two months, with multiple developer branches and having to do a painful, error prone merge process at the end, changes are merged every day, by the person who wrote the code.
– Breaks to the build, tests, or divergence in design philosophy are spotted quickly and early
3
What is Test Driven Development
• Test Driven Development is generally defined as:
– a development technique where unit tests are written before the actual functional code
– all function units can be exercised and tested in an repeatable manner
• What does this really mean?
– At any time, any piece of code, software module or entire product can be verified by running the unit test(s) associated with it
– It provides early feedback to the developer that• The code they have written is correct• The code they have changed is still correct• They have not unintentionally broken another part of the system
– Any break of a unit test immediately highlights a problem
4
CI and TDD create better quality software
• Employing CI and TDD together ensures that
– You will always know when someone has broken the build
– You will always know when someone has broken a unit test
– You will always know when someone has broken the functional
logic of your code/module/application
– A developer’s code base never diverges too far from the rest of
the team
• Identifying problems as early as possible in the development
process leads to higher quality software products that are cheaper
to develop
ContinuumBeyond Continuous Integration:
Continuous Project Insight
6
What is Continuous Project Insight?
• We live in a networked, hyperlinked world• Our development tools should do too
• Continuous Project Insight is:– A step further than purely integrating code and tests– All tools and the information they provide is interlinked– Knowledge and information is easy to share and navigate
through– Project reports and documentation is always current and
accurate– Developers and management can connect to the heartbeat of a
project with minimal effort
7
What is Continuum?
• Continuum is:
– a world class CI and TDD tool stack
– created to initially support the NextGen project
– assembled from cheap or Open Source components
– primarily focused on (but not limited to) Java development
• It supports:
– Automated builds, unit tests, code coverage, project documentation, etc
– Integrated SCM, Knowledge capture, Issue tracking, Peer Code review
– Access is via• All major web browsers• From within developer tools
– Notification via email, RSS, IM client
8
Why Continuum?
• The Goals for Continuum were:
– Create the best possible CI/TDD tool stack possible
– Be a demonstrable experiment of what CI/TDD can and should
be
– Enable and encourage developers to adopt CI/TDD practices
– Enable managers and project leaders to easily view project
status
– Provide a mechanism for tying all aspects of the development
process into a coherent, interlinked whole
9
Continuum Tool Stack
• The CI/TDD tools used in Continuum are:
– Subversion – Source code repository
– Fisheye – Web based source repository viewer
– Pulse – Continuous Integration build server
– Jira - Issue tracking software
– Confluence Wiki – Knowledge sharing/collaboration
– Crucible – Web base peer code review
– OpenLDAP – user credential management
– Clover – code coverage reports and metrics
– Ant – build system
– testNG – Unit test framework
– yDoc – Javadoc + UML documentation
10
Continuum Tool Relationships
11
Subversion• Source Code Control System• Like CVS, is a Concurrent Versioning System• The bedrock of all the CI stack
– All tools support/integrate with it • Encourages continuous integration in developers
– I.e. developers have local copy and check in frequently– Discourages multi-branch explosion
• No performance issues – everyone has a local copy• Developers can work in disconnected manner
• Works over the VPN – accessed via port 80 • It’s free and open source• All tools to support using it are free• Integrates into the explorer shell (TortoiseSVN)• The manual and books are free• Support for it on all OS platforms• Support for it in all major IDEs
12
Fisheye
• Web based Subversion repository viewer• Supports
– Browsing of project files via the web– Multiple repositories– Related checkins are grouped into changesets– Viewing diffs of file versions– Extensive search capabilities– Displays annotated views of source files (i.e. who changed what)– Historical display of source churn– Graphing of line counts (project, dir, file)– Graphical display of a file’s branch/tag relationships– Linking to issue tracking systems– RSS feeds for change notifications at any level of repository
13
Pulse
• Continuous Integration Build server• Manages continuous integration builds• Triggers any time a developer checks in code• Executes automated
– builds, unit tests, documentation generation, code coverage– and any other artifact we need automated
• Supports RSS, email and Jabber (IM) notification• Links to issue tracking and SCCS viewers• Supports distributed agent builds• Supports scheduling• Has remote API (xml-rpc)• Can build with Ant, Maven, Make or any arbitrary binary
14
Jira
• Issue tracking system• Multi project support (i.e. different repositories)• User configurable dashboard • Fine grained permissions model• Customizable workflow models• Sub-tasks• Time tracking• Configurable reports• Links to Confluence, Subversion and Fisheye• Notification via email, RSS feeds• Export to xml, excel• Remote API• Plugin extension architecture• …
15
Confluence
• The Wiki– Used for knowledge capture, sharing and collaboration– New pages– Threaded comments– News items– Attachments
• WYSIWYG editing of content (as easy as using Word)• Maintains revision history (just like a SCCS)• Fine grained permissions on per space or page• Notification via Email and RSS feeds• Powerful search• Extensive set of user contributed plugins
– Managed from Plugin Manager within Confluence• Dynamically pulls content from Jira• Links to Fisheye (or any system that pushes RSS)• Remote API • …
16
Crucible
• Web based tool for peer code review• Easy to manage review process through web interface• Dynamically pulls changeset information from Fisheye• Workflow notification via Email and RSS• Review information captured in-situ via web browser• Ability to add inline, threaded comments at the review, file, and line
level. • Full auditing capability with historical review access • Annotated view of repository artifacts with inline review comments• Summary reporting of review statistics and metrics • Current running version 0.7. Extensive set of features still to be
added in the future.
17
Clover
• Code coverage analysis tool• Measures effective coverage of unit tests• Fast, accurate, configurable, detailed coverage reporting of Method,
Statement, and Branch coverage • Accurately measures per statement coverage, rather than per-line
coverage • Creates historical trending graphs of code coverage• Full IDE integration• Reports in HTML, PDF, XML or a Swing GUI
18
Coding tools
• Intellij– Preferred IDE– Huge selection of plugins– Most powerful support for power coding– Supports (via plugins) Subversion, Clover, Jira, Ant, TestNG
• Ant– Used for building everything– Lowest common denominator– Industry standard– IDE Support
• testNG– Unit testing framework– Created to solve many deficiencies in JUnit– IDE support
• yDoc– Javadoc extension/replacement– Automatically creates UML diagrams from source at build time– Customizable colors
Implementing Continuum
20
Physical Implementation*
• Hardware:– Single 2U machine– Dual CPU, 4GB Ram– Suse Enterprise Linux 10– Raid 1 disk configuration
• Software (not including tools)– Apache2 web server serves as gateway– AJP or mod_proxy used to pass through to other servers– OpenLDAP – Mysql database– Custom user management web application
• Account creation, verified from corp ActiveDirectory server
• Backups– Nightly backups of all important data– Backed up locally, and to two offsite locations– Rolling 14 day backups
21
Total Cost
• Manpower
– 3 man weeks
• Hardware
– Scavenged box – effectively free
• SoftwareJira $4800
Confluence $4000 (but no cost to this project)
Pulse $1300
Fisheye $1500
Clover $2500
yDoc $1490
Crucible $0 (not a 1.0 GCA product yet)
Total $11590