Strategies for Developing and Deploying Free/Open Source

1

Strategies for Developing andDeploying Free/Open Source

Software

Walt ScacchiInstitute for Software Research

School of Information and Computer ScienceUniversity of California, IrvineIrvine, CA 92697-3425 USA

wscacchi@uci.eduhttp://www.ics.uci.edu/~wscacchi

http://www.ics.uci.edu/~wscacchi/Presentations/OSS-Strategies/

2

Overview

• Background

• Results from recent studies

• Open Laboratory

• Open Source Processes

• Strategies

• References

3

Research methodology

• Individual case studies: significantdetails, but limited (and premature)generalization, little/no comparativeanalysis.

• Few studies examine multiple OSSDprojects in multiple domains– Such studies offer higher degree of

comparative analyses and generalization ofresults.

4

5

6

7

8

Configuration management andwork coordination

• Use CM to coordinate and control who gets toupdate what part of the system– Many F/OSSD projects use CVS (single centralized

code repository with update locks) and frequentreleases (daily releases on active projects)

– Linux Kernel: BitKeeper (multiple parallel builds andrelease repositories)

– Collab.Net and Tigris.org: Subversion (CVS++)

– Apache: Single major release, with frequent “patch”releases (e.g., “A patchy server”)

– GNU arch: Bitkeeper + Subversion + (more)

9

What is free/open source softwaredevelopment?

• Free (as in “freedom”) vs. open source– Freedom to access, browse/view, study, modify and

redistribute the source code– Free is always open, but open is not always free

• F/OSSD is not “software engineering”– Different: F/OSSD can be faster, better, and cheaper

than SE

• F/OSSD involves more software developmenttools, Web resources, and personal computingresources, compared to traditional SE methods.

10

Who is investing in F/OSSD?• Large corporations: (IT and Financial)

– IBM-Eclipse, Sun-NetBeans and OpenOffice, HP-Gelato, Apple-Darwin, Microsoft Research-Rotor, SAP-SAPDB/MySQL, etc.

– Barclays Global Investors, Dresdner Kleinwort Wasserstein,etc.

• Research Universities:– MIT, UC, Indiana, Michigan, Stanford

• Federal Government:– DoD, NSF, NASA

• Mid-size corporations:– RedHat, Novell, Borland

• Small (start-up) companies:– ActiveState (Sophos), Collab.Net, Jabber, Ximian (Novell), JBoss,

Compiere, etc.

11

Findings from F/OSS Studies

• CIO 2002-2003:– OSS primarily for new system deployments– OSS benefits

• enable lower TCO• lower capital investment• greater reliability

– OSS weaknesses:• lack of in-house skills or skills in labor market,• lack of vendor support or vendor viability

– How best to support OSS apps? Do it yourself vs. join aconsortium vs. hire consultants vs. contract with a big vendor

• switching costs

12

Findings from F/OSSD Studies• Hars and Ou 2002:

– >60% of F/OSS developers work on 2-10 F/OSSprojects

• Madey, et al. 2003:– <5% of OSS projects on SourceForge.net sustained;

>90% have only one contributor (i.e., Power Law)• Nichols and Twidale 2003:

– Usability of F/OSS systems generally neglected• Scacchi 2001-2004:

– Open source processes enable org. transformation– Largest F/OSSD projects sustain exponential growth;

most F/OSSD projects fail to grow to any sustainableeffort

• von Hippel and von Krogh 2003:– OSS success relies on “private-collective” innovation model

13

Processes for F/OSS Requirementsor Design (Scacchi 2002)

• F/OSS Requirements/Designs– not explicit

– not formal

• F/OSS Requirements/Designs are embeddedwithin “informalisms”– Examples: threaded email discussion lists, Web

sites, FAQs, source code directories, licenses

• F/OSS Requirements/Design processes aredifferent from their SE counterparts.

14

15

16

17

18

19

Open source processes forRequirements

• Elicitation

• Analysis

• Specification andmodeling

• Validation

• Communicating andmanaging

Partially ordered,centralized process

• Post-hoc assertion• Reading, sense-

making, accountability• Continually emerging

webs of discourse• Condensing and

hardening discourse• Global access to

discourseContinuous,

decentralized process

20

Discovering, modeling and(re)enacting F/OSSD processes

• (Semi)Automated techniques for discoveringprocesses in OSSD projects

• Multi-mode process models– Informal hypermedia, semi-structured, formal

computational enactments/simulations

• Formal models can be visualized, analyzed andvalidated via “process enactment simulators”,and also reused and redistributed.

21

NetBeans.org OSSD ProjectEcosystem

JCP

OpenOffice

W3C

Conflict

Coordination

Coordination

Conflict

Coordination

Conflict

Coordination

Conflict

Mozilla

NetBeans

Apache

Conflict

Coordination

Coordination

Coordination

ConflictConflict

22

Indirect process interactions across projects

Changes in: HTTP, CCS,DOM, URI/URL, XML,

XHTML standards

Bugzilla, compliance with W3C standardprotocols/data formats, compressed HTTP

module support, Javascript support

Browser-specific actions,browser-errorworkarounds,

Tomcat integration intoNetBeans, compliancewith W3C standards,

Apache Ant integrationinto NetBeans

NetBeans workaroundsfor Mozilla shortcuts

NetBeans and Mozilladevelopers collaborate onspell-checking module,

NetBeans adopts Mozillasuper review process

Apache releases newversion of Tomcat

23

Project management and careerdevelopment (Scacchi 2004)

• F/OSSD projects self-organize as a layeredmeritocracy via virtual project management– Meritocracies embrace incremental mutations over

radical innovations– VPM requires people to act in leadership roles based

on skill, availability, and belief in project community

• F/OSS developers use what they build, want tohave fun, exercise their technical skill, try outnew kinds of systems to develop, and/orinterconnect multiple F/OSSD projects (freedomof choice and expression).

24

25

Open Laboratory?

• Free/open source software development– encourages sharing, examination, reuse, modification,

and redistribution

• E-Laboratory– encourages adoption of E-Commerce or E-Business

capabilities in Laboratory operations, administration,research, education

• F/OSSD + E-Lab = Open Laboratory– embraces and facilitates open source and E-Laboratory

systems, processes, practices, and communities

26

Why Open Laboratory?

• Help make Laboratory work faster, better,cheaper.

• Empower interested Laboratory employees,contractors, faculty, students and interestedcitizens to offer help and capture theircontributions.

• Enable creation of public test-beds whereexisting/new Laboratory processes can bedemonstrated, manipulated, and refined.– This may facilitate “participatory science” projects

and science learning/game grids.

27

Open source processes

• Free/open source software does not embody theprocesses for how to develop, deploy, use orsustain them– Deploying F/OSS is low-cost, but often inefficient and

sub-optimal

• Closed source software development,deployment, use, and support is also inefficientand sub-optimal– Explicit open source processes could also help closed

source systems.

28

Motivation for open sourceprocesses

• Closed source processes:– Typically opaque or tacit, difficult to improve, subject

to inappropriate automation or patent by vendors.

• Open source processes:– Enable continuous process improvement and

organizational learning through open access to the“source code” of enterprise processes.

29

Strategies for Developing andDeploying F/OSS

• Requirements and design artifact sharing– Emergent, continuously evolving– Structured vs. semi-structured vs. ad hoc

• Cost information/analysis sharing– Determine ongoing “business value” of F/OSS efforts

• Community and career development– Join/form F/OSSD consortia– Enhance local skill base– Encourage community ownership over individual

contribution/fault– Leverage private-collective action, when possible

30

Strategies for Developing andDeploying F/OSS

• Capture and codify laboratory process domainexpertise as open source processes:– F/OSS systems analysis and design– Deployment and support of F/OSS systems– Usability capture and feedback– Organizational transformation

• Stimulate/facilitate UC-based research intoF/OSSD through partnerships– UC Institute for Software Research (ISR)– UC Computer Game Culture and Technology Lab

• Game Grids for Science Learning (Science Learning Grids)

31

Strategies for Developing andDeploying F/OSS

• Establish Open Lab F/OSS portals andclearinghouse– Create/share process toolkits, libraries, repositories

• e.g., “ProcessSourceForge.net” “SourceForge.NL.org”

• “Co-source” development of OL processes– amortize and share OL development costs across

laboratories• Construct operational OL system and process

demo’s– OL prototypes and public test-beds– Exportable processes for laboratory operations,

administration, research, and education

32

Another game M&Senvironment

33

Open sourcesoftware research

Web site atUCI

34

Acknowledgements

• Project collaborators:– Mark Ackerman, UMichigan, Ann Arbor– Les Gasser, UIllinois, Urbana-Champaign– John Noll, Santa Clara University– Margaret Ellliot, Chris Jensen, Justin Erenkrantz,

Richard Taylor, Jason Robbins, UCI-ISR– Julia Watson, The Ohio State University

• Funding support:– National Science Foundation ITR#-0083075, ITR#-

#0205679, ITR#-0205724, and ITR#-0350754.– No endorsement implied.

35

References

• A. Hars and S. Ou, Working for free? Motivations forparticipating in open source projects, InternationalJournal of Electronic Commerce, 6(3), Spring 2002.

• E.von Hippel and G. von Krogh, Open Source Softwareand the “Private-Collective” Innovation Model: Issues forOrganization Science, Organization Science, 14(2), 209-223, 2003

• G. Madey, V. Freeh, and R. Tynan, Modeling the F/OSSCommunity: A Quantitative Investigation, in Free/OpenSource Software Development, ed., Stephan Koch, IdeaPublishing, forthcoming.

• D.M. Nichols & M.B. Twidale, The Usability of OpenSource Software, First Monday, 8(1), January 2003.

36

Referencessee http://www.isr.uci.edu/research-open-source.html

• C. Jensen and W. Scacchi, Discovering, Modeling, and Re-enacting Open Source Software Processes: A Case Study,Working Paper, Institute for Software Research, March 2004.

• W. Scacchi, Free/Open Source Software Development Practicesin the Computer Game Community, IEEE Software, Special Issueon Open Source Software, 21(1), 59-67, January-February 2004.

• W. Scacchi, When is Free/Open Source Software DevelopmentFaster, Better, and Cheaper than Software Engineering? WorkingPaper, Institute for Software Research, UC Irvine, April 2003.

• W. Scacchi, Understanding the Requirements for DevelopingOpen Source Software, IEE Proceedings--Software, 149(1), 24-39, 2002.

• W. Scacchi, Redesigning Contracted Service Procurement forInternet-based Electronic Commerce: A Case Study, J.Information Technology and Management, 2(3), 313-334, 2001.

• This presentation can be found at:http://www.ics.uci.edu/~wscacchi/Presentations/OSS-Strategies/