®

IBM Software Group

© 2005 IBM Corporation

The Open Internet – Open Source, Open Standards and the Effects on Collaborative Software Development

September 26, 2005, High Performance Transaction Systems Workshop (HPTS)

Dr. Danny SabbahGeneral Manager, Rational Software, IBM

IBM Software Group

2

Software Development Needs to radically change

Why??? Increasing abstraction and forced integration = forced complexity

The Ugly reality of developing distributed, internet based solutions

The pluses and minuses of Moore’s Law and Cheap and pervasive interconnection

Mass Adoption (individual computing) meets Mass Interconnection (internetworking)

How??? Rethinking modularity and granularity of software

Increasing focus on ‘decomposition’ for re-composition

Leveraging network effects introduced by Open Source development methodology

Open Source development is not just about ‘Open’ Source

IBM Software Group

3

Destabilizing forces converge to significantly intensify global competition

-5

5

15

25

35

45

55

65

75

1928 '38 '48 '58 '68 '78 '88 '98 '08 '18

Year

Imp

lied

Lif

etim

e in

S&

P 5

00

Yea

rs

Average Lifetime of S&P 500 Companies

Source: Creative Destruction, Richard Foster

Technology innovation and long-term public policy shifts are consistently destabilizing the marketplace (and it’s accelerating)

Technology systematically reduces interaction costs and extends global reach

Constant worldwide public policy shifts alter regulatory climate and promote or discourage competition

In a world where speed through IT automation is now largely assumed

IBM Software Group

4

How Moore’s Law and Unlimited Bandwidth play: As environmental and economic factors (globalization, risk

management, innovation), force CONSTANT, accelerating fundamental business change (processes,deconstruction/refocus, partnerships)

Chaos results from conflicting goals from multiple generations of ‘captured intelligence’ in the form of code/business rules mixed in with new generations of assumptions (Mainframe to C/S to Peer distributed – and variants)

Technology adds to disruptive forces unless properly managed and accounted for. NO ONE starts from SCRATCH. Software EVOLVES.

The unintended side effects of Moore’s law can become a curse to innovation if an organization’s Information technology isn’t allowed to advance at the same rate.

IBM Software Group

5

E01-EDI

Data Warehouse(Interfaces to and from theData Warehouse are not

displayed on this diagram)

G02 - GeneralLedger

A05 - AP

S01 - SalesCorrections

I01 POReceiving

I03 Return toVendor

I06 WarehouseManagement

Mainframe

PC/NT apps Unix apps

3rd Party Interface

S06 - Credit App

P15 EES EmployeeChange Notice

OTHER APPS - PCAP - Collections/Credit

TM - Credit Card DB

ACCTS REC APPS - PC990CORBad Debt

Beneficial FeesBeneficial Reconcile

JEAXFJEBFAJEBKAJEDVAJESOAJEVSAJEVSFNSF

TeleCredit Fees

INVENTORY CONTROL APPS - PCCode Alarm

Debit ReceivingsDevo Sales

Display InventoryIn HomeJunkouts

Merchandise WithdrawalPromo CreditsRTV Accrual

ShrinkAP Research - Inv CntrlAP Research-Addl Rpts

Book to Perpetual InventoryClose Out Reporting

Computer Intelligence DataCount Corrections

Cross Ref for VCB DnldsDamage Write OffDebit Receivings

DFI Vendor DatabaseDisplay Inventory ReconcileDisplay Inventory Reporting

INVENTORY CONTROL APPS - PCDPI/CPI

IC BatchingInventory Adj/Count CorrectInventory Control Reports

Inventory LevelsInventory Roll

Merchandise WithdrawalOpen ReceivingsPI Count Results

PI Time Results from InvPrice Protection

Sales Flash ReportingShrink Reporting

SKU Gross MarginSKU Shrink Level Detail

USMVCB Downloads

Journal Entry Tool Kit

Scorecard - HR

L02-ResourceScheduling(Campbell)

P09 - P17Cyborg

M02 - Millennium

M03 - Millennium 3.0

Banks - ACH and Pos toPay

Cobra

B01 - StockStatus

S03-Polling

P14 On-line NewHire Entry

CTS

Plan Administrators(401K, PCS, Life,

Unicare, SolomonSmith Barney)

D01 Post LoadBilling

I04 HomeDeliveries

I02 -Transfers

Arthur Planning

I07 PurchaseOrder

I12 EntertainmentSoftware

I05Inventory Info

E13E3 Interface

S04 - Sales Posting

V01-Price ManagementSystem

I10 Cycle PhysicalInventory

I55 SKUInformation

K02Customer Repair

TrackingI35 Early Warning

System

B02 MerchandiseAnalysis

I13- AutoReplenishment

U18 - CTO

Intercept

I09 Cycle Counts

E02-EmployeePurchase

Texlon 3.5

ACH

Stock Options

I17 Customer PerceivedIn-Stock

U16-Texlon

SiteSeer

C02 - CapitalProjects

F06 - FixedAssets

US Bank ReconFile

Star Repair

EDICoordinator

Mesa Data

NEW SoundscanNPD Group

AIG Warranty Guard

Resumix

Optika

Store BudgetReporting

P16 - Tally Sheet

Cash Receipts/Credit

S05 - HouseCharges

Ad Expense

L01-PromoAnalysis

V02-PriceMarketingSupport

BMP - Busperformance Mngt

StoreScorecard

I11 PriceTesting

Valley Media

P09Bonus/HR

I15 Hand ScanApps

Roadshow

POS

S08 - VertexSalesTax

A04 - CustRefund Chks

Equifax

ICMS Credit

CellularRollover

S09 - DigitalSatelliteSystem

NPD,SoundScan

Sterling VANMailbox (Value)

I18SKU Rep

X92-X96Host to AS400

Communication

S02 -Layaways

Washington,RGIS,

Ntl Bus Systems

V04-SignSystem

I14 Count CorrectionsNARM

P01-EmployeeMasterfile

I06 - CustomerOrder

FrickCo

UAR - Universal AccountReconciliation

DepositoryBanks

S07 - CellPhones

S11 - ISPTracking

AAS

Fringe PO

Cash Over/Short

L60 MDFCoop

SKU SelectionTool

SKUPerformance

SupplierCompliance

1

I35 - CEI

ASIS

Misc Accounting/Finance Apps - PC/NTCOBA (Corp office Budget Assistant)

PCBS(Profit Center Budget System)Merchandising Budget

AIMSMerch Mngr Approval

Batch ForcastingAd Measurement

AIMS Admin

AIMSReportingAd

Launcher

V03- MktReactions

SpecSource

CTO2.Bestbuy.com

RebateTransfer

SignSystem

CopyWriter'sWorkspace

ELTPowerSuite

StoreMonitor

AIS Calendar

Stores & Mrkts

Due Dates

Smart Plus

InsertionsOrders

BudgetAnalysis Tool

Print CostingInvoice App

AIS Reports

BroadcastFilter

Smart PlusLauncher

GeneralMaintenance

Printer PO

PrinterMaintenance

VendorMaintenance

Vendor Setup

Connect 3

Connect 3Reports

Connect 3PDF Transfe

Spec SourceSKU Tracking

S20-SalesPolling

Prodigy

PSP

In-HomeRepair

WarrantyBillingSystem

Process Servers(Imaging)

Prepared by Michelle Mills

IT Software Engineering Realities

IBM Software Group

6

Unraveling the Chaotic Mess of conflicting abstraction and undifferentiated monoliths

I see 2 major dimensions: Modularity for the Internet age: Very Loosely coupled architectures and

systems focusing on largely ‘declarative’ I/F’s based on OPEN standards (necessarily stemming from the Internet) -- Decomposable Systems

Standards, Standards, Standards…. For interop NOT portability

Commonly termed “Service Oriented Architectures” based on XML and Web Services standards

Only made possible by the ‘positives’ in Moore’s law (i.e. no one in their right mind would have suggested this 10 years ago)

‘The discipline of Programming in the Very Large’ through internet community growth and governance (e.g. Open Source Communities governed by meritocracy)

Meritocracies force minimal design and controlled extensibility.. A form of collaborative innovation

Exposure early and often forces accountability and better quality

Establishing pervasive, default distribution for STANDARDS

IBM Software Group

7

‘Open’ Computing: Staying Open for BusinessGoals of openness:•Flexibility•Resilience•Avoid limiting network effects •Maximize freedom of action and innovation

Open standards:•Promoting interoperability by using openpublished specifications for APIs, protocols and data and file formats

Open architecture:•Building loosely coupled, flexible, reconfigurable solutions

Open source software:•Leverages community development and collaborative innovationcollaborative innovation•Optimizes network effects and ‘minimalism’ in design

IBM Software Group

8

Controlling/minimizing Complexity

Large, monolithic systems and packaged apps are the result of uncontrolled, undisciplined experimentation driven by quickly dissolving software business models

Persistent software systems (OS’s, Middleware, P. Apps) must practice controlled evolution with simpler, decomposable designs.

Software Components must be driven to ‘just enough’

This is Hard! Simple does not mean ‘simplistic’:

“Designs should be as simple as possible, but no simpler”

This applies to all ‘aspects’ of systems designs: internal and external

IBM Software Group

9

The ‘Open’ Proposition: Industry business models must change

Change From “Control”Change From “Control” To “Sustained Value Add”To “Sustained Value Add”

Own standard

Own customer relationship

Control price

Control pace of development

Benefit from network effects

Use economy of scale

Increase availability of skills

Increase Speed of Innovation

Letting Go of Control

IBM Software Group

10

Emergence of Open Source Software communities are being fueled by:

Internet based cost efficiencies and near universal connectivity Standards Efficient collaboration/community tools (email, Wikis, newsgroups,

Collaborative Development platforms) Social aspects Emergence of “good enough” programming models (Occam’s

razor at work) Perception of innovation opportunities Alignment of Interests (commercial and strategic): Development

expense subsidy and incumbency disruption Growing commercial acceptance

IBM Software Group

11

Community TypesThere are two major open source community archetypes, though most projects

have some characteristics of both:

Voluntary model Apache, Linux projects. 46% do not earn any money for their participation, 16% paid to develop; 18% paid

to administer; 12% paid to support it; 26% indirect financial comp Projects such as Apache and Linux that started out as pure voluntary model are

trending towards a greater commercial aligned interests model and now have the majority of their code contributed by professional commercial developers

Vendor-initiated model Eclipse, Open Office, Ingres, for example Often used for establishing or diverting ecosystem vitality Initially dominated by donor company May mature to broad based aligned interest model May have direct or indirect business model plays Vendor initiated projects may take on greater voluntary model aspects as they

mature

IBM Software Group

12

IBM contributes to 150+ OSS projects

More than 1000 developers involved in OSS projects IBM leads 80+ OSS projects

1999 - 2001 IBM forms Linux

Technology Center – contributions to serviceability, performance

Leads Apache XML projects Xalan Xerces, SOAP

Forms Open Source Steering Committee

Creates OSI-approved IBM & Common Public Licenses

Participation in Mozilla

Participated in founding of Eclipse

2002 Linux contributions

to scalability (8-way+), reliability (stress testing, defect mgmt, doc)

Leads Apache Web Services projects WSIF and WSIL

Leads Eclipse projects GEF (editing), EMF (modeling), XSD (XML Schema)

IBM contributes eServer support for Globus Toolkit 2x

2003 IBM and SuSE

achieve EAL2+ Common Criteria security cert

Leads Apache projects Pluto (Portlet API) and WSRP4J (Remote Portal)

Leads Eclipse projects Hyades (testing), AspectJ, Equinox rich client

Globus Toolkit 3 contributions for OGSA, OGSI

2004 IBM and Novell/SuSE achieve

EAL3+ and Common Operating Environment compliance

Linux additional RAS

Incubates Apache project Derby (Cloudscape Java database)

Dialog components to Apache Jakarta taglibs

Eclipse becomes independent org – IBM contributes UML2, Web Tools, Voice Tools and eRCP

Globus Toolkit 4 to be WS-I compliant

2005 Contributions to Xen hypervisor, Linux accessibility

Contributions to Apache WSDL4J 2.0 (Woden), Web Services Security

• Database extensions to PHP

• Redeploy 30+ developerWorks projects on SourceForge.net

• IBM pledges 500 US patents to OSS

• Geronimo commitment; purchase of Gluecode

IBM Open Source History/Involvement

IBM Software Group

13

An Example: EclipseEclipse continues gaining popularity and a broad community

Eclipse formed as an independent organization in February 2004, establishing scalable governance model for Open Source projects and commercial participation

There have been over 18 million download requests since 4Q2001 that we can track (many more that we don’t)

By attracting more subprojects and third party contributors, the Eclipse community is establishing a de facto standard for an Integrated Development Environment

Founding strategic developers and strategic consumers are Ericsson, HP, IBM, Intel, MontaVista Software, QNX, SAP, and Serena Software

Over 30 companies are committed to Eclipse via IBM WebSphere Studio WorkBench

There are over 40 Eclipse add-in providers, including Borland, Novell, Oracle, Red Hat, VA Software, webMethods, and Wind River

More than 175 tools vendors are delivering tools for the Eclipse platform

IBM Software Group

14

Lessons Learned: Not all Open, Collaborative projects Succeed

Critical Factors for a successful Commercial Open Source CommunityBusiness perspective

Must provide “perceived value" to end customersActive sponsorship and marketing by 1 or more significant sponsorsSolid overall vendor support Critical skills available and actively engagedLong-term plans for improvements - plans, design,

development, test, documentation; rigorous IP policies

Developer perspectivePassionate interest in developing and enhancing code

Significant overlap between set of users and developers - common wavelength

Personal rewards - able to build status and recognition in developer community

Diverse, interactive community

Strong overall project/code leadership (e.g. Linus Torvalds -Linux)

IBM Software Group

15

Licensing: Can Open Source Software co-exist with Commercial Software? YES

Most (not all) open source software licenses allow combination and distribution of open source software and Commercial source code under a commercial license.

Some commonly encountered open source software licenses (BSD, MIT, X11, Apache) don't require modifications to original open source software to be published upon redistribution.

GPL allows commercial applications to be built on top of Linux to remain commercial Application can be licensed under commercial license of choice. No need to disclose source code of such applications.

LGPL Libraries can be dynamically linked to arbitrary commercial code No requirement to release commercial code under LGPL.

Decision to use open source software is just another business decision with risk/reward tradeoff

License terms need to be understood before beginning to work with open source software .

IBM Software Group

16

Internal Community Source (Open Source) Model

An open source-style development environment where consumers can contribute code to shared components

Producers get Collaboration

Broader testing and tuning

More folks using their code

Consumers get Fast access to capabilities

Ability to invest to tune to a unique specification

Broader testing and tuning

IBM Software Group

17

• Provides a collaborative environment accessed via a Web-based portal for shared, distributed development and testing; 225+ projects to date engaging 3,000 developers

• Controlled “white box” approach for development of component software• Explore shared development of components intended for reuse• Seeded by respective development organizations

Benefits Encouraging reuse over reinvention Improving information flow between teams

(availability of source materials, decisions and discussions)

Leveraging broader IBM community skills (technical and non-technical communities)

Improving quality through peer reviews and user feedback (defects and forums)

Positively impacting our ability to deliver more function on shorter schedule (collaboration and contribution)

Most valuable assets get the most attention (based on reuse)

Facilitate development

Community Source: Open Source Within IBM

InternalDevelopmentCommunity

Key Features: Access Control Product builds, fixes and

test drivers Discussion Forums Reference information (API

specs, programming documentation, education, demo, etc.)

Defect Reporting Feature Requests Code Storage and Version

Control

IBM Software Group

18

Community Source Infrastructure: How?

Register the project, meeting established criteria

Set up mailing lists and forums

Establish Project Leads & Committers

Publish materials, which may include: 1. Source (logic, test cases, binaries)2. Documentation, specs3. News, bulletins4. Education materials5. Patches, fixes

Build an interested community

Process requirements and contributions

IBM Software Group

19

Community Source Infrastructure: Who?

Contributors: Developers who submit code changes through

committers Provide additional testing Provide additional documentation Submit patches and features Submit requirements Help fulfill consumer requirements

Committers: Developers with write access to the repositories Often the Project Leaders Write/update source materials Control code contributions from contributors

IBM Software Group

20

Community Source Infrastructure: Who else?

Project Administrators: Individuals responsible for enabling and

regulating the smooth operation of the project

Operations and access control

Sets up mailing lists and forums

Publishes news and announcements

Users: Reviewers of source materials and documentation

May submit defects or feature requests

May be non-technical

May participate in discussion forums

IBM Software Group

21

Community Source Background IBM Internal Open Source Bazaar (IIOSB) was established January 2000

Community Source became a subset of the IIOSB around August 2002 Community Source has been referred to as the “IBM Product License zone” in IIOSB

Community Source was moved out of IIOSB infrastructure to its own dedicated server at the end of June 2005 Today based on open source, collaborative development infrastructure

Alphaworks and DeveloperWorks offer technology preview and an IBM open source infrastructure available for outside participation since 1999

IBM Product License zone

Projects – Community

Source

IBM Internal Open Source Bazaar

(IIOSB)

SourceForge Enterprise

GForge

Community SourceIBM Product

Licenseand SWG projects

IIOSBNon-IBM

confidentialprojects

GForge

2005

IBM Software Group

22

Executive Summary: Statistics & Measurements Important Community Source statistics as of August 15, 2005:

Active project count on Community Source: 224

07/15/05-08/15/05: 33 new project registrations approved

Active member count on Community Source: 3,014

Most active projects (> 50% activity level*): 17 projects

*Note: Activity level refers to the site-wide project usage of the key Community Source features (tracker, CVS, patch, file publisher etc.)

0

100

200

300

400

500

600

700

800

Functions

Downloads

Bugs

Support

Patches

Tracker

Tasks

Sit

e-w

ide

Fu

nc

tio

n U

sag

e

Measuring Usage & Leverage

0

50

100

150

200

250

May-05 Jun-05 Jul-05 Aug-05

Months

Measuring Community Growth

Ne

w R

eg

istr

ati

on

IBM Software Group

23

Project Pattern Usage1. Community Growth2. Information Sharing3. Source Code Development4. Production Environment5. Service & Support Mode

0

10

20

30

40

50

60

70

80

90

100

Functions

Mailing ListsDiscussion ForumsFile PublisherCode Library SystemDefect Tracking SystemPatch Tracking SystemFeature Request TrackingTech Support Tracking

% P

roje

cts

Usi

ng F

unct

ion

1 ….. 2 ….. 3 ..… 4 ..… 5

IBM Software Group

24

Community Source Example: Constellation Project Created to address the need to componentize the IBM Software Development Platform

Toolset

Scope For Caspian focus is (Rational Application Developer, WebSphere Integration Developer,

WebSphere Business Integration Modeler, Rational Software Architect) Cross product usage pain point for Atlantic High degree of overlap and lack of reuse

Goals: Create an internal open source environment (modeled after eclipse)

Governance Rules of Engagement

Define a set of common tools and processes

• Status:• Established cross brand Project Management Committee

• Created component inventory working closely with WebSphere Integration Developer• Set of component classifications/layers

• Sizing component recommendations for Caspian plan

IBM Software Group

25

Building the Community… SWG Developers Portal

Find resources

Learn strategy

“How to” integrate crossbrand

Share components/plans

Developers Developers IntranetIntranet Collaborate

Recommend resources

Locate experts

Share content with other portals

Developers Developers PortalPortal

Customized info by role

Developments tools integrated with Web

Integrated with individual teamrooms/intranets

Developers Developers WorkplaceWorkplace

IBM Software Group

26

EclipseClient

ProjectTeamServer

Team Bench awareness, team places UI, collaboration

Wor

kIte

ms

Bui

ld

Req

uire

men

ts

Rep

ortin

g/H

ealth

File

syst

em/V

CM

Tes

t

RA

S

VerticalServerPlug-ins

VerticalClientPlug-ins

Pla

nni

ng

Gui

dan

ce

…

Wor

kIte

ms

Bui

ld

Req

uire

men

ts

Rep

ortin

g/H

ealth

File

syst

em/V

CM

Tes

t

RA

S

Pla

nni

ng

Gui

dan

ce

…

Project Client API/Extensibility

Workflow Search Collaboration

Web/Portal

Relational data

Project home

Proj. Health

Communication: wiki, blog, etc

Web Services

Admin

VerticalWeb UI’s

….

Tomcat/Geronimo->WAS

ND->WAS XD

Cloudscape->DB2/

Oracle/…

Notification Query Persistence

Login/ACL

Platform API/Extensibility

Localdata

IBM Software Group

27

Platform Goals and Services Network effects/pervasive integration – Global development

Simple but scalable deployment and management

Significantly Improved Communication and collaboration (human and agent)

Basic interoperability with selected external solutions to ease evolution and integration

Loosely coupled workflow with fine grained security – peer to peer distribution model

Fine grained, configurable components through plug-in extensibility architecture

Out-of-the-box schema, metadata, metrics, process, views, roles, security model

IBM Software Group

28

Summary

Business models must adapt to survive in today’s business environment. Openness fosters adaptability and flexibility.

Software Development must promote greater ‘experimental’ discipline, incorporate finer modularity and leverages community based feedback/innovation

Letting go of control and loosening the reins on the development environment leads to greater innovation and business value. Programming in the large must become programming in the VERY large over internet protocols and standards

Fostering community source projects can speed development and ultimately, result in greater advances for the organizations. Even for proprietary software.

Commercial, open source and hybrid software models all deliver distinct value to the marketplace. They are leading to the emergence of new software business models