Conceptual Framework for Agent-Conceptual Framework for Agent-Based Modeling and Simulation:Based Modeling and Simulation:

The Computer ExperimentThe Computer Experiment

Yongqin GaoYongqin Gao Vincent FreehVincent Freeh Greg Madey Greg MadeyCSE DepartmentCSE Department CS DepartmentCS Department CSE Department CSE DepartmentUniversity of Notre DameUniversity of Notre Dame NCSUNCSU University of Notre Dame University of Notre Dame

NAACSOS ConferenceNAACSOS Conference

Pittsburgh, PAPittsburgh, PAJune 25, 2003June 25, 2003

Supported in part by theSupported in part by the

National Science Foundation - Digital Society & Technology ProgramNational Science Foundation - Digital Society & Technology Program

The Computer ExperimentThe Computer Experiment

Agent-Based Simulation asAgent-Based Simulation asa Component of thea Component of theScientific MethodScientific Method

Modeling(Hypothesis)

Agent -BasedSimulation(Experiment)

Observation

Social NetworkModel of F/OSS

Grow ArtificialSourceForge

Analysis ofSourceForge

Data

OutlineOutline

•• Investigation: Free/Open Source Software (F/OSS)Investigation: Free/Open Source Software (F/OSS)•• Conceptual framework(s)Conceptual framework(s)•• Model descriptionModel description•• ER modelER model•• BA modelBA model•• BA model with constant fitnessBA model with constant fitness•• BA model with dynamic fitnessBA model with dynamic fitness•• SummarySummary

Open Source Software (OSS)Open Source Software (OSS)•• Free Free ……

–– to view sourceto view source–– to modifyto modify–– to shareto share–– of costof cost

•• ExamplesExamples–– ApacheApache–– PerlPerl–– GNUGNU–– LinuxLinux–– SendmailSendmail–– PythonPython–– KDEKDE–– GNOMEGNOME–– MozillaMozilla–– Thousands moreThousands more

LinuxGNU

Savannah

Free Open Source Software (F/OSS)Free Open Source Software (F/OSS)

•• DevelopmentDevelopment–– Mostly volunteerMostly volunteer–– Global teamsGlobal teams–– Virtual teamsVirtual teams–– Self-organized - often peer-based meritocracySelf-organized - often peer-based meritocracy–– Self-managed - but often a Self-managed - but often a ““charismaticcharismatic”” leader leader–– Often large numbers of developers, testers, support help, endOften large numbers of developers, testers, support help, end

user participationuser participation–– Rapid, frequent releasesRapid, frequent releases–– Mostly unpaidMostly unpaid

TypicalTypicalCharismaticCharismatic

Leaders?Leaders?

Linus TolvaldsLinux

Larry WallPerl

Richard StallmanGNU Manifesto

Eric RaymondCathedral and Bazaar

F/OSS: SignificanceF/OSS: Significance

•• Contradicts traditional wisdom:Contradicts traditional wisdom:–– Software engineeringSoftware engineering–– Coordination, large numbersCoordination, large numbers–– Motivation of developersMotivation of developers–– QualityQuality–– SecuritySecurity–– Business strategyBusiness strategy

•• Almost everything is doneAlmost everything is doneelectronically and available inelectronically and available indigital formdigital form

•• Opportunity for Social ScienceOpportunity for Social ScienceResearch -- large amounts of onlineResearch -- large amounts of onlinedata availabledata available

•• Research issues:Research issues:–– Understanding motivesUnderstanding motives–– Understanding processesUnderstanding processes–– Intellectual propertyIntellectual property–– Digital divideDigital divide–– Self-organizationSelf-organization–– Government policyGovernment policy–– Impact on innovationImpact on innovation–– EthicsEthics–– Economic modelsEconomic models–– Cultural issuesCultural issues–– International factorsInternational factors

SourceForgeSourceForge

• VA Software• Part of OSDN• Started 12/1999• Collaboration tools• 58,685 Projects• 80,000 Developers• 590,00 RegisteredUsers

SavannahSavannah• Uses SourceForgeSoftware• Free SoftwareFoundation•1,508 Projects•15,265 RegisteredUsers

F/OSS: Importance

Major Component of e-Technology Infrastructure with majorpresence in

e-Commercee-Sciencee-Governmente-Learning

Apache has over 65% market share of Internet Web serversLinux on over 7 million computersMost Internet e-mail runs on SendmailTens of thousands of quality productsPart of product offerings of companies like IBM, Apple

Apache in WebSphere, Linux on mainframe, FreeBSD in OSXCorporate employees participating on OSS projects

Free/Open Source SoftwareFree/Open Source Software

•• Seems to challenge traditional economic assumptionsSeems to challenge traditional economic assumptions•• Model for software engineeringModel for software engineering•• New business strategiesNew business strategies

–– Cooperation with competitorsCooperation with competitors–– Beyond trade associations, shared industry research, andBeyond trade associations, shared industry research, and

standards processes standards processes —— shared product development! shared product development!

•• Virtual, self-organizing and self-managing teamsVirtual, self-organizing and self-managing teams•• Social issues, e.g., digital divide, internationalSocial issues, e.g., digital divide, international

participationparticipation•• Government policy issues, e.g., US software industry,Government policy issues, e.g., US software industry,

impact on innovation, security, intellectual propertyimpact on innovation, security, intellectual property

Research ModelResearch Model

Parameter Values

Structural Features

Parameter Values

Cross Validation

Structural Features

Combined Data MiningParameter Values

Understanding the Social and Task

Dynamics that Predict Developer Behaviors

Social Network Analysis: Longitudinal

Study of Preferential Attachment and Dynamic

Attachment

Conceptual Explanatory Model of

OSS: Agent-Based Modeling and Simulation

Data Collection Data Collection —— Monthly Monthly

•• Web crawler (scripts)Web crawler (scripts)–– PythonPython–– PerlPerl–– AWKAWK–– SedSed

•• MonthlyMonthly•• Since Jan 2001Since Jan 2001•• ProjectIDProjectID•• DeveloperIDDeveloperID•• Almost 2 million recordsAlmost 2 million records•• Relational databaseRelational database

PROJ|DEVELOPER8001|dev3788001|dev89758001|dev99728002|dev276508005|dev313518006|dev125098007|dev193958007|dev46228007|dev356118008|dev8975

15850 dev[46]dev[83] 15850 dev[46]

dev[48]

15850 dev[46]dev[56]

15850 dev[46]dev[58]

6882 dev[58]dev[47]

6882 dev[47]dev[79]

6882 dev[47]dev[52]

6882 dev[47]dev[55]

7028 dev[46]dev[99]

7028 dev[46]dev[51]

7028 dev[46]dev[57] 7597 dev[46]

dev[45]

7597 dev[46]dev[72]

7597 dev[46]dev[55]

7597 dev[46]dev[58]

7597 dev[46]dev[61]

7597 dev[46]dev[64]7597 dev[46]

dev[67]

7597 dev[46]dev[70]

9859 dev[46]dev[49]9859 dev[46]

dev[53]

9859 dev[46]dev[54]

9859 dev[46]dev[59]

dev[46]

dev[83] dev[56]

dev[48]

dev[52]

dev[79]

dev[72]

dev[51]

dev[57]

dev[55]

dev[99]

dev[47]

dev[58]

dev[53]

dev[58]

dev[65]

dev[45]

dev[70]

dev[67]

dev[59]

dev[54]

dev[49]

dev[64]

dev[61]

Project 6882

Project 9859

Project 7597

Project 7028

Project 15850

F/OSS Developers - Social Network ComponentDevelopers are nodes / Projects are links

24 Developers5 Projects

2 Linchpin Developers1 Cluster

Models of the F/OSS Social NetworkModels of the F/OSS Social Network(Alternative Hypotheses)(Alternative Hypotheses)

•• General model featuresGeneral model features–– Agents are nodes on a graph (developers or projects)Agents are nodes on a graph (developers or projects)–– Behaviors: Create, join, abandon and idleBehaviors: Create, join, abandon and idle–– Edges are relationships (joint project participation)Edges are relationships (joint project participation)–– Growth of network: random or types of preferentialGrowth of network: random or types of preferential

attachment, formation of clustersattachment, formation of clusters–– FitnessFitness–– Network attributes: diameter, average degree, power law,Network attributes: diameter, average degree, power law,

clustering coefficientclustering coefficient•• Four specific modelsFour specific models

–– ER (random graph)ER (random graph)–– BA (scale free)BA (scale free)–– BA ( + constant fitness)BA ( + constant fitness)–– BA ( + dynamic fitness)BA ( + dynamic fitness)

ER model ER model –– degree distribution degree distribution

• Degreedistribution isbinomialdistribution whileit is power law inempirical data

• R2 = 0.9712 fordevelopernetwork

• R2 = 0.9815 forproject network

ER model - diameterER model - diameter

•• Average degree isAverage degree isdecreasing while it isdecreasing while it isincreasing in empiricalincreasing in empiricaldatadata

•• Diameter is increasingDiameter is increasingwhile it is decreasing inwhile it is decreasing inempirical dataempirical data

ER model ER model –– clustering coefficient clustering coefficient

•• Clustering coefficient isClustering coefficient isrelatively low around 0.4relatively low around 0.4while it is around 0.7 inwhile it is around 0.7 inempirical data.empirical data.

•• Clustering coefficient isClustering coefficient isdecreasing while it isdecreasing while it isincreasing in empiricalincreasing in empiricaldatadata

ER model ER model –– cluster distribution cluster distribution

•• Cluster distribution in ERCluster distribution in ERmodel also have power lawmodel also have power lawdistribution with Rdistribution with R22 as 0.6667 as 0.6667(0.9953 without the major(0.9953 without the majorcluster) while Rcluster) while R22 in empirical in empiricaldata is 0.7457 (0.9797data is 0.7457 (0.9797without the major cluster)without the major cluster)

•• The actual distribution isThe actual distribution isdifferent from empirical datadifferent from empirical data

•• The later models (BA andThe later models (BA andfurther models) have similarfurther models) have similarbehaviorsbehaviors

BA model BA model –– degree distribution degree distribution

•• Power laws in degreePower laws in degreedistribution, similar todistribution, similar toempirical data (+ forempirical data (+ forsimulated data and x forsimulated data and x forempirical data).empirical data).

•• For developer distribution:For developer distribution:simulated data has Rsimulated data has R22 as as0.9798 and empirical data has0.9798 and empirical data hasRR22 as 0.9712. as 0.9712.

•• For project distribution:For project distribution:simulated data has Rsimulated data has R22 as as0.6650 and empirical data has0.6650 and empirical data hasRR22 as 0.9815. as 0.9815.

••

BA model BA model –– diameter and CC diameter and CC

•• Small diameter and highSmall diameter and highclustering coefficient likeclustering coefficient likeempirical dataempirical data

•• Diameter and clusteringDiameter and clusteringcoefficient are bothcoefficient are bothdecreasing like empiricaldecreasing like empiricaldatadata

BA model with constant fitnessBA model with constant fitness

•• Power laws in degree distribution,Power laws in degree distribution,similar to empirical data (+ forsimilar to empirical data (+ forsimulated data and x for empiricalsimulated data and x for empiricaldata).data).

•• For developer distribution:For developer distribution:simulated data has R2 as 0.9742simulated data has R2 as 0.9742and empirical data has R2 asand empirical data has R2 as0.9712.0.9712.

•• For project distribution: simulatedFor project distribution: simulateddata has R2 as 0.7253 and empiricaldata has R2 as 0.7253 and empiricaldata has R2 as 0.9815.data has R2 as 0.9815.

•• Diameter and CC are similar toDiameter and CC are similar tosimple BA model.simple BA model.

BA model with dynamic fitnessBA model with dynamic fitness

•• Power laws in degreePower laws in degreedistribution, similar todistribution, similar toempirical data (+ forempirical data (+ forsimulated data and x forsimulated data and x forempirical data).empirical data).

•• For developer distribution:For developer distribution:simulated data has R2 assimulated data has R2 as0.9695 and empirical data has0.9695 and empirical data hasR2 as 0.9712.R2 as 0.9712.

•• For project distribution:For project distribution:simulated data has R2 assimulated data has R2 as0.8051 and empirical data has0.8051 and empirical data hasR2 as 0.9815.R2 as 0.9815.

Advantage of BA with dynamic fitnessAdvantage of BA with dynamic fitness

•• Intuition: Fitness should decreasing with time.Intuition: Fitness should decreasing with time.

•• Statistics: project has life cycle behavior whichStatistics: project has life cycle behavior whichcan not be replicated by BA model withcan not be replicated by BA model withconstant fitness but can be replicated by BAconstant fitness but can be replicated by BAmodel with dynamic fitnessmodel with dynamic fitness

Conceptual FrameworkConceptual FrameworkAgent-Based Modeling and SimulationAgent-Based Modeling and Simulationas Components of the Scientific Methodas Components of the Scientific Method

Observation

Hypothesis

Experiment

SummarySummary

•• We use ABM to model and simulate theWe use ABM to model and simulate theSourceForge collaboration network.SourceForge collaboration network.

•• Conceptual framework is proposed for agent-Conceptual framework is proposed for agent-based modeling and simulation.based modeling and simulation.

•• Case study of this framework: SourceForgeCase study of this framework: SourceForgestudy through ER, BA, BA with constant fitnessstudy through ER, BA, BA with constant fitnessand BA with dynamic fitness.and BA with dynamic fitness.

Thank youThank you