Embed Size (px)
Citation preview
Mining Projects from (Un)Structured Data
Project mining
Saimir Bala, WU (Vienna University of Economics and Business)
12 JUNE 2017
You the manager of a software development company
You are applying best practices, established project management guidelines and tools
But…
What is really going on in your software development project?
Why are deadlines not met? Why are the costs superior to the planned?
Why is does the your software product require more maintenance than what you thought?
PAGE 2
Motivation
Software processes are carried out in a project-oriented fashion
Goal is a release of a software product
Artifact-centric processes
Software development methodology (e.g., Scrum, Waterfall)
Artifacts are tracked by means of Version Control Systems
Should follow best practices (e.g. Principles of good modularization)
PAGE 3
About software projects
Q: How can we help the manager to gain transparency on the software project?
PAGE 4
Project-Oriented Business Processes
Project-Oriented Classic Processes
Plan (e.g. Gantt, PERT) Process Model (e.g. Petri Net, BPMN)
One time (fixed goal and resources)
Recursive, Cyclic
Single instance Many instances
Workpackages, Modules, Units Activities
Subworkpackages, Submodules Suprocesses
PAGE 5
Project-Oriented vs. Classic Business Processes
Project-Oriented Classic Processes
Plan (e.g. Gantt, PERT) Process Model (e.g. Petri Net, BPMN)
One time (fixed goal and resources)
Recursive, Cyclic
Single instance Many instances
Workpackages, Modules, Units Activities
Subworkpackages, Submodules Suprocesses
PAGE 6
Project-Oriented vs. Classic Business Processes
?
Software projects are supported by a variety of tools
Examples
Project management, Bug-tracking
Development
Dependency management
Testing
Continuous integration
Documentation
Version Control System
PAGE 7
Software Projects Data
PAGE 8
State of the Art: Activity Mining
Kindler et al. 2006
PAGE 9
State of the Art: Dotted Chart
Song and van der Aalst. 2007
PAGE 10
State of the Art: Evolution Storylines
Ogawa and Ma 2010
PAGE 11
State of the Art: Visual Software Analytics
Wettel and Lanza 2007
Identification of the problem
Objectives of the artifact
Artifact Development
DemonstrationEvaluationCommunication
PAGE 12
Methodology: DSR
Peffers 2007
Mining the Real Gantt Chart
PAGE 13
Mining the Gantt Chart of a Project
14
Example from the SHAPE project
“pm” work package
“example” work package
single events within files
Associated information:
1. User2. Type of change
3. Comment
Project structure
Activity inferred from single events1. Threshold based
2. Activated when a tree node in the
project structure is collapsed3. Decomposed when node is
expanded15
Indicators
Data from the VCS
Authors
Files
Type of change, Etc
Coverage, i.e. work-intensity
the ratio between active working periods (i.e., the time spans of activities) and the total work package duration
Expected active time between commits (tc)
average work speed (commit frequency) during active times
16
The following assumptions are made:
1. Meaningful file structure
Project participants organize the files in a representative (e.g., spatially separating documentation from testing into different folders).
2. Regular commits
Project participants systematically commit their changes in the VCS
3. Descriptive comments.
Project participants write descriptive comments that allow others members to understand the changes made to the software
PAGE 17
Assumptions
Resource Classification from Commit Messages
18
PAGE 19
Resource Classification from Commit Messages
Developer
PAGE 20
Resource Classification from Commit Messages: Developer vs. Tester
Tester
PAGE 21
Learning Decision Trees from Projects
Mining Hidden Work Dependencies
22
PAGE 23
Artifact Evolution as Time Series
Are they similar?
Correlation!
PAGE 25
Characterization of Projects wrt Dependencies
Bala, S., Cabanillas, C., Mendling, J., Rogge-Solti, A., Polleres, A.: Mining Project-Oriented Business Processes. In: BPM. pp. 425–440 (2015).
Agrawal, K., Aschauer, M., Thonhofer, T., Tomsich, N., Bala, S., Rogge-Solti, A.: Resource Classification from Version Control System Logs. In: EDOC Workshops (2016).
Bala, S., Havur, G., Sperl, S., Steyskal, S., Haselböck, A., Mendling, J., Polleres, A.: SHAPEworks: A BPMS Extension for Complex Process Management. In: BPM (Demos). pp. 50–55. (2016).
Bala, S., Revoredo, K., Mendling, J., Santoro, F.: Uncovering the Hidden Co-Evolution in the Work History of Software Projects. In: BPM. 2017 (conditionally accepted)
PAGE 26
Publications
Identification of the problem
Objectives of the artifact
Artifact Development
DemonstrationEvaluationCommunication
PAGE 27
Next steps
Peffers 2007
Identification of the problem
Objectives of the artifact
Artifact Development
DemonstrationEvaluationCommunication
PAGE 28
Next steps
Peffers 2007
Identification of the problem
Objectives of the artifact
Artifact Development
DemonstrationEvaluationCommunication
PAGE 29
Next steps
Peffers 2007
DEPARTMENT OF INFORMATION SYSTEMS AND OPERATIONSINSTITUTE FOR INFORMATION BUSINESS
Welthandelsplatz 1, D2/1.0261020 Vienna, Austria
M.SC. SAIMIR BALA
T +43-1-313 36-5304F +43-1-313 [email protected]
PAGE 32
Questions?
Backup slides
PAGE 33
PAGE 34
Data Model for SQL Querying VCS logs
Mining the Real Gantt Chart
PAGE 35
Mining the Gantt Chart of a Project
36
Example from the SHAPE project
“pm” work package
“example” work package
single events within files
Associated information:
1. User2. Type of change
3. Comment
Project structure
Activity inferred from single events1. Threshold based
2. Activated when a tree node in the
project structure is collapsed3. Decomposed when node is
expanded37
Indicators
Data from the VCS
Authors
Files
Type of change, Etc
Coverage, i.e. work-intensity
the ratio between active working periods (i.e., the time spans of activities) and the total work package duration
Expected active time between commits (tc)
average work speed (commit frequency) during active times
38
The following assumptions are made:
1. Meaningful file structure
Project participants organize the files in a representative (e.g., spatially separating documentation from testing into different folders).
2. Regular commits
Project participants systematically commit their changes in the VCS
3. Descriptive comments.
Project participants write descriptive comments that allow others members to understand the changes made to the software
PAGE 39
Assumptions
Some Real World Projects
Project Description Commits Users Files Duration tc coverage
Opendata bot Open Data AT Assistant: Data
Pioneers Create Camp project28 1 3507 16 0 100.00%
MiningVCS Gantt chart visualization of projects 84 1 111 61 1.9 87.00%
MSR paper Writing a conference paper 35 2 78 44 12.8 70.00%
Progit2 Pro Git 2nd Edition 1292 134 955 481 118.6 60.00%
GHDiscovery GitHub Activities Discovery
repository.11 1 97 29 6.6 53.00%
SHAPE Joint research project on railway
automation624 13 6470 1127 21.8 38.00%
papers from
siemens
Repository form Siemens to keep
track of paper writing processes649 5 1791 1853 26.4 23.00%
Facebook-ads-
java-sdk
Java SDK for Facebook Ads APIs 38 8 428 324 18.2 22.00%
Biglist-of-
naughty-strings
Strings which have a high probability
of causing issues when used as user-
input data.
202 60 15 530 53.3 10.00%
We set the aggregation threshold to 7 days (i.e. two events belong to the same activity only if their temporal distance is one week or less)
40
Open-Data Helper Bot
Open Data AT Assistant: Data Pioneers Create Camp project
Helps search for an open dataset
28 commits, 1 user 3507 files, 16 days
0 tc, 100% coverage
41
Mining VCS Software
This software project
84 commits, 1 user, 111 files, 62 days
tc 1.9 hours, coverage 87%
42
MSR Paper
Preparation of a conference paper
35 commits, 2 users, 78 files, 44 days
12.8 tc, 70% coverage
43
Book Writing Project
Progit book 2nd edition
1292 commits, 134 users, 955 files, 481 days
tc 118.6 hours, coverage 60%
44
Students Project: Disovering Github Activities
Student class project 1 user, 72 files, duration
29 days tc 8.8 hours, coverage
59%45
SHAPE Project
Joint research project on railway automation
6470 files, 13 users, duration 1127 days
tc 21.8 hours, coverage = 38%
46
Writing Papers Project (from Industry)
Repository for papers writing process taken from SHAPE project
649 commits, 5 users, 1791 files, 1853 days duration tc 26.4 hours, coverage 23%
47
Facebook ads java sdk
Java development kit for Facebook ads 38 commits, 8 users, 428 files, 324 days 18.2 tc, 22% coverage
48
Big List of Naughty Strings
An evolving list of strings which have a high probability of causing issues when used as user-input data.
202 commits, 15 files, 51 users, 531 days tc 53.3 hours, coverage 10%
No activities found in the subdirectories, i.e. no continuous work for in the same subdirectory within the given aggregation threshold
49
Uncovering the Hidden Co-Evolution in the Work History of Software Projects
PAGE 50
PAGE 51
Requirements
How can we use data generated from the software project to help gaining transparency on the status and work history?
R1 (Extract the work history)
•Discover the process of how artifacts evolve in the project as a labeled set of steps
R2 (Uncover Work-Related Dependencies)
•Identify that parts of the work are connected to other parts co-evolution of two artifacts?
R3 (Measure Dependencies)
•How strongly depend two artifacts on one another?
PAGE 52
State of the Art
MSR
• Mostly solving R2 (Uncover work-related dependencies and R3 (Measure dependencies)
• Zaidman et al. 2008, Zimmerman et al. 2008, D‘Ambros et al. 2009, Lindeberg et al. 2016
Process Mining
• Mostly addressing R1 (Extract the work history)
• Kindler et al. 2006, Goncalves et al. 2011, Poncin et al. 2011, Bala et al. 2015
Visualization
• No approach addressing R1, R2, and R3 simultaneously
• Voinea and Telea 2006, Ripley et al. 2007, Greene and Fischer 2015
PAGE 53
Approach
How to capture events?
How to obtain the work history from the events?
What are important informations we need to consider in order to identify dependencies?
How to analyze the data?
How to measure work-dependency?
PAGE 54
Challenges
The following assumptions are made:
1. Meaningful file structure
Project participants organize the files in a representative (e.g., spatially separating documentation from testing into different folders).
2. Regular commits
Project participants systematically commit their changes in the VCS
3. Descriptive comments.
Project participants write descriptive comments that allow others members to understand the changes made to the software
PAGE 55
Assumptions
PAGE 56
Concepts
Artifact evolution
• Changes made to an artifact during its lifetime, meaured in Lines of Code
Dependency• High similarity in the evolution of two
software artifacts
PAGE 57
Metrics
Degree of Co-
Evolution
•Strength of the connection. A value in the interval [0,1], where 1 is the highest degree of co-evolution
File Distance
•Distance between two files in the file tree. Equal to the length of the path traversing the least common ancestor.
Generate events from software repository data into a log file
Preprocess log data
Compute artifact evolution
Compute Dependencies
Compute MetricsSelect set of
highly dependent & distant files
Potential wok-dependencies
PAGE 58
Method
PAGE 59
Computing Dependencies
Are they similar?
Correlation!
PAGE 60
Results
PAGE 61
Co-Evolution versus Distance
PAGE 62
Characterization of Projects wrt Dependencies
PAGE 63
Zipf law on real projects: 100%
Camunda Operationcode Caret
PAGE 64
Zipf law on real projects: top 80%
Camunda Operationcode Caret
PAGE 65
Zipf law on real projects: top 50%
Camunda Operationcode Caret
PAGE 66
Stories
Are they similar?
Mining project-oriented business process is difficult
Provide hints for the project manager
Work dependencies not easy to be seen without analysing the work history
Future work:
Improve method for comparting time series
Semantic analysis of process labels
PAGE 67
Conlusion
