Upload
ishwinder-brar
View
409
Download
1
Tags:
Embed Size (px)
Citation preview
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 2
Team HandSimDroid 3
Development/ Support Manager
Team Lead Planning Manager
Process/Quality Manager
Mentors
Team HandSimDroid 4
Bosch RTC
UC Berkeley
• Elizabeth Latronico • Charles Shelton
• Christopher Brooks • Edward Lee
Bosch Research & Technology Center (Client)
Bosch uses an open-source tool called Ptolemy for modeling, simulation, and design of concurrent, real-time, embedded systems.
Android application that can run simulations of Ptolemy models on handheld devices.
Team HandSimDroid 5
Team HandSimDroid 6
• Show simulations running on the handheld
• Enable UI customization by model and per user
• Create demos that showcase usefulness of functionality to engineers
Act as a proof of concept for ASCET tool ◦ Inspire innovation at Bosch
Freely extend open source software
Improve operations & reduce cost of calibration ◦ Running simulation on the handheld on the go
◦ Customize UI for different purposes & users
Team HandSimDroid 7
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 8
Team HandSimDroid 9 http://bitURL.net/handsimdroid
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 10
Show simulations of three Ptolemy models on android.
Enable UI customization by model and per user.
Strict adherence to the coding, styling, and commenting standards set forth by the Ptolemy Project
Complete all must-have requirements
Adhere to (and adapt) the architecture
Team HandSimDroid 11
Priority Completed Total
Must Have 21 21
Good To Have 11 16
Nicety 1 5
Team HandSimDroid 12
What? Why?
Define integration points Difficult to tell when components could be integrated together
Update workbooks every Tues/Thurs/Sat
Weekly updates were no longer sufficient with 192 hr weeks
Provide weekly engineer reports Communication breakdown with task status
Calculate correlations between data to better estimates
Universal fudge factor of 30% was no longer accurate because of varied types of tasks
Team member who closes most bugs get free lunch
New features took precedence over addressing defects
Team member who reports most bugs gets free ice cream
Wanted to ensure that corner-cases were addressed
Team HandSimDroid 13
Hours get lost in the context switching, during breaks and incorrect tracking ◦ Measure the gap and allocate it as buffer
Group your tasks into sprints for easier tracking ◦ Helps in keeping the big picture in mind without being
bogged down with details
Improve and tailor your processes continuously, use postmortems to add PIPs to the process.
Team HandSimDroid 14
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 15
Team HandSimDroid 16
Release 1 Release 2 Release 3 Release 4
2011 Summer
May June July Aug.
Cycle 5 Cycle 6 Cycle 7 Cycle 8
ptdroid
Tool Setup
Homer
ptserver
Documentation
Bug fixing
0
500
1000
1500
2000
2500
1/19/2011
2/2/2011
2/16/2011
3/2/2011
3/16/2011
3/30/2011
4/13/2011
4/27/2011
5/11/2011
5/25/2011
6/8/2011
6/22/2011
7/6/2011
7/20/2011
8/3/2011
Actual Hours
Earned Value
Planned Value
Team HandSimDroid 17
Continuous problems: - Logging time (4 PIPS) and TSP tool - Only task hours, no context
switching - Some tasks not logged
PIP 8 PIP 15
PIP 17
PIP 19
Snapshot: 8/3
813.81
47%
565.62
33%
118.13
7%
3%
3% 2%
1% 1% 1% 1% 1% 0% CODE
MGMT
DOC
PLAN
IT
DLD
CODEINSP
PM
REQ
HLDINSP
DLDINSP
REQINSP
Was 42% during Spring. Less meetings overall as cycles became longer.
Team HandSimDroid 18
y = 1.26x - 0.21
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0
Actu
al ti
me (hours
)
Estimated time(hours)
Overestimated Tasks
(Visualization interface)
Underestimated Tasks
(UI tasks, Homer infrastructure)
Team HandSimDroid 19 Correlation: 84.51%
Challenges Possible solutions
Continuous tracking of time data was lacking and imprecise. Continuous effort is required.
• Tools help, but don’t really solve the problem currently. Context-aware applications could be a big help for developers.
• Team members being more disciplined. This didn’t work quite well for us.
Task volatility and tracking, especially since the TSP tool is not collaborative.
• Standups helped coordinate and mark tasks. • TSP workbooks are hard to be updated during
these meetings. Centralized customized workbooks or other management tool (e.g.: Jira)
Task boundaries and exit criteria were not defined well, due to unknowns or uncertainties.
• Spend more time defining the tasks. • Facilitate more frequent communication between
the members.
Team HandSimDroid 20
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 21
Risk Mitigation
Team had trouble in integrating modules; schedule and quality might be affected
• Design Reviews focusing on integration points
• Continuous Integration
Team is becoming more relaxed as functionality is implemented; Schedule might be affected - not enough time to focus on quality
• Baseline modules • Allocate last cycle for quality
activities • Friendly competition on bug
bashing
Team developed some code without following Ptolemy style guidelines; Berkeley team might become alienated; project does not have impact
• Checkstyle & continuous integration for publicly tracking style issues
Team HandSimDroid 22
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 23
0
5
10
15
20
25
30
35
Ptdroid Ptserver Homer General
Normal
Minor
Major
Enhancement
Critical
Blocker
Team HandSimDroid 24
Injection Phase Detection Phase
35
56%
27
44%
Design Code
37
60%
20
32%
4
6% 1
2%
Test Code Review Design Review Code
Team HandSimDroid 25
34.12
4%
860.11
96%
Time Spent
Review Code/Test
Team HandSimDroid 26
24
39%
38
61%
Bugs Found
Review Code/Test
Name
ptserver.actor 51% 42/82 67% 115/172
ptserver.communication 80% 60/75 87% 291/336
ptserver.control 42% 45/108 69% 231/335
ptserver.data 49% 35/72 73% 134/183
ptserver.data.handler 97% 58/60 95% 186/195
ptserver.util 43% 73/169 46% 149/325
Conditionals Lines
Team HandSimDroid 27
• Total LOC: 4235
• Total Comments: 4279
• Defects: 33 ptserver
• Total LOC: 5810
• Total Comments: 5591
• Defects: 23 ptdroid
• Total LOC: 3624
• Total Comments: 2888
• Defects: 7 homer
Team HandSimDroid 28
Defining a fault model would have resolved several design defects (state & concurrency).
Needed to realign development goals with stakeholder goals when working on Homer (functionality over quality).
Continuous integration was extremely helpful in getting a quality snapshot at any point.
Team HandSimDroid 29
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 30
Team HandSimDroid 31
1
2
3 4
QA Priority Difficulty Scenario
Performance High High Latency between capturing sensor data from a source and processing it on server is no more than 2 seconds per source Latency between completion of the data processing on server and piping it into a sink is no more than 2 seconds per sink
Extensibility High High A new graphical actor or source sensor is incorporated in UI designer and operational in the Android application within 2 person/weeks
Reliability High High In case of network issues, simulation errors or runtime exceptions, system must display error message, reset to default state and log the error and gracefully end simulation on the client and server.
Usability Medium Medium The UI of a model looks conceptually the same as it was shown in the desktop preview.
32 Team HandSimDroid
Dynamic Perspective
Proxy ModelInfrastructure
Legend
Object
Android Handheld
PtolemyServer
Thread
MQTT Broker
A B
A sends attributechange token to B
A B
A sends aremotetoken to B
A B
A sends an MQTT message to B
A B
A sends a token to Busing Ptolemy filter connector
Process
Ptolemy SimulationEngine
Multiplicity
Proxy Sinks
ActorsProxy
Sources
Blocking Queues
Token Listener
Token Publisher
Token Batch
Monitoring System
A B
A blocks B's thread
Ptolemy Server Hessian Servlet
A B
A sends synchronous command to B (http)
Proxy Model Infrastructure
SinksSources
Blocking Queues
Token Listener
Token BatchMonitoring
System
Proxy Sinks
Proxy Sources
Proxy Attributes
Attribute Change Listener
Attribute Change Listener
Token Publisher
New Component
Existing Component
Modified Component
Team HandSimDroid 33
Sources & Sinks must run on handheld
Reliability: Ping / Echo
Performance: Heavy Processing
on Server
Performance: Thread throttling
1 2
3
4
5
6
7
8
Mirrored Design
Architecture was largely detailed enough to be implemented as designed ◦ Minor issue: Didn’t design how to inject proxy sinks and sources
on the client side
Runtime view was critical for understanding the system and reasoning about it ◦ Code (static view) is sometimes tightly coupled for non-obvious
reasons Thread synchronization and locking Hard to understand this without runtime view
◦ Had an edge case that threatened implementation of our design Bosch and Berkeley team helped us overcome it with the help of runtime
perspective diagrams Issue: Type inference system was not designed for distributed
environment
Team HandSimDroid 34
Homer (UI Designer) Static Perspective
35
Actors depend directly on Desktop Java [Lattix]
Extensibility: Abstract Factory & dependency injection - Easy to add new Actor for Android
Usability – one to one mapping from preview to the UI
Generally worked out quite well ◦ Detail level was high enough since complexity was lower
Good decision: Netbeans Visual Library ◦ Designed WYSIWYG layout editor in 3+ weeks ◦ A lot more usable that what Ptolemy currently has ◦ Saved a lot of time
Not so good decision: Google Guice dependency injection framework ◦ Guice’s license was compatible with Ptolemy’s
Berkeley team asked us to remove it because of potential complication with licensing issues
Core Ptolemy had to depend on Guice
◦ Relatively easy fix for us because only small portion of Guice was used
◦ Reflection: keep an eye on licensing issues and if functionality is simple enough, implement it yourself even if this impacts maintainability
Team HandSimDroid 36
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 37
Tool Name
Continuous Integration Jenkins
Coding Style Checkstyle
Code Reviews Rietveld
Static Analysis FindBugs
Unit Testing JUnit
Test Coverage Cobertura
Bug Tracking Bugzilla
Team HandSimDroid 38
Used Jenkins (Hudson) ◦ Build system for 4 trees ptserver & Homer
Trunk Actor porting branch
ptdroid Trunk Actor porting branch
Rationale: stable code within trunk and code breaking core Ptolemy in the branch ◦ Merge the branch once it’s stable ◦ Risk mitigation: Didn’t want to break open source project
with our changes and impact release schedule of Berkeley team
◦ Increased configuration management complexity
Team HandSimDroid 39
less
than 20
min.
55%
20 -
60
min.
10%
60 - 120
min.
7%
120- 240
min.
7%
more
than 240
min.
21%
Duration with a broken
build (in minutes)
less than 20 min. 20 - 60 min.
60 - 120 min. 120- 240 min.
more than 240 min.
less
than 20
min.
47%
20 - 60
min.
22%
60 - 120
min.
3%
120- 240
min.
5%
more
than
240
min.
23%
Duration with failing tests
(in minutes)
less than 20 min. 20 - 60 min.
60 - 120 min. 120- 240 min.
more than 240 min.
Team HandSimDroid 40
Time to setup: ~10 hours * Based on 690 builds
JUnit tests and Cobertura for test coverage metrics ◦ Most tests were more system tests rather than unit
tests focusing on individual functionality
End to end client server simulation
Hard to write unit tests for multi threaded systems running in distributed environment
◦ Jenkins often complained about test breakage
Reason: concurrency & timing issues that were not encountered during development
Continuous integration helped catch those problems
Team HandSimDroid 41
Checkstyle ◦ Check the code base for conformance with naming and style
conventions ◦ Very important for Berkeley team ◦ Initially Berkeley team reviewed our every commit Hard to keep up with reviews while the module is in the development
◦ With Checkstyle integration we had a public venue to track and monitor style issues
FindBugs ◦ Static analysis tool ◦ Runs slowly on developer machine ◦ Jenkins allowed us to track FindBugs trends continuously and
know which commit introduced potential issues ◦ Issues were fixed organically before development on a feature was
complete – no need to file a bug
Team HandSimDroid 42
Synchronous Reviews with Bosch and Berkeley ◦ Mostly focused on knowledge sharing and
understandability of the code and comments ◦ Did not identify major bugs
Asynchronous Reviews within the team ◦ Rietveld tool Submit a change set to the website and ask team
mates to review it asynchronously
Inline comments
Saves time – no need to review code that was not changed
Team HandSimDroid 43
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 44
Don’t treat every task that we can earn value on ◦ Timeboxed tasks don’t give clear indication of
progress
Granularity of the tasks inhibited ability to see big picture
Architectural experiments and granularity of the architecture was helpful in identify issues and communicating them to stakeholders
Team HandSimDroid 45
Tailor TSP to be more agile with sprints to realize short term goals and make adjustments
Jenkins was helpful in managing technical stakeholder expectations since it acted as continuous deployment tool
Comments were useful in understanding the project given the size (350 KLOC) and we appreciated the importance of comments
Team HandSimDroid 46
Project Overview Demonstration Operations and Processes Planning and Adjusting Risks Quality Measures System Architecture Tools Overview Lessons Learned Client Feedback
Team HandSimDroid 47
“This was a very challenging project and the team did a great job meeting and exceeding my expectations.”
“I was very impressed at how quickly they were able to produce code and make changes and improvements weekly based on our feedback. This was one of the best MSE teams I've had the opportunity to work with.”
Team HandSimDroid 48
“The team did quite well meeting the functional requirements. The Ptolemy II code base is about 350K lines of non-commented code. The team needed to insert functionality at locations fairly close to the kernel, which requires understanding a large system and carefully inserting new functionality.”
“Overall, the project went quite well, the HandSimDroid team was very productive.”
Team HandSimDroid 49
“Great job all around! Very high professionalism. Nicely done!”
“It contributed immensely to our research objectives ”
Team HandSimDroid 50
Thanks!
Team HandSimDroid 51
A.k.a. slides to go into more details.
Team HandSimDroid 52
PIP Problem Suggestion
8 The team is not reviewing the metrics being collected to make better estimations and do better planning/tracking
Send out updated copy of handsimdroid stats on the weekly basis and add as agenda item with highlights what to look at in the stats - Planning manager would do that day before team meeting (24 hours)
15 With the larger number of hours, its more difficult to track where we are in the project and whether or not we will meet/miss our milestones
Individual workbooks will be updated every Tuesday, Thursday, and Saturday evenings
17 The team is having trouble tracking the members' involvement.
Upload timesheets each day after standup meetings (TL makes sure), and consolidate and validate the data right after (PM).
19 Workbooks are not updated at the end of every weekday common time
Team member in violation will pay $1 for each day that their workbook is out-of-date
Team HandSimDroid 53
0
100
200
300
400
500
600
700
800
Monday Tuesday Wednesday Thursday Friday Saturday Sunday
Earn
ed V
alu
e
Team HandSimDroid 54
Working odd hours: - Common times from 5pm-8pm - Part-timer mostly after 5pm, Friday,
and weekend - All meetings on Fridays and
weekends
0
5
10
15
20
25
30
35
40
45
50
8/29/2011 9/29/2011 10/29/2011 11/29/2011
Hou
rs
UPDATING PTOLEMY BUGZILLA
UPLOAD DOCUMENTS TO DOGBERT
UPDATING DOCUMENTATIONS
POSTER PRESENTATION
POSTMORTEM
HANDLING CHANGE REQUESTS
TESTING AND MAINTANANCE
REFLECTION PAPER
Team HandSimDroid 55
Total 632 hours
0
5
10
15
20
25
30
35
Ptdroid Ptserver Homer General
RESOLVED
IN_PROGRESS
CONFIRMED
Team HandSimDroid 56
0
2
4
6
8
10
12
14
Blocker Critical Major Normal Minor Enhancement
Design
Code
Team HandSimDroid 57
FindBugs #
BC_UNCONFIRMED_CAST 1
DC_DOUBLECHECK 1
DLS_DEAD_LOCAL_STORE 1
DMI_HARDCODED_ABSOLUTE_FILENAME 1
IS2_INCONSISTENT_SYNC 2
NP_NULL_ON_SOME_PATH_FROM_RETURN_VALUE 4
OBL_UNSATISFIED_OBLIGATION 1
SE_BAD_FIELD 2
SE_NO_SERIALVERSIONID 2
SP_SPIN_ON_FIELD 1
ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD 1
Total 17
Team HandSimDroid 58
37
60%
20
32%
4
6%
1
2%
Test
Code Review
Design Review
Code
Team HandSimDroid 59
Inspections: 3.8% time Yield: 39% bugs
Team HandSimDroid 60
Before After
Inline comment
The team had trouble integrating modules due to everybody having a big picture; Lots of time is wasted integrating; code does not meet quality standards.
◦ Mitigation: Design reviews focusing on integration points, Checkstyle and
Cobertura for tracking quality, Bugzilla for assigning tasks focusing on quality issues (i.e. write test case)
We becoming more relaxed as we get the most functionality done; this might affect our schedule since we might not have enough time to close bugs or meet quality metrics.
◦ Mitigation: Baseline the code (implement all high and medium priority
requirements), allocate last cycle to focus on quality, friendly competition on bug bashing – free lunch and ice cream for winners
Team started developing code without following proper Ptolemy guidelines; Berkeley team might become alienated and/or we might not get needed help from them, project might not have impact.
◦ Mitigation: Integrate Checkstyle with cont. integration and publicly track it
Team HandSimDroid 61
0
20
40
60
80
100
1/19/2011
1/26/2011
2/2/2011
2/9/2011
2/16/2011
2/23/2011
3/2/2011
3/9/2011
3/16/2011
3/23/2011
3/30/2011
4/6/2011
4/13/2011
4/20/2011
4/27/2011
5/4/2011
5/11/2011
5/18/2011
5/25/2011
6/1/2011
6/8/2011
6/15/2011
6/22/2011
6/29/2011
7/6/2011
7/13/2011
7/20/2011
7/27/2011
8/3/2011
Perc
enta
ge
Normalized Planned EV Normalized Actual EV
Team HandSimDroid 62
Architecture compliance review (6/27)
Release 4: Documentations
MOSP
SRS
Architecture docs
Release 2: ptdroid
Release 3: Homer
Release 1: ptserver
EOSP
Snapshot: 8/3
Bugzilla ◦ Initial set up was hard but once set up worked well
Easy to organize bugs using different criteria
Status update emails when bug status changes
Daily reminder emails to ensure that bugs don’t get stale
◦ Added custom fields
Bug Injection Phase
Bug Detection Phase
Bug Removal Phase
Team HandSimDroid 63
Performance measure: latency 150-300ms
Extensibility: all actors were ported in less than 2 weeks. ◦ Plotter was the hardest one (8KLOC)
Reliability: latency monitoring stops the system
Usability: UI in Homer maps almost one to one to Android
Team HandSimDroid 64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
Entities (actors or filters), links, and the pipe‐and-filter architecture within Ptolemy. The entities are linked and can only communicate through existing links connected with relations
79
Element Responsibility
Entity Entities are the filters in the supported pipe-and-filter architecture. They are responsible for computation within the system. An entity can contain any number of ports.
Port Port is a point of connection for the entity. It can serve both as output and input, and knows if there’s a token waiting for processing.
Relation The relation is responsible to keep track of links. It controls the creation and termination of links
Token An encapsulation of a data message. Token serves to identify the message type, size and data boundaries
Link A link defines an association between an entity's port and a relation.
A sample communication flow of the Ptolemy pipe-and-filter architecture based model. The model uses immutable objects called tokens to transfer any kind of data.
80
Element Responsibility
Entity Entities are the filters in the supported pipe-and-filter architecture. They are responsible for computation within the system. An entity can contain any number of ports.
Port Port is a point of connection for the entity. It can serve both as output and input, and knows if there’s a token waiting for processing.
Relation The relation is responsible to keep track of links. It controls the creation and termination of links
Token An encapsulation of a data message. Token serves to identify the message type, size and data boundaries
Link A link defines an association between an entity's port and a relation.