Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
Data Collection Questionnaire
Application Development & Enhancement
Agile - Scrum
Version : 1.0 Final Release Date : 7 January 2019
© ISBSG 2019
`Data Collection Questionnaire : Agile-Scrum
Table of Contents
Introduction.............................................................................................................................................................4
The International Software Benchmarking Standards Group...............................................................................4
Submitter..................................................................................................................................................................6
General Information.................................................................................................................................................6
Organizational Information......................................................................................................................................7
Application Information...........................................................................................................................................8
Application...............................................................................................................................................................9
Application type...................................................................................................................................................9
Application Type Details.......................................................................................................................................9
Application Environment......................................................................................................................................9
Users....................................................................................................................................................................9
Application Complexity..........................................................................................................................................10
Organization.......................................................................................................................................................10
Application.........................................................................................................................................................10
Technology.............................................................................................................................................................11
Package (Commercial of the shelve)...................................................................................................................11
Programming Language......................................................................................................................................11
Database............................................................................................................................................................11
Architecture...........................................................................................................................................................11
Environment...........................................................................................................................................................12
Middleware........................................................................................................................................................12
Instances............................................................................................................................................................12
Team......................................................................................................................................................................13
Scrum Attributes....................................................................................................................................................15
Agile Practices........................................................................................................................................................16
Tools.......................................................................................................................................................................17
Quality Metrics.......................................................................................................................................................18
Scrum Metrics........................................................................................................................................................19
Test Automation Metrics........................................................................................................................................20
Continuous Integration, Delivery and Deployment................................................................................................21
© ISBSG Page: 2
`Data Collection Questionnaire : Agile-Scrum
Estimation..............................................................................................................................................................22
Actual Effort and Cost.............................................................................................................................................23
Size Estimate Other than Story Points....................................................................................................................24
Team and User Satisfaction Survey........................................................................................................................25
Additional Information...........................................................................................................................................26
© ISBSG Page: 3
`Data Collection Questionnaire : Agile-Scrum
Introduction
The International Software Benchmarking Standards Group
The International Software Benchmarking Standards Group, (ISBSG), is a not-for-profit organization.
The ISBSG Software Project Repository exists: to provide software development practitioners with industry output standards against which they may
compare their aggregated or individual projects, and; to provide real data of international software development that can be analyzed to help improve the
management of IT resources by both business and government.
Benefits of Your InvolvementContributing to the ISBSG Project Repository provides you with an opportunity to see how your projects compare with the industry. You will be able to answer the question:
"Where do we fit in and why?" In addition, you are contributing to a body of software engineering knowledge that is open, available
and used for the betterment of the ICT profession.
AnonymityThe anonymity of contributors to the ISBSG repository is assured. The ISBSG has a process in place that ensures that the identity of contributors remains confidential.
Submitting informationPlease use the e-mail address [email protected] (administrator) for all communication with ISBSG and submitting your information.
Further steps1. The administrator assigns a submission ID to this submission.2. The administrator removes all data from the sheet that references to the submitting organization.3. The administrator forwards the anonymized submission to the repository manager.4. The repository manager analyzes the data and may have some questions.5. The repository manager sends the questions to the administrator.6. The administrator contacts the submitter in order to have the questions resolved.7. The repository manager processes the submission.8. The repository manager creates a benchmark report for the submission.
Thank you for submitting data! You are helping the industry advance step by step!In case of questions, the front office will contact you on the provided e-mail address:
© ISBSG Page: 4
`Data Collection Questionnaire : Agile-Scrum
Glossary of TermsA Glossary of Terms is available as a separate document that you can download from http://www.isbsg.org
© ISBSG Page: 5
`Data Collection Questionnaire : Agile-Scrum
SubmitterContact Information Answer
Contact person E-mail Role Organization
General InformationQuestion Answer Notes
Amendment to earlier submission <Previous ISBSG ID, if any> Please provide ISBSG IDDevelopment Type New Devlopment Please give details in case of
‘N/A’
What is the Project ID you wish to use for this project?
<Project Identification> This is for your own reference ISBSG will assign an ISBSG ID as well
Start date of the data/benchmark period DD-MM-YYYYEnd date of the data/benchmark period DD-MM-YYYYWhat was the most important objective for this release?
<Select Answer>
What was the second most important objective for this release?
<Select Answer>
What was the third most important objective for this release?
<Select Answer>
Release Definition of Done (DoD) <DoD> Please list the most important DoD criteria In case DoD criteria s not defined please mention ‘N/A
Was Release Definition of Done (DoD) completed?
<Select Answer> Did the release comply to all DoD criteria? Specify details in case the Release DoD was not completed.
Release Elapsed duration in months <Number> Mention the duration planned for the release in months.
For e.g.., 3 months, 1.5 months, 0,5 months
Release Delivered Date DD-MM-YYYY Mention the date the release is delivered
© ISBSG Page: 6
`Data Collection Questionnaire : Agile-Scrum
Organizational InformationQuestion Answer Notes
In which industry(s) do the software’s end users work?
If other or N/A, please specify
<Select Answer> End users are the people who physically interact with the software.
<Industry Details> Please give details in case of Others or N/A’
What is the type of the organization the software’s end users work?
<Organization Type> For instance, Banking or Insurance
End user’s organization location(s)/countries <Location(s) / Country Name(s) >
Conversion rate of hours per full time equivalent (FTE) per year
<Number> The number of hours an FTE works in the country where the software was developed
© ISBSG Page: 7
`Data Collection Questionnaire : Agile-Scrum
Application InformationQuestion Answer Notes
Is the application developed/maintained / supported internal (in house) or external?
Internal (in house) Please give details in case of ‘N/A’
Is the application developed/maintained/ supported on an onshore, nearshore or offshore location?
☐ Onshore
☐ Nearshore
☐ Offshore
Please select multiple options, as applicable
Onshore Country <Country Name> If Work was carried out in an onshore country, which country was this?
Nearshore Country <Country Name> If Work was carried out in a nearshore country, which country was this?
Offshore Country <Country Name> If Work was carried out in an offshore country, which country was this?
Is the application bespoke or Commercial off the Shelf (CotS) software?
<Select Answer> Example of packages: SAP, Peoplesoft, Oracle
If Commercial of the shelf, please specify the supplier
<Supplier Name>
Application go live date DD-MM-YYYY
Remarks
<Additional remarks, if any>
© ISBSG Page: 8
`Data Collection Questionnaire : Agile-Scrum
ApplicationQuestion Answer
Notes
Application typeWhat type of application, within the selected application group, has the project produced or enhanced?
Real-time Application Please provide details if answer is ‘N/A’
In case of a Business Application, select from the following list
<Select Answer> Please provide details if answer is ‘N/A’
In case of a Realtime application, select from the following list
<Select Answer> Please provide details if answer is ‘N/A’
In case of a Mathematically intensive application, select from the following list
<Select Answer> Please provide details if answer is ‘N/A’
If other, please specify <Specify>
Application Type DetailsIf you have selected an application within more than one application group, please estimate amount of functionality provided in each application group.
Business application <nn.nn>%Real-time application <nn.nn>%Mathematically-intensive application <nn.nn>%
Application EnvironmentImplementation platform <Select Answer> Please provide details if
answer is ‘N/A’If other, please specify <Specify>If mobile or device embedded, please specify <Select Answer> Please provide details if
answer is ‘N/A’If other, please specify <Specify>
UsersQuestion Answer Notes
Number of active (distinct) end users <Number>Number of key users <Number>Average number of concurrent users <Number>Maximum number of concurrent users <Number>Number of distinct installations <Number>
© ISBSG Page: 9
`Data Collection Questionnaire : Agile-Scrum
Application ComplexityQuestion Answer
Notes
OrganizationNumber of different countries servicedNumber of different user locationsNumber of different user profiles
ApplicationApplication importance for the organization
☐ Process not important
☐ Process important
☐ Process critical
☐ Business not important
☐ Business important
☐ Business critical
☐ N/A
Please specify details if answer is ‘N/A’
Total number of interfaces <Number>Number of batch interfaces <Number>Number of real time-online interfaces <Number>
Operating System (OS) on which the application is deployed
<Specify>
Hardware (Brand, type) on which the application is deployed
<Specify>
Age of the application <Years>Expected date of retirement/replacement DD-MM-YYYYNumber of releases in this benchmark period <Number>Percentage code changed (in benchmark/data period)
<nn.nn>%
Percentage functionality changed (in benchmark / data period)
<nn.nn>%
Percentage of changes caused by rework (in benchmark / data period)
<nn.nn>%
© ISBSG Page: 10
`Data Collection Questionnaire : Agile-Scrum
TechnologyQuestion Answer
Notes
Package (Commercial of the shelve)Degree of customization (%) <nn.nn>%
Programming LanguageMain programming language <Specify>2nd programming language <Specify>3rd programming language <Specify>…. programming language <Specify>
DatabaseMain database <Specify>2nd database <Specify>3rd database <Specify>…. database <Specify>
ArchitectureQuestion Answer Notes
Architecture <Specify> Architecture of the application: stand alone, multi-tier, Client-server, multi-tier with web public interface
Client / Server <Select Answer> Indicator of whether the application or product requires more than one computer to operate different components or parts of it
Client roles <Specify> The roles performed by the computers that provide interface to the software’s external users.
Server roles <Specify> The services provided by the host/server computer(s) to the software application or product.
Type of server <Specify> A description of the server to the software application or
© ISBSG Page: 11
`Data Collection Questionnaire : Agile-Scrum
Question Answer Notes
product.Client server description <Specify> A description of the
architecture of the client/server software application or product.
Web development <Specify> A derived indicator of whether the project data includes any comment that it is a web-development.
EnvironmentQuestion Answer Notes
MiddlewareWeb server <Specify>Message server <Specify>Another platform <Specify>
InstancesNumber of production instances <Number>Number of acceptation instances <Number>Number of test instances <Number>Number of development instances <Number>Number of other instances <Number>
Remarks
<Additional remarks, if any>
© ISBSG Page: 12
`Data Collection Questionnaire : Agile-Scrum
TeamQuestion Answer Notes
Scrum Training Received by team Details of Scrum training received by the team. Also mention if any role/work specific training undergone by the teamSpecify details of training conducted onshore, offshore and near shore
Number of Agile teams working on this release <Number>Number of Distributed teams <Number> Mention the number of
distributed teams that are part of the release. Teams are considered as ‘co-located’ if all team members are located in same office or city. Otherwise team is ‘distributed’ team
Number of Co-located Teams <Number> Mention the number of co-located teams that are part of the release. Teams are considered as co-located if team members are located in same office or city. Otherwise team is distributed team
Team Cross functional <Select Answer>
Average No. of people in a team <Number> Average Team Size.Average per day Productive Hours considered <Number> Specify the average per day
productive hours considered for a sprint.
For e.g. 6.5 Hours/ day or 7 Hours/Day or 8 Hours/Day
Number of Scrum masters with more than 5 years of experience
<Number>
Number of Scrum masters with experience between 3 years to 5 years
<Number>
Number of Scrum masters with less than 3 years of experience
<Number>
Number of times Scrum Master replaced during the release
<Number> Number of times scrum master changed during the project/release.
© ISBSG Page: 13
`Data Collection Questionnaire : Agile-Scrum
Number of Product Owners with more than 5 years of experience
<Number>
Number of Product Owners with experience between 3 years to 5 years
<Number>
Number of Product Owners with less than 3 years of experience
<Number>
Number of times Product Owners replaced during the release
<Number> Number of times product owner changed during the project/release.
Number of Scrum team members with more than 5 years of experience
<Number>
Number of Scrum team members with experience between 3 years to 5 years
<Number>
Number of Scrum team members with less than 3 years of experience
<Number>
Average per day Productive Hours considered <Number>Total Number of Sprints in this Release <Number>Sprint duration (in weeks) <Number>
Remarks
<Additional remarks, if any>
© ISBSG Page: 14
`Data Collection Questionnaire : Agile-Scrum
Scrum AttributesQuestion Answer Notes
Total Number of Sprints in a Release <Number> Mention the number of sprints executed in the release. Note: Provide the sum of sprints executed by all teams part of the releaseE.g., Release duration is 3 months, Sprint duration is 2 weeks. So, 6 sprints by a team in a releaseNumber of teams part of release: 3Total number of sprints in a release : 18
Sprints Duration (in weeks) <Number> Ideally sprint duration should be within 2-4 weeks
Sprint Definition of Done <SelectAnswer>
Acceptance Criteria Defined for each user story <SelectAnswer>
Backlog Refinement Session <SelectAnswer>
Sprint Review <SelectAnswer>
Customer Demo performed at regular intervals during the project
<SelectAnswer>
Sprint Retrospection <SelectAnswer>
Product Backlog <SelectAnswer>
Mention if the teams are using the product backlog to maintain user stories, bugs and other work items
Sprint Backlog <SelectAnswer>
Mention if the teams are using the sprint backlog to maintain the committed user stories, bugs and other work items for the sprint.
Impediment List <SelectAnswer>
Mention if the team is using the impediment list artifact to maintain impediments encountered during sprint
Architecture/ Design/ Functional specification <SelectAnswer>
Mention, if any specification document is used. Please provide details in case of ‘N/A’
Remarks
<Additional remarks, if any>
© ISBSG Page: 15
`Data Collection Questionnaire : Agile-Scrum
© ISBSG Page: 16
`Data Collection Questionnaire : Agile-Scrum
Agile PracticesQuestion Answer Notes
Test Driven Development <Select Answer>Pair Programming <Select Answer>Business Driven Development <Select Answer>Feature driven development <Select Answer>Other development practices <specify>
Remarks
<Additional remarks, if any>
© ISBSG Page: 17
`Data Collection Questionnaire : Agile-Scrum
ToolsQuestion Answer Tool Name Notes
Agile Project Management Tool used?
<SelectAnswer>
<Specify tool name, in case answer is yes>
Code Quality Tools used? <SelectAnswer>
<Specify tool name, in case answer is yes>
Test Automation Tools used? <SelectAnswer>
<Specify tool name, in case answer is yes>
Build / CI Tools used? <SelectAnswer>
<Specify tool name, in case answer is yes>
Test Management tools used? <SelectAnswer>
<Specify tool name, in case answer is yes>
Performance Testing Tools used?
<SelectAnswer>
<Specify tool name, in case answer is yes>
Vulnerability Testing Tools used?
<SelectAnswer>
<Specify tool name, in case answer is yes>
Release Automation / Continuous Delivery Tool used?
<SelectAnswer>
<Specify tool name, in case answer is yes>
Any other tools used <SelectAnswer>
<specify any other tools used that are not listed elsewhere>
Remarks
<Additional remarks, if any>
© ISBSG Page: 18
`Data Collection Questionnaire : Agile-Scrum
Quality MetricsQuestion Answer Notes
Customer Reported Minor Defects <Number> Minor defects reported within three months of release Go Live
Customer Reported Major Defects <Number> Major defects reported within three months of release Go Live
Customer Reported Extreme Defects <Number> Extreme (Blocker and Critical) defects reported within three months of release Go Live
Total Defects Reported by Customer <Number> Total number of defects reported within three months of Go Live This column shows the total of defects reported (Minor, Major and Extreme).or Where no breakdown is available, the single value is specified, if known.
Defects Density <Number> Defects per story points calculated as: Total Defects Reported by Customer / Total Release Story Points delivered
Measures the quality of software in terms of defects delivered against total story points delivered for the release
Code Coverage Percentage 1st Language <%> Code Coverage(%) is calculated as: Lines of Code Covered by Automated Testing / Total Lines of Code
Code Coverage Percentage 2nd Language <%> Code Coverage(%) is calculated as: Lines of Code Covered by Automated Testing / Total Lines of Code
Code Coverage Percentage 3rd Language <%> Code Coverage(%) is calculated as: Lines of Code Covered by Automated Testing / Total Lines of Code
Remarks
<Additional remarks, if any>
© ISBSG Page: 19
`Data Collection Questionnaire : Agile-Scrum
Scrum MetricsQuestion Answer Notes
Committed Velocity (in Story Points) <Number> Sum of story points for the committed Features/User Stories for the release
Release Velocity (in Story Points) <Number> Sum of story points for completed Features/User Stories delivered in the release
On Time Delivery (%) <%> Mention the percentage of functionality/requirements delivered within the agreed timelines for the release
Is all Committed Functionality Released within committed time?
<SelectAnswer>
Mention Y/N if the agreed scope / functionality for the release are delivered within the committed time
‘Y’ indicates the agreed scope are delivered within time‘N’ indicates the agreed scope were not delivered within time
Remarks
<Additional remarks, if any>
© ISBSG Page: 20
`Data Collection Questionnaire : Agile-Scrum
Test Automation MetricsQuestion Answer Notes
Automated Unit testing – 1st Language <%> Mention the percentage of unit test automation done for the 1st language
Automated Unit testing – 2nd Language <%> Mention the percentage of unit test automation done for the 2nd language
Automated Unit testing – 3rd Language <%> Mention the percentage of unit test automation done for the 3rd language
Automated API testing <%> Mention the percentage of API Testing automation
Automated Functional testing <%> Mention the percentage of automation done for Functional Testing
Automated Regression testing <%> Mention the percentage of automation done for Regression Testing
Automated Performance testing <%> Mention the percentage of automation done for Performance Testing
Automated Vulnerability testing <%> Mention the percentage of automation done for Vulnerability Testing
Any other test automation <%> Mention if any other testing performed and percentage of automation done for that testing type
Overall Automation Testing <%> Mention the overall percentage of automation for a Product/Release
Remarks
<Additional remarks, if any>
© ISBSG Page: 21
`Data Collection Questionnaire : Agile-Scrum
Continuous Integration, Delivery and DeploymentQuestion Answer Notes
Build Frequency <Specify> Mention the frequency as on demand, multiple times in a day, daily, weekly, bi-weekly, monthly etc.
Build Trigger <Select Answer> <
Mention the mode of build triggered Manual/Automated
Is automated unit testing executed with build <SelectAnswer>
Common Artifact Repository <Specify> Mention if versions are maintained in common repository for every build that is successful
Deployment frequency to Production <Specify> Mention the frequency as Hourly, daily, weekly, bi-weekly, monthly etc.,
How many times code was deployed in production?
<Number>
Number of sites to be deployed <Number>
Remarks
<Additional remarks, if any>
© ISBSG Page: 22
`Data Collection Questionnaire : Agile-Scrum
EstimationQuestion Answer Notes
Size estimate method <Specify> Method used to estimate the story size e.g. planning poker
Release size estimate in Story Points <Number> Estimated story points for the release
Release Capacity – (FTE Days) <Number> Mention the available or planned capacity for the release. Capacity is calculated based on the total team size for the release, release duration and removing the vacations, training, organizational activities during the release duration
Release Delivery Date Committed <DD-MM-YYYY>
Delivery date estimate: Estimate of the delivery date for the release made during the planning activity.
Release Cost Estimate <Number> Estimated cost of Release made during the planning activity.
Cost Estimate Currency <Specify> USD, GBP, Euro etc.Currency Exchange rate to the USD at the time of estimate
<Number> Mention the currency conversion rate to USD
Cost estimate method <Specify>Method used to estimate the cost. <Specify>Estimation comments <Specify> Additional comments related to estimation
Remarks
<Additional remarks, if any>
© ISBSG Page: 23
`Data Collection Questionnaire : Agile-Scrum
Actual Effort and CostQuestion Answer Notes
Total Team Effort Estimate <Number> Estimate of the total effort of the release of the teams.
Average hour rate of all hours spent in the Project/Release
<Number> The sum of all effort hours spent divided by all cost related to these hours.
Actual Total Team Effort The sum of all effort hours spent on the project/release. Mention the effort only for scrum team including scrum master
Team Cost <Number> The sum of the cost of all effort hours spent on the project/release. Mention the total cost only for scrum team including scrum master
%Effort spent on Requirements <nn.nn>% Indicate the percentage of the effort hours spent on requirements gathering and analysis by the team, not the product owner.
%Effort spent on Design <nn.nn>% Indicate the percentage of the effort hours spent on User story definition and possible other design documentation
%Effort spent on Coding <nn.nn>% Indicate the percentage of the effort hours spent on Coding, configuring or parametrization and bug fixing after testing
%Effort spent on Test <nn.nn>% Indicate the percentage of the effort hours spent on all Test activities, e.g. test case design, development and execution
%Effort spent on Implementation <nn.nn>% Indicate the percentage of the effort hours spent on brining the software in a live production environment
Total Release Cost Estimate <Number> Estimate of the total cost of the release. This includes Human Resource, Software, Hardware, Infrastructure, Facilities etc.,
Currency <Specify> USD, GBP, Euro etc.Currency Exchange rate to the USD at the time of estimate
<Number> Mention the currency conversion rate to USD
Actual cost comments <Specify> Additional comments related to actual cost
© ISBSG Page: 24
`Data Collection Questionnaire : Agile-Scrum
Remarks
<Additional remarks, if any>
Size Estimate Other than Story PointsQuestion Answer Notes
Function Points (Project Size) <Number> The number of function points measured for the Product/Release
Functional Size Measurement Standard <Specify> Specify the method used to measure the functional size: IFPUG, Nesma, COSMIC, FISMA, Mark II, CISQ
Version of the Standard <Specify> For instance, Nesma 2.3 or IFPUG 4.11Method for calculating Project Size <Select
Answer>Did you measure the Project Size using the following formula?Project Size = FP Added + FP modified + FP deleted?
Nesma FPA enhancement <SelectAnswer>
Did you the Nesma FPA for Enhancement method? This method can also be applied to IFPUG. It considers impact factors for changed functionality, based on complexity of the change, and deleted functionality.
Is the FPA analyst certified? <SelectAnswer>
Please specify if the FPA analyst holds a valid certification for the method used to measure.
Lines of Code <Number> The number of the source lines of code (SLOC) produced for the Product/Release.
Tools used to capture LoC <Specify> Specify the tools used to calculate LoCOther Size Units <Specify> Specify any other Sizing methods used e.g.
Function pointsOther Size <Number>
Remarks
<Additional remarks, if any>
© ISBSG Page: 25
`Data Collection Questionnaire : Agile-Scrum
Team and User Satisfaction SurveyQuestion Answer Notes
Team Happiness Rating <Select Answer>
1 is Low and 5 is High
User Satisfaction Rating <Select Answer>
1 is Low and 5 is High
User Satisfaction method <Specify> Specify the method/tool used to capture user satisfaction ratinge.g.., Questionnaire, Net Promoter Score etc.,
Remarks
<Additional remarks, if any>
© ISBSG Page: 26
`Data Collection Questionnaire : Agile-Scrum
Additional InformationQuestion Answer Notes
Is the team involved in implementing Process Improvement Program?
<Select Answer> A process improvement program comprises a planned series of actions to improve how the software development team does its work
Does your organization have standard metrics program?
<Select Answer> Specifies whether the organisation has implemented a metrics program.
Metrics program duration (Years) <Number> Duration of the organisation's metrics program (in years) in case the organisation has implemented a metrics program.
Remarks
<Additional remarks, if any>
© ISBSG Page: 27
`Data Collection Questionnaire : Agile-Scrum
Thank You
© ISBSG Page: 28
Please send to [email protected]