Università degli Studidell’Insubria
An Evaluation of Simple Function Pointas a Replacement of IFPUG Function Point
Luigi Lavazza
Dipartimento di Scienze Teoriche e Applicate
Università degli Studi dell'Insubria - Varese
Roberto Meli
SiFPA
IWSM 2014
Contents
Research motivations
Simple Function Point: definition
FP vs. SiFP as size measures: statistical analysis
FP vs. SiFP as effort predictors: statistical analysis
Expected advantages in using SiFP
Rotterdam, October 20142
Two crucial and intriguing questions …
Why should anybody abandon the IFPUG FSMM ?
Why should anybody need a brand new FSMM just right now ?
Rotterdam, October 2014IWSM 2014 3
Why should anybody abandon the IFPUG FSMM ?
Because you can get the same results with less effort, time and costSetup of the measurement system
– training, individual study, certification, field experience, guidelines, ...
Execution of the measures– measurement productivity
Documentation of the measures– registered details
Measure’s verification & validation– audit, investigation, contradictory, litigation, ...
Rotterdam, October 2014IWSM 2014 4
Why should anybody need a brand new FSMM just right now ?
Because the current most popular alternative FSMMs present the same problems of the IFPUG method: too much details and complexity.
Because the market is now declining and the organizations need to cut off all non strictly crucial costs.
Because the "agile movement" is conquering considerable attention and business spaces and actual FSMMs are not perceived as agile at all.
Because, despite everything, you can not loose the investments made over the years on IFPUG measures and no alternative FSMM is highly convertible with IFPUG.
Rotterdam, October 2014IWSM 2014 5
What’s the need then ?
Today’s software market requires
fast, agile, effective functional size measurement methods with low impact on production processes,
which require not too specialized skills, that are reliable in results, not dependent on expert’s opinions and technology.
The resulting measures should be adequately correlated to effort, cost, duration of a new development or functional enhancement software project.
Current Functional Size Measurement Methods (FSMM) are only partially compliant with these needs.
Rotterdam, October 2014IWSM 2014 6
IWSM 2014
IFPUG Function Point: the software model
SW application functional specifications
Logical file Transaction
Data Element Type
Record Element Type
FTR
0..*
I/O
1..*
Allan J. Albrecht
Rotterdam, October 2014
Primary intent
7
IWSM 2014 8
Simple Function Point: the software model
SW application functional specifications
Logical file Transaction
Data Element Type
Record Element Type
FTR
0..*
I/O
1..*
SW application functional specifications
UGDG UGEP
Rotterdam, October 2014
Primary intent
UGEP
"An atomic set of functional user requirements conceived for processing purposes. It refers to an informational or operational goal considered significant and unitary by the user and includes all automated, mandatory and optional activities needed to meet the goal. After an UGEP is concluded, the MSA to which it belongs must be in a logically consistent state."
Unspecified Generic Elementary Process
IWSM 2014 9 Rotterdam, October 2014
UGDG
Fundamental data group
Auxiliary or non functional data group
Only the first category (the fundamental data group) meets user functional requirements and can therefore be identified as a UGDG.
Role
Unspecified Generic Data Group
Employee
"An atomic set of user requirements having a storage purpose. It refers to a single logical data set of interest to the user, for which information must be kept persistently."
IWSM 2014 10 Rotterdam, October 2014
SiFP Definition process
With the IFPUG measurement, weights are computed for each data and transaction function, by analyzing its details.
With the SiFP measurement, weights are fixed Weights were derived from the analysis of the ISBSG dataset in order
to maximize convertibility
SiFP = 4.6 #UGEP + 7 #UGDG
IWSM 2014 11 Rotterdam, October 2014
The number of elementary processes, without considering the primary intent
The number of logical data files, without considering if they are internal or external
Measurement Procedure
IWSM 2014 12 Rotterdam, October 2014
Measurement process: FP vs. SiFP
IWSM 2014 13
Gather the available documentation
Identify application boundaries
Determine the measurement goal and scope
Locate Elementary Processes and Logical Data Files
Identify transactions’ primary intents, internal/external files, RET, DET, FTR and
determine complexity
Calculate the functional size
Document and present the measurement
[SiFP]
[IFPUG]
5%
25%
60%
10%
Rotterdam, October 2014
Activities Cost fraction
5%
25%
60%
10%
Software Architecture
Technical Component Layer
(Technical Generalized Services)
Software
Layers
App 1Application Layer App 2 App ‘n’
Operating System Layer (File System)
Database Management System Layer
DBMS 1 DBMS 2
Application Component Layer
(Business Generalized Services)
Enterprise System
IWSM 2014 14 Rotterdam, October 2014
Where can you find reference manuals ?
In the public
domain
for free !
www.sifpa.org
IWSM 2014 15 Rotterdam, October 2014
Just one final highlight
SiFP is not an IFPUG approximation technique like Early & Quick FP, Indicative FP, extrapolative FP and many others based on average scores…
SiFP is a FSMM which is (actually) higlhy “convertible” with IFPUG method and it gives “precise”, not approximate, measurements of software functional requirements.
Rotterdam, October 2014IWSM 2014 16
Research questions
How well are IFPUG FP and SiFP correlated?
How large is the difference of FP and SiFP size measures for the same software application?
To answer these questions, we analyzedThe ISBSG dataset (also used for the definition of the
BFC weights)A second business dataset
IWSM 2014 17 Rotterdam, October 2014
Plot FP vs. SiFP (ISBSG)
IWSM 2014 18
Rotterdam, October 2014
Regression lines (ISBSG)
IWSM 2014 19
SiFP = 0.998 UFP
Without outliers
With outliers
Rotterdam, October 2014
% differences (ISBSG)
IWSM 2014 20
Including outliers!
Rotterdam, October 2014
Regression lines (ISBSG)
IWSM 2014 21
In practice 1 SiFP = 1 UFP.
But is this statistically correct?
Rotterdam, October 2014
SiFP = 0.998 UFP
Line SiFP = UFP in in the 95% confidence interval.
Plot FP vs. SiFP (validation dataset)
IWSM 2014 22
Rotterdam, October 2014
IWSM 2014 23
% differences (validation dataset)
Rotterdam, October 2014
What happens if we consider sw portfolios?
Till now, we have considered measures taken on the same application using the two FSMM.
What happens when we consider a portfolio of applications and we are interested into the global portfolio size difference which is affected by the compensations among positive differences and negative differences in the relation between IFPUG FP and SiFP ? This is the case of application asset evaluations or of broad procurement contracts.
IWSM 2014 24 Rotterdam, October 2014
25
Results from the entire ISBSG DB
Rotterdam, October 2014
Individual differences
UFP SiFP DELTA DELTA% ABS(DELTA%)somma 284.005,0 282.881,6 -1.123,4 -0,4%minimo 10,0 9,2 -735,0 -47,4% 0,0%media 370,8 369,3 -1,5 11,6%
mediana 215,5 216,4 0,0 9,9%massimo 3.886,0 3.468,2 485,4 41,4% 47,4%
SiFP-UFP portfolio size difference
-60%-50%-40%-30%-20%-10%
0%10%20%30%40%50%
0 1000 2000 3000 4000 5000
Distribuzione Delta%
IWSM 2014
Results from ISBSG DB (various portfolio sizes)
UFP SiFP DELTA DELTA% ABS(DELTA%)Minimo 10.002,00 9.378,60 -725,40 -7,18% 0,00%Media 10.235,56 10.212,62 -22,94 2,01%Mediana 10.193,50 10.184,40 -13,00 1,58%Massimo 10.769,00 11.030,40 553,80 5,43% 7,18%
UFP SiFP DELTA DELTA% ABS(DELTA%)Minimo 25.005,00 24.236,00 -1.061,00 -4,19% 0,01%Media 25.429,83 25.468,33 38,50 1,51%Mediana 25.257,50 25.427,50 -3,50 1,16%Massimo 27.039,00 27.252,20 1.375,20 5,31% 5,31%
UFP SiFP DELTA DELTA% ABS(DELTA%)Minimo 50.001,00 48.757,40 -1.247,60 -2,49% 0,02%Media 50.451,70 50.604,99 153,29 1,11%Mediana 50.264,00 50.549,50 228,90 0,98%Massimo 52.187,00 52.109,40 1.594,20 3,18% 3,18%
IWSM 2014 26 Rotterdam, October 2014
Portfoliosize
Learnability
180 professionals with no previous knowledge of FSMMGrouped in 75 teams3 hours SiFP training 30-45 minutes for measuring the case study (~270 SiFP)
Class Frequency % cumulative-35% 1 1,33%-27% 2 4,00%-18% 9 16,00%-10% 24 48,00%
-1% 14 66,67%7% 18 90,67%
16% 2 93,33%24% 3 97,33%
>24% 2 100,00%
-10%<x<+7% 75%-18%<x<+16% 89%
IWSM 2014 27 Rotterdam, October 2014
Another research questions
If we estimate development effort based on SiFP instead of IFPUG FP, the achieved accuracy is the same, better or worse?
To answer this question, We derived the (best) model of effort as a function of UFP size, via
OLS log-log regression using ISBSG data We derived the (best) model of effort as a function of SiFP size, via
OLS log-log regression using ISBSG data We estimated efforts using both models, and computed residuals
(i.e., actuals-estimates) We compared the residuals
IWSM 2014 28 Rotterdam, October 2014
The models of effort
As it could be expected, the models found were very similar:Effort = 2.9204 × UFP0.9074 Effort = 2.773 × SiFP0.92
IWSM 2014 29 Rotterdam, October 2014
0 500 1000 1500 2000 2500 3000 3500 4000 45000
1000
2000
3000
4000
5000
6000
UFP / effortSiFP / Effort
Functional Size (UFP / SiFP)
Eff
ort
(h
ou
rs)
IWSM 2014 30
Absolute relative residuals (ISBSG)
Rotterdam, October 2014
Neither Wilcoxon sign rank test not Mann-Whitney (Wilcoxon rank sum) test reject the hypothesis that absolute relative residuals are equal in the two cases.
A problem
We performed the analysis using the ISBSG dataset
In the ISBSG dataset there are many application such that the UFP and SiFP measures are very similar: in such cases, the effort estimates will also be very similar.
To overcome this problem, we repeated the analysis considering only the applications whose FP and SiFP measures are different by no less than 20%.
IWSM 2014 31 Rotterdam, October 2014
Applications having UFP and SiFP measures substantially different
IWSM 2014 32
We considered only the points out of this area
Rotterdam, October 2014
For instance, this application is about 2300 SiFP and over 3000 UFP
Results obtained considering only application with size measure difference ≥ 20%
IWSM 2014 33
Models
Effort =29.23 × UFP0.789 Effort = 17.22 × SiFP0.86
Adjusted R2 0.551 0.539
MMRE 56.8% 58.1%
MdMRE 41% 39.3%
Pred(25) 33.3% 40%
Error range [ -85% .. 247% ] [ -88% .. 269%]
Rotterdam, October 2014
Results are very similar!
Results obtained considering only application with size measure difference ≥ 20%
IWSM 2014 34 Rotterdam, October 2014
Neither Wilcoxon sign rank test not Mann-Whitney (Wilcoxon rank sum) test reject the hypothesis that residuals are equal in the two cases.
Residuals of effort estimates
So, according to statistical analysis …
SiFP and IFPUG measures are equivalent!
IWSM 2014 35 Rotterdam, October 2014
IWSM 2014 36
On the ability to explain development effort
SW application functional specifications
Logical file Transaction
Data Element Type
Record Element Type
FTR
0..*
I/O
1..*
R2 = 0.551
SW application functional specifications
Logical file Transaction R2 = 0.539
Rotterdam, October 2014
IWSM 2014 37
On the ability to explain development effort
The performed analysis tell us that: To estimate the development effort, counting RET, DET, FTR, etc. is
of no use. In fact, SiFP do not count them, and get the same estimation
accuracy
Instead, we need models of this type:
Effort = f(S, p1, p2, …, pn)
S is the size, measured in FP or in SiFP
p1, p2, …, pn are parameters that characterize the software product (complexity, reliability, etc.) and the development process (develeopers’ capacity and experience, usage of tools and methodologies, etc.)
Rotterdam, October 2014
Conclusions
A new functional method based on just two BFC: Unspecified Generic Elementary Process (UGEP) Unspecified Generic Data Group (UGDG)
Totally compliant with IFPUG benchmarks It requires less details
2 to 5 times faster to measure Easy to learn and master Applicable by non specialists Less occasions for disagreement between clients and providers
1 SiFP =1 UFP Same development cost as IFPUG FP Conversion of historical data is immediate
IWSM 2014 38 Rotterdam, October 2014
Conclusions
By means of rigorous statistical analysis we were able to show that:The difference between UFP and SiFP measures is
generally small– Very small, when considering applications portfolios
For effort estimation, using SiFP instead of UFP does not cause the estimation accuracy to decrease
Conclusion: SiFP appear usable instead of UFP (and models based on UFP can be used with SiFP measures)
Anybody may test the model using his/her data with no additional effort provided that the number of EI, EO, EQ, ILF, EIF is known
IWSM 2014 39 Rotterdam, October 2014
IWSM 2014 40 Rotterdam, October 2014