Software Reliability Engineering Tool Nederlandse Testdag 27 november 2008 Alessandro di Bucchianico, Lime – TU/e Ed Brandt, Refis VOF

Software Reliability EngineeringTool

Nederlandse Testdag

27 november 2008Alessandro di Bucchianico, Lime – TU/e

Ed Brandt, Refis VOF

22008© Refis/Lime-TU/e

Definition of “Reliability”

"The probability that an item will perform a required function without failure under stated conditions for a stated period of time”

"The probability that an item will perform a required function without failure under stated conditions for a stated period of time”


Reliability growth curve

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

1

1 2 3 4 5 6 7

interval

reliability

period = 5




Step 1: Data collectiondocument data sources

record data collection procedures

clean-up data


Step 1: Data collectionID CategorySeverityReproducibility Date SubmittedDate Submitted Reporter Assigned To Priority Status Summary Description310 Defect major always 11-06-07 11:35 6-11-07 griffioenr griffioenr high closed Error due to the format of the I SI N code when adding new securityThis issue adresses more than 1 error. Error 1. The technical error occurs at more places within Thaler. Error 2. When adding a new security an error occurs when you add the I SI N code. I t rejects the format. Error 3. when adding the market behind the I SI N297 Defect major always 11-02-07 12:19 2-11-07 schmitzp schmitzp normal resolved intrest calculation on contract A7B16TDXPS000000 not accurateAfter first rollover duration is 15D but Thaler calculates interst over 17 days.293 Defect major always 11-01-07 17:49 1-11-07 MThiele MThiele high resolved ClientI D is one character short in the XML ClientI D in the XML (I DTCLI of _0000) consists of 8 characters while it consists of 9 characters in Thaler. The last character is cut off in the XML (Thomas Crahay is +000000L instead of +000000LK).287 Defect major always 10-31-07 08:36 31-10-07 vastenburgl junnej high resolved Reference rate is wrong When i add an FX spot contract with a private client, Thaler automaticly comes up with the reference rate. I n case of a spot contract, the reference rate shoudl be te spot rate (from FCM8). Now, Thaler comes up with a forward rate. See attached file.285 Defect block always 10-30-07 16:30 30-10-07 houtertp houtertp immediate resolved I nformation in CGS2 replaced by questionmarksWhen using CGS2 to consult accounting history the information supposed to be listed in screen CGS202 is replaced by questionmarks. This occurs with every operation reference. For the accounting testing process this isse is blocking, therefore high priorit279 Defect major always 10-29-07 17:02 29-10-07 MThiele MThiele immediate closed XML generator on TST not up to date The XSD generated by Thaler TST is not the same as Thaler PRG. As the XSD (constraints on XML) is based ont the XML form PRG, no Client Output can be generated. To resolve: update the XML-generator of TST with XML-generator of PRG265 Defect major always 10-23-07 16:38 23-10-07 schmitzp schmitzp normal closed accrual not payed after cancellation of renewable capitalized TD via TDD1 does not appear in P&LWhen deleting via TDD1 a Term deposit without payment of the accrued interest to the sight account Thaler ends the contract per value date (is start date) allthough the request was end date (is cancellation date) This is a default setting in thaler, for c260 Defect major always 10-23-07 09:22 23-10-07 vastenburgl donckersn high closed Contract Status went from 2100 to 1000 Yesterday i entered some fx contracts. I validated them and three went to status 2100. Contract: A7A30FXLVA000006 A7A30FXLVA000008 A7A30FXLVA00000A This morning the status isn't 4000 but 1000. What is the explenation for this?255 Defect major always 10-22-07 14:14 22-10-07 bruijnp bruijnp normal closed Error message deleting security account I n TST-environment for client +G6 I added a security account- I used an income and capital account that do not belong to this portfolio- I blocked it with code 28 (so nothing happened at the account). I then tried to delete it, but then I received an erro254 Defect major always 10-22-07 10:52 22-10-07 boelhouwerp boelhouwerp normal resolved I nterest position sheets and swaps; showing unexpected valuesMargin not included on PI sheets250 Defect major always 10-22-07 09:38 22-10-07 boelhouwerp boelhouwerp high resolved Creation of SWI FT messages SWAP contractsThaler does create a paper confirmation where a SWI FT MT360 is expected. A7A30I SPBO000001248 Defect major always 10-18-07 18:55 18-10-07 turabazd turabazd normal resolved BetOpd with wrong accountnumber in the HeaderBetOpd files created by Thaler have wrong I dB account number in the header!! Equens will not process the file with wrong account number. Example : Thaler : 01BETOPD04 0709279990501440 Should be : 01BETOPD04 0710090265999995247 Defect major always 10-18-07 18:45 18-10-07 turabazd donckersn high resolved Wrong Accounting of clieops in Thaler The Accounting of ClieOps within Thaler should be the same as the accounting of a manual transaction. I ncasso clieops are booked debit on clients account. That is wrong. The clients account must be credited after receiving the funds thru TapeH. Reference 244 Defect block always 10-18-07 09:02 18-10-07 turabazd turabazd urgent resolved I t is not possible to process TapeH files in ThalerI t is not possible to process the H file, because there is a problem with Autosys/file watcher. This problem must be solved asap.243 Defect major always 10-17-07 09:34 17-10-07 vastenburgl vastenburgl high resolved Adding a spot contract isn't possible While trying to add an spot contract, i get an error message 'this isn't a spot contract'. I t is not the firts time it happens, see both attached files.239 Defect major always 10-16-07 10:38 16-10-07 simulag donckersn normal closed inputing value date for internal trf is not possible!in menu NTY2 it proved not possible to input the value date!!229 Defect major always 10-02-07 12:33 2-10-07 romijne romijne high closed Created cliënts can't be used From this week I get the message "rootnumber does not exist" while the cliënt does exist. Untill last week I had no problems and I could use this cliënts. I have shown this problem to Nicolas Donckers and he told me to put this up in Mantis because in his222 Question major always 09-24-07 09:21 24-09-07 boelhouwerp boelhouwerp high closed PB module in relation with position sheets I dB position of the bank will be administrated for via diverse portfolio's books. Some of these portfolio's are a mixture of products (being treasury contracts and securities) could you explain how to administrate? Goal is to have one portfolio/book conta220 RFC??? major always 09-20-07 14:11 20-09-07 vastenburgl vastenburgl high resolved I nterpolate of MM contracts When adding an MM contract with a duration of 3,5 months Thaler picks up the 4 months rate. The rate should be interpolated between 3 and 4 months.216 RFC??? major always 09-17-07 10:49 17-09-07 vastenburgl veldhuizenj high resolved No negative margin allowed on FX contract While adding an FX contract with a non-bank as counterparty it isn't possible to have an exchange rate lower then the reference rate.192 Defect major always 09-05-07 14:23 5-09-07 bruijnp verbeeckp high closed I mpossible to validate limit on current accountsee attachment190 Defect major always 09-04-07 14:39 4-09-07 bruijnp bruijnp high resolved Step 32 in Delete account in Framework does not workWhen I tried to delete an account (bouwedeposit)via WFS2 it all works well till step 32. When I try to execute step 32, i get a message that is shown in the attachment.189 Defect major always 09-04-07 10:56 4-09-07 bruijnp verbeeckp normal closed When adding a specific condition impossible to fill in calculation ruleWhen adding a specific condition impossible to fill in calculation rule, because there's no drill down possible, so you don't know what kind of code you should use.187 Defect major have not tried 09-03-07 10:11 3-09-07 vastenburgl vastenburgl normal resolved Revaluation didn't took place The revaluation for I RS contract A6K30I SPRB000005 didn't took place.185 Question major always 08-29-07 13:44 29-08-07 boelhouwerp boelhouwerp normal closed Relations with regard to position sheet Can it be explained how relations are set up with regard to position sheets? I t seems now that when entering an order/securities transaction containing an implied fx transaction that the position does appear on position sheet OMS. I t is not clear to me wh181 Defect block sometimes 08-28-07 16:05 28-08-07 bruijnp bruijnp high resolved I mpossible to add a I nvestment credit I t is impossible to add an investment credit, since at screen I CX1073 I receive the following warning message (see attachment): Het veld is is verboden (not allowed). Bij Periodicity of interest.175 Defect major always 08-27-07 09:16 27-08-07 huiga verbeeckp normal closed No Bo lists after EOM After the EOM from friday there are only 3 Bo lists in the citrix environment.173 Defect major always 08-22-07 11:40 22-08-07 bruijnp bruijnp normal resolved I mpossible to enter ERP cost with early repayments (error message)I mpossible to enter ERP cost with early repayments (error message), received an error message.168 Defect major always 08-21-07 10:43 21-08-07 vastenburgl vastenburgl high closed Wrong rate with I CS contract Added an I CS contract with a bank as counterparty. Filled in the field borrowing currency, amount and the lender currency. Thaler doesn't automaticly calculates the lender amount. I s that correct? Secondly, The lender amount should be between the rate and165 Defect major always 08-21-07 10:22 21-08-07 vastenburgl vastenburgl high closed Contract stays at status 2001 After validating a I RS contract the status stays at 2001, technical error. A6K30I SPRB000000 A6K30I SPRB000001 A6K30I SPRB000002164 Defect major always 08-21-07 10:21 21-08-07 vastenburgl vastenburgl high resolved Can't add settlement instructions after validation dealWhen filling in the contract and fill in 'unk' by means of payment, its impossible to change it later into cpt (if its a non bank counterparty). Error message 'I SX10038' appears; no receipt instruction has been found.161 Defect major always 08-13-07 15:31 13-08-07 huiga huiga normal closed Dateformat not correct 38 of the 144 BOlists that are avalable at in the folder 2006-11-29 have two date formats. The execution date is 08/10/2007 and pdf creationdate is 10/08/2007160 Defect major always 08-13-07 10:43 13-08-07 huiga huiga normal closed CGL191 dubbel I n O:C&WBOlistststc32006-11-29 you find the report CGL191 dubbel. Only the "chain" is difference.159 Defect major always 08-13-07 10:39 13-08-07 huiga huiga normal closed FXL021 dubbel I n O:C&WBOlistsTSTC32006-11-29 you kan find FXL021 report dubbel. Only the PDF creationtime is difference.158 Defect major always 08-13-07 09:49 13-08-07 huiga huiga normal closed 72 BO list missing after EOD After the EOD from 29/11/2006 at 10/8/2007 there were 72 BOlists missing with CW priority must or should. See attachment for more info.157 Defect major always 08-13-07 09:45 13-08-07 huiga verbeeckp normal closed no Unix lists after EOD At the end of day from 29 nov 2006 at 10 aug 2007 were no unixlists found on O:C&WBOListststC3ReportsJ 3151 Defect major always 08-07-07 11:09 7-08-07 huiga huiga normal closed French in unix lists French in unix lists. See attachments for details135 Defect major always 08-01-07 15:46 1-08-07 huiga verbeeckp normal closed Layout and language problem BO lists There are a lot of list were the layout is nog correct or there is still french inside. See attachement for more information.134 Defect major always 07-30-07 11:19 30-07-07 bruijnp bruijnj urgent closed I mpossible to validate route sheet Tryinmg to close an account, entered a route sheet vua Framework, but it is impossible to validate route sheet. See attachement 1.12 seconds error melding133 Defect major always 07-26-07 13:20 26-07-07 bruijnp bruijnp urgent resolved I mpossible to delete limit of current accountSee document DEPO_RECO_VBKR_C03_T01_defect001 at T:Projecten20142_The Eden Project1 ProgrammaDeelprojectenTestenI DBTestresultatenMTFVT after 14 seconds. I tried to delete a limit of a current account. The following error message popped up: Error while cal130 Defect major always 07-23-07 11:46 23-07-07 boelhouwerp boelhouwerp urgent closed availability of an overview of journal postings per contract/positionWe do not have the possibility to have an accummulated/consolidated overview of accounting events (journalpostings) as per contract/position right now. This must be availalbe in order to be able to check accouning output per product per event.127 Defect major always 07-18-07 17:17 18-07-07 huiga huiga high closed 40% of the Unix lists is missing 40% of the Unix lists is missing. See attchment for details.123 Defect major always 07-17-07 13:57 17-07-07 huiga troisfontainen high closed Language of BO lists in not correct Some of the BO lists is the title in french and the report also. Other lists are in english but the title still french. Some of the reports have no content. See the attachment122 Defect major always 07-17-07 13:35 17-07-07 huiga huiga high closed Only 20% of the reports is available Only about 20$ of the BO reports is available. See attached documnet for details121 Defect major always 07-16-07 10:54 16-07-07 boelhouwerp boelhouwerp normal closed Reset date floating leg swap Not able to enter a reset date on the floating leg of a swap different from interest date120 Defect major always 07-16-07 10:51 16-07-07 boelhouwerp boelhouwerp high closed I ncorrect positions interest rate swaps I t seems that interest and nostro positions are not being updated properly118 Question major always 07-12-07 13:20 12-07-07 goolbergm goolbergm normal closed No cash divivdend philips We have tried to instruced a cash dividend for the share philips but the system does not generated this. Please look into this I nfo Bernoit Reported by Marco van de Goolberg117 Defect major always 07-11-07 15:37 11-07-07 boelhouwerp verbeeckp high closed see defect 109 missing a number of reports; reports not filled with data. See attached excel file for acccurate infomation. I n this file the Reports that are marked green where present in the Reports directory O:C&WBOListtstC32006-11-22. The other files are missing. Bes


Step 1: Data collection

totaal

plann

ing &

beh

eer

testv

oorb

ereid

ing

testo

ntwer

p

testu

itvoe

ring

incide

ntbe

heer

evalu

atie

leeglo

op

over

ig

week-1 480 95,5 162 25 46 30 9 93 0 0 0 0 0 20week-2 523 86,7 96 42 233 36,5 23 6 0 0 0 0 0 0week-3 489 90 21 40 248 40 6 42 0 0 0 0 0 2week-4 407 56,5 27 39 201 31 4 46 0 0 0 0 0 2week-5 121 14 16 8 64 9 0 1 0 0 0 0 0 9week-6 0 0 0 0 0 0 0 0 0 0 0 0 0 0

2020 343 322 154 792 147 42 188 0 0 0 0 0 33



Step 2: Determine Growth

formal tests (Laplace, MIL-HB, LRT,…)

time

cumulativenumber of

defects



Step 3: Initial Model Selection



Step 4: Estimate Models

parameters

cumulativenumber of

defectslikelihood

ML LS

time



Step 5: Validate Model

goodness-of-fit plots (TTT-plot, u-plot, QQ-plot, residual plots)goodness-of-fit tests (Kolmogorov, Zhao-Wang)

time

cumulativenumber of

defects

maximalabsolutedeviation



Step 6: Interpret Model

time

cumulativenumber of

defects

use confidence bounds to learn about precision



SREtool: goalsprovide support for software reliability analyses

enable to apply best practices from statistics

make available state-of-the-art statistical algorithms


SREtool: featuresavailable for Windows, Unix and Linux

import and export functions

data editor

wizard for initial model selection

consistency check on data


SREtool: implementationstatistical procedures written in R

R is open source, free statistics software

R is maintained by a core group of world-class experts in computational statistics

interface written in Java

combination of R and Java using tool kit from University of Augsburg (Prof. Unwin)


Sample data cumulatief aantal defects

0

20

40

60

80

100

120

140

160

180

200

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37

testinterval (dagen)

aan

tal d

efe

cts

basis gegevens t/m 8/11

basis gegevens t/m 16/11

geschat aantal defects Yamada S-shaped


Conclusionsstatistics can be a useful additional method to evaluate software reliability

a new tool will soon be available for software reliability analyses


Some of Refis clients and services

Reliability analysis

Quality metrics dashboard

Test and quality management

Process improvement


Merellaan 53722 AK Bilthoven

T +31 30 225 36 37F +31 30 225 36 49M +31 6 543 12 528

W www.refis.nlE [email protected]

Postbus 513 5600 MB Eindhoven T +31 40 247 48 75 F +31 40 244 24 89 W www.lime.tue.nlE [email protected]

http://www.refis.nl/

mailto:[email protected]

http://www.lime.tue.nl/

mailto:[email protected]

Documents

Software Reliability Engineering Tool Nederlandse Testdag 27 november 2008 Alessandro di Bucchianico, Lime – TU/e Ed Brandt, Refis VOF