Upload
henri-haarmans
View
639
Download
0
Tags:
Embed Size (px)
DESCRIPTION
I held this presentation during the Eurostar 2009 conference in Stockholm. It gives a good impression in how a simulation of a SIT helps you in determining defects during the design stage of your SOA landscape.
Citation preview
Bring requirements to life with a paper SIT
- Simulated System Integration Test -
3 december 2009 | Henri Haarmans / Arjan Steltenpool
2
Goal and program
GoalAfter the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape.
Program Company and IT-environment; Problems of a SOA-environment; Simulated System Integration Test; Back at the office ...
3
Our roles within Nationale-Nederlanden
Henri Haarmans >25 years experience in IT with
Nationale-Nederlanden; Several roles within IT; For Group Pension, e-Business,
Mortgages and Life Retail; Currently:
Lead Analyst for Life Retail;
Facilitator.
Arjan Steltenpool >10 years experience in IT with
Logica (CMG); Test Analyst, Test Manager,
Trainer; Logica is the preferred supplier for
testing at Nationale-Nederlanden.
4
Nationale-Nederlanden is part of the ING Group
IT Department
Employees: 510 Internal;
220 External from preferred suppliers;
IT change budget for software about € 85 M;
CMM-i level 3.
Nationale-Nederlanden
Employees: about 4250 internal; One of the biggest insurance
companies in the Netherlands; Two locations: The Hague and
Rotterdam; Turnover in 2008 about € 5 Bn; Product lines:
Life Retail;
Pension;
Property& Casualty;
Income;
Mortgages.
We use the Business Domain Model (BDM) for the representation of our organization.
5
The BDM contains all our business processes and each domain is a component with a particular business knowledge
1. Relationship Management
3. Performance Management & Corporate Support
14 Provider Management
141 Provider Selection
11 Customer (Intermediary and End-Client) Management
111 Customer Insight &
Segmentation
114 Prospecting & Campaign
Management
112 Customer Interaction
Policy
142 Sourcing & Contracting
34 Corporate Support
32 Performance Management
321 Customer & Channel
Performance
322 Financial Performance
342 Knowledge Mgt Prod/Claims
343 Communic.Intern / Ext.
344 Human Resources
345 Finance &Accounting
347 Architecture Mgt
IntermediarySales ForceEnd-client
Industry Software(e.g. VA
package, Rolls, RDW)
Internet
IntermediarySales ForceEnd-client
Printing at POS
Internet
346 ORM &Fraude
113 Customer InteractionExecution
143 Provider Performance
323 Operational Management
2. ServiceDeliveryChannels
341 Legal & Compliance
2. OutputDelivery
Channels
348 Quality Management
13 Sales & Distribution Management
131 Sales Force Support
133 Intermediary Agreement
132 Intermediary Support
31 Process Support
311 Workflow & Navigation
33 Accounts Receivable Payable
331 Collection & Disbursement
21 Input messages
211 Paper Data Capture
213 Voice
214 Web
212 Electronic
215 Email
22 Output messages
221 Publishing (Paper)
223 Voice
224 Web
222 Electronic
225 Email
8. Banking4. LifeRetail
6. Property&Casualty
7. Income5. Pensions
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards
4. Simulationcalculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards
4. Simulationcalculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Conditional: accommodation, infrastructure and security
12 Product Cockpit
121 Product Developmentmanagement
122 Product & Rules Building
312 Archiving
Product lifecycle Life Retail:
management and processing
Product lifecycle Property & Casualty:
management, processing and claim handling
Product lifecycle Income:
management and processing
Product lifecycle Pensions:
management and processing
Product lifecycle Mortgage:
management and processing
61. Relationship Management
3. Performance Management & Corporate Support
14 Provider Management
141 Provider Selection
11 Customer (Intermediary and End-Client) Management
111 Customer Insight &
Segmentation
114 Prospecting & Campaign
Management
112 Customer Interaction
Policy
142 Sourcing & Contracting
34 Corporate Support
32 Performance Management
321 Customer & Channel
Performance
322 Financial Performance
342 Knowledge Mgt Prod/Claims
343 Communic.Intern / Ext.
344 Human Resources
345 Finance &Accounting
347 Architecture Mgt
IntermediarySales ForceEnd-client
Industry Software(e.g. VA
package, Rolls, RDW)
Internet
IntermediarySales ForceEnd-client
Printing at POS
Internet
346 ORM &Fraude
113 Customer InteractionExecution
143 Provider Performance
323 Operational Management
2. ServiceDeliveryChannels
341 Legal & Compliance
2. OutputDelivery
Channels
348 Quality Management
13 Sales & Distribution Management
131 Sales Force Support
133 Intermediary Agreement
132 Intermediary Support
31 Process Support
311 Workflow & Navigation
33 Accounts Receivable Payable
331 Collection & Disbursement
21 Input messages
211 Paper Data Capture
213 Voice
214 Web
212 Electronic
215 Email
22 Output messages
221 Publishing (Paper)
223 Voice
224 Web
222 Electronic
225 Email
8. Banking4. LifeRetail
6. Property&Casualty
7. Income5. Pensions
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards
4. Simulationcalculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards
4. Simulationcalculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Change Cockpit
1. Productspecifications
2. Textsfor quotations
& policies
3. Establishing standards 4. Simulation
calculation
7. Monitoring
8. Determining actual results
9. EV calculations
10. Adjustmentor removal
5. Profittesting
6. Productimplementation
Conditional: accommodation, infrastructure and security
12 Product Cockpit
121 Product Developmentmanagement
122 Product & Rules Building
312 Archiving
Product lifecycle Life Retail:
management and processing
Product lifecycle Property & Casualty:
management, processing and claim handling
Product lifecycle Income:
management and processing
Product lifecycle Pensions:
management and processing
Product lifecycle Mortgage:
management and processing
For example: a new product for Life Retail results in changes in all the 15 green domains
7
The BDM is implemented in a real Service Oriented Architecture (SOA)
Characterization: Distributed requirement and software development of the applications; Communications between domains goes via an Enterprise Service Bus
(ESB).
Complexity: We have various hardware platforms; We use various programming languages; We use in-house built software and vendor applications; Transition from the middle-ware in the ESB to Tibco.
Consequence of a SOA environment:
Defects in the connectivity between applications arise during the System Integration Test (SIT) for the first time.
8
Goal and program
GoalAfter the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape.
Program Company and IT-environment; Problems of a SOA-environment; Simulated System Integration Test; Back at the office ...
9
In a traditional environment the V-model is a good model for the development and verification of software
Paul E. Brook 1986
A Quality Management System with quality gates ( )and baseline audits provide extra certainty.
ExpectationWish
Customer Requirements
System Requirements
Functional and technical Design
Unit Built
Unit Test
System Test (Functional Acceptance Test)
User Acceptance Test
System Integration Test
Regression Test
Production AcceptanceTest
Test design
One team is developing all the
software!
Unit Design
10
A Quality Management System gives no extra certainty in a SOA environment
The big challenge is:How can we secure that all applications work together as one process to the customer?
ExpectationWish
Regression Test
Production Acceptance Test
System Requirements
per domain
User Acceptance Test
per domain
In a SOA environment one team develops the Customer Requirements and for each domain different teams develop the System Requirements and the software!
Our solution:
Test the process after developing the system and interface requirements.
System Integration Test
?Customer Requirements
11
Different stakeholders, different concerns
Our solution the Simulated SIT (SSIT) provides also answers to the questions from stakeholders.
Test Manager
Understanding of the application landscape;
Reduction in time to test and fix.
Business representative
Do the software developers
understand my requirements?
Software developer
Do I understand the Customer Requirements correctly?
Lead Analyst
Does my solution Work?
Project and Business Management
Certainty of due date and quality of the system.
12
Goal and program
GoalAfter the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape.
Program Company and IT-environment: Problems of a SOA-environment; Simulated System Integration Test:
Goal;
Preparation;
The test itself;
Roles;
Follow-up;
Benefits;
Back at the office ...
13
Main goal of the SSIT: Find defects in the process flow as soon as possible
A SSIT is: A simulation of the process between the main applications:
One analyst and one business representative per domain simulate the application;
Manually filling in Service Request Forms on paper;
Manually delivering these forms between domains to represent the IT-interfacing;
There is no face-to-face contact between the domains; Databases and screens are on paper; Defects and questions are filled in by the people who are performing the test.
We don’t test: The functions of an application.
Good preparation and facilitation is essential for a successful SSIT.
14
A good preparation is the basis for success
When should a SSIT be executed? After definition of the System and Interface Requirements, the Technical Application
Architecture and Technical Interface Architecture and the Business Use Cases; Before starting to build the applications.
Precondition of a SSIT: More than one Business Use Case or one complex Business Use Case; More than one application; More than one Business Domain.
Test cases made by the Lead Analyst and Test Analyst: Usually 15 - 20 test scenario’s are sufficient; A test scenario maps 1-1 with the Business Use Cases; For every Business Use Case more than one test case; Database information for each application.
The preparation for a SSIT depends on complexity, but shouldn’t be more than 40 hrs.
The Facilitator of the
SSIT plays the role of
the devil’s advocate.
15
The testing day in a nutshell (1 of 3)
Kick-off with all the participants in a central meeting: Project urgency presented by the Project Manager;
Rules of the road presented by the Facilitator;
Explanation of the working of the SSIT presented by the Facilitator.
16
The testing day in a nutshell (2 of 3)
4 test sessions during one day:
Testing is done by filling in Service Request Forms, the templates, logbook etc.;
After one hour, a session ends with an evaluation in every ‘break-out’.Service Request Form:
Between and
Output
Input
Purpose
Service
Testcasenr.:
Questionform
Testcase:
Application:
Name:
Question:
Answer:
Defect form
Testcase:
Application:
Name:
Defect:
Solution:
Action Holder:
17
The testing day in a nutshell (3 of 3)
A central meeting after every test session for discussing the defects, discussing the questions and optimizing the test process;
Between the test sessions: exercises for getting back the energy;
At the end of the day:
Conclusions of the test cases and actions for solving the defects;
Action for outstanding questions;
Evaluation of the day; Afterwards:
The facilitator makes a SSIT test report including defects and questions.
18
Visualization of the simulation
Table 6
Table 1
Table 7
Table 2
Table 5
Table 3
Business Rules
Back Office
Calculation Box
Collection & Disbursement
Front Office
Output
On every table…
Facilitator
Test Manager
Information AnalystLead Analyst
‘Knowledge bank’
19
The shopping list for the SSIT (1 of 2)
A room which can be divided in separate break-outs;
Magnetic white boards (not essential but very convenient);
A facilitator for: The process of the SSIT;
Exercises for bringing back the energy;
Interim evaluations for: Solving problems;
Evaluation of the test scenario’s during the
day.
20
The shopping list for the SSIT (2 of 2)
For every business domain: A break-out;
An in-box and an out-box;
Instructions for the SSIT;
Templates for: Reporting defects;
Questions;
The evaluation;
Screen prints for the online applications;
A log book;
Database content;
Service request forms to other applications.
21
All participants play their own role during the SSIT
Lead AnalystWalking encyclopedia
Business PeopleMonitors that the right business rules are used.
Information Analyst
Processor of the Service Request Forms.
FacilitatorControls the process, initiates a test case, facilitates the evaluation etc.
Test ManagerPostman between the domains
22
Beware: Follow up after a SSIT is crucial
Detailed review on field-level of the services between two applications;
Defects have to be solved; Questions have to be
answered; Test scenarios can be input
for the real SIT; Baseline for working
procedure.
23
Some results from SSIT executed in the last year
A new product for Life Retail: 20 Defects during the SSIT:
6 Missing and wrong services in the TIA;
3 Defects in data model;
3 Defects in the manual processing;
2 misinterpretations of the CR;
6 Defects in the services;
Expected delay when not found: > 3 months;
Online Transaction Module: 17 Defects during the SSIT:
3 Missing services in TIA;
7 Defects in the control function;
1 Defect in the data of a service;
4 Defects in business rules;
2 Defects in the manual processing;
Expected delay when not found: > 6 weeks.
24
Benefits: One day hard work during a SSIT results in answers to the concerns of the stakeholders, and also in:
Detecting logical errors in the process: Interfaces; Services; Order of services.
Detecting differences between the data models in different domains; Participants:
Get the same view of the objective of the project; Get understanding of the importance of the project; Knowing each other promotes efficient communication during testing.
Business: Check on the right process; Information about and understanding of the complexity of the test; Input for the Knowledge Bank and guidelines.
SIT: Test cases of the SSIT are the base for test cases in the SIT. Quality of the SIT becomes higher.
Test Manager: Gets a better understanding of the process to test.
Less or no delay in the delivery of the software.
Defects can be discussed immediately
25
Goal and program
GoalAfter the presentation you should understand how the Simulated SIT helps you in determining defects in the design stage of your SOA landscape.
Program Company and IT-environment; Problems of a SOA-environment; Simulated System Integration Test; Back at the office ...
26
Back at the office …
Do you recognize the problem? Painful SITs due to lack of communication between business domains in previous project phases?
Convince project/program management for the necessity of a SSIT; Keep in mind:
The Information Analyst is responsible for all requirements, review activities including the SSIT, but a Test Manager can help facilitate the SSIT;
So …
Thank you for your [email protected]