Upload
techwellpresentations
View
54
Download
0
Tags:
Embed Size (px)
Citation preview
- 1 -
Automating End To End Scenarios
Presenters:Sandra Alequin (Senior Manager, Allstate Insurance)
Monika Mehrotra (Senior Manager, lnfosys Ltd)
- 2 -
Enterprise testing (ETest) strategy was established in pursuit of a strategic objective: to improve agency and customer experience with a goal of minimizing production problems and reducing the risk due to major application releases on critical business functions.
ETest strategy implementation included establishing new processes, tools, team and environment. ETest consists of 200+ applications with over 600+ backend services and equally high number of interacting databases.
33 key business functions, which are highly integrated, are tested end-to-end prior to being put in production under this strategy.
Enterprise Testing (ETest) Strategy at Allstate
- 3 -
Executive Summary
Java Applications
.NET Applications
Siebel Applications
Mobile Applications
Open Web Applications
Mainframe
Presentation Layer Middle-Tier Data Layer
Mainframe
Web Servers
Application Servers
LDAP
MQ
TIBCO
SOAP
REST
JMS
EJBs
JDBC
Store Procedure
FTP
Mobile
Web Browsers
Applications
Enterprise Testing Environment – An Overview
Dat
abas
eD
atab
ase
Dat
abas
e
- 4 -
Over 200+ application from a mix of legacy (AS400, Mainframes) and new technologies (.NET, Java)
600+ services on different technologies like SOAP, REST etc
Unresponsive Backend Services can cause failures in tests and hence need constant pinging to keep them alive
Time consuming Defect Root Cause Analysis due to large number of systems and integrations
Deactivation of Test IDs post Data Refresh activities
Complexities in Enterprise Testing Environment
- 5 -
Validation points introduced across different layers for deeper and wider test coverage
5
Executive Summary
Java Applications
.NET Applications
Siebel Applications
Mobile Applications
Open Web Applications
Mainframe
Dat
abas
e
Presentation Layer Middle-Tier Data Layer
Mainframe
Web Servers
Application Servers
LDAP
MQ
TIBCO
SOAP
REST
JMS
EJBs
JDBC
Store Procedure
FTP
Mobile
Web Browsers
Applications
Existing validation points Newly introduced validation points
Solution to address E-Test Environment challenges
Dat
abas
eD
atab
ase
- 6 -
Executive Summary
• Implementation of trouble shooting at service layer to reduce effort spent root cause analysis
• Reduction in time spent on debugging and enable faster fixes for defects
• Validation of middle tier components that otherwise are not tested as part of GUI testing scenarios
• Deeper and wider coverage introduced into architecture with middle tier validation
• Validated integration and interaction points ensuring quality in systems integration
• Data navigating from one end to another in network is tracked for quality and reliability
Smoother Integration Better Coverage Defect Isolationa b c
Users - Presentation layerHeadless clients
consuming services
Data Access Logic Data Utilities Data Sources
Presentation – Middle Tier Integration
Middle Tier – Data Layer Integration
Service Service Service Service Service Service Serviceb
a
a
c
c a
b
Addition of Middle Tier Validation in E-Test
- 7 -
Complexity, size and time to execute E-Test regression suite warranted the need to automate the regression suite
An assessment on then E-Test processes were performed and an Integrated (frontend and backend) automation solution was arrived at to target improvement in efficiency and effectiveness of E-Test regression suite execution by giving deeper and wider test coverage.
De-risked and cost effective 2-Phased approach was adopted to build out the integrated automation.
Phase 1 was performed for feasibility analysis and devise solution for automation requirements.
Phase 2 was implementing the solution and rollout
Necessity for Integrated Test Automation Solution
- 8 -
QTP Automation Framework
Service Validation Framework
Frontend Layer
UI Components
Presentation Layer Logic
Backend Service Layer
Service Orchestration
Services
Backend Data Layer
Data Access Components
Data utilities
Data Validation using CA LISA
E Test Environment
Leverage LISA for early validation at service/data layer
Validating UI layer on a more stable
Backend
Integrated Automation Solution
HP Quality Center
HP Quick Test Professional
CA LISA
CA LISA
Backend (Service and Data layer) validation will be automated with CA LISA and Frontend (UI layer) validation will be automated with HP QTP. The solution will be orchestrated using HP QC and provides flexibility to validate layers in integrated or silo mode.
- 9 -
Switch of test machine is required to continue execution
Stop for Batch CycleResume after Batch Cycle
Data needs to be captured and transferred between the test scripts/test sets
LISA Scripts integration with QTP and HPQC for backend scripts execution
Typical ETest End-to-End Scenario for Automation? – Questions to be answered and solutions to be proven out
Customer does inquiry and requests for a Quote
Mobile Search Web Search
Agent contacts customer and the quote is converted to a request for new business
New business is received and reviewed by underwritersHome Office Staff verifications Underwriters
Underwriter provides approved , New business documents generated and Customer access documents
Document Management System
Customer Access Endorsement confirmationPolicy EndorsementRegistration
Renewal Validations
Validate Renewals processing after payments
Policy Endorsements, Renewals and billing
Endorsement Documents
Renewal of Policies
Renewal Billing
Renewal Documents:
Endorsement processing Renewals Payments
Sample ETest Scenario
??
?
?
Exclusive Agent Applications
Independent Agent Applications Home Office Staff
- 10 -
Tests were run on specific workstations to mimic customer and user experience. So how to switch test machines to continue automated execution?
How to handle the execution of a single test case across multiple job batches?
How would the data required for execution across batch of same test case maintained throughout the execution of the case, which may span across days?
How to integrate HPQC (Test Management), HPQTP (UI Automation) and CA LISA (Service layer validation) tools to perform an integrated test run?
Automation Challenges in ETest End-to-End Scenarios
- 11 -
Test Automation challenges addressed
Data Extraction & Persistence
Multiple Machines Execution
Execution Inline with Daily Batch cycles
LISA Scripts integration with QTP and HPQC
1 2 3 4
Customer does inquiry and requests for a Quote
Mobile Search Web Search
Agent contacts customer and the quote is converted to a request for new business
New business is received and reviewed by underwritersHome Office Staff verifications Underwriters
Underwriter provides approved , New business documents generated and Customer access documents
Document Management System
Customer Access Endorsement confirmationPolicy EndorsementRegistration
Renewal Validations
Validate Renewals processing after payments
Policy Endorsements, Renewals and billing
Endorsement Documents
Renewal of Policies
Renewal Billing
Renewal Documents:
Endorsement processing Renewals Payments
Sample ETest Scenario
21
3
4
Exclusive Agent Applications
Independent Agent Applications Home Office Staff
- 12 -12
# 1-Data Extraction & Persistence Data Persistence in XML Files
Dat
a Pe
rsis
tenc
e a
cros
s m
achi
nes
Dat
a Pe
rsis
tenc
e
acro
ss b
atch
runs
Customer Inquiry, Quote & New Business
Customer Machine: Customer Inquiry and
Begins Property Quote: (Mobile and Web)
Agent Machine: New Business
Purchase
Under Writer Machine; New
Business
Agent Machine: Endorsement : Agent
Customer Machine: Customer: Login to
“Customer Account", Register
Batch 1: Customer Inquiry and Begins Property Quote: (Web)
Batch 2: New Business Purchase: Agent
Batch 3: New Business : Agent
Data is getting read by different machines from XML
Reference # is persisted in XML and used by script for Policy # generation
1
1Policy # is persisted used by script and later used other scripts2
2
1 2
•Data is persisted in Shared LAN drive which is accessible by all machines•Data is persisted in form of XMLs for each test case.•XPATH Queries are employed for faster data retrieval
- 13 -13
Test Execution Dependency- Across Batches
Customer Inquiry, Quote & New BusinessCustomer Agent Search :
(Mobile and Web) Customer creates Quote
Customer Inquiry, Quote & New Business
Manage Agency by Agent Purchases quote and email validation
Validation of New Business
Batch 2
• Automated Script generates Execution Control XML for each Test Case• Test Script execution status with other details are captured in XML structure • Execution sets Test Set_Status “Y” in case of Pass “N” for failure• Next Test Script reads XML data and determines if execution is required or not
Invoke Next Batch test case if
Status=“Y”
Stop Execution for test case if
Status=“N”STOP
Determine Execution is required/not from
XML
Batch 1
# 2 - Execution Inline with Daily Batch cyclesCross Test Set Dependency with XMLs
- 14 -14
1A
1B
1C
E Test Release 1
HP QC Test Lab Folders at Scenario level
HP QC Test Lab Folder at Release level
Batch1
Batch2
BatchN…
Batch1
Batch2
BatchN…
Batch1
Batch2
BatchN…
HP QC Test Sets at Batch run levels
Each Test Set contains specific test scripts to be run as part of E Test Case
HPQC identifies available machine in the host group and triggers execution on the available machine within the group. Upon execution completion, test results are reported back to quality center and execution is moved to next test script
ACT MBTIVW UWR
Claims
Machine Groups
HP QC Test Cases
Batch 1 for 1A contains scripts to be executed before batch cycle process runs
#3 - Multiple Machines ExecutionHosting in HPQC
- 15 -15
# 4 - CA LISA integration with HPQTP & HPQC
CA LISA
- 16 -16
HP Quality Center
Mainframe Web Java
Enterprise Automation Components(Reusable Functions, Keyword functions, Framework Driver Engine)
Platforms Under Test
Automation Framework
Remote Machines
Testers
Java Specific Components
Web Specific Components
Mainframe Specific Components
Framework to handle Application Diversity
- 17 -
Test Case Modeling
17
Application Workflow
Payment
Work flow Data
Data1
DataWork flow
Work flow
Data
Data2
Data3
Data1
Data2
Data3
Data1
Data2
Data3
Modularizing the application workflow by creating multiple reusable Business Process Components (BPCs)
Workflow created in excel using action keywords
Multiple data sets can be mapped to the same workflow to easily create different data oriented test cases
Customer Registration Agent Search
Diagrammatic representation of test case modeling
Test Case modeling with the framework
- 18 -
Test Case Modeling
18
ETest Implementation Approach
Assessment of ETest
Environment and Test
Scenarios
ETest Automation Feasibility Analysis
ETest SOA Validation
Implementation
ETest SOA validation
rollout
ETest UI Automation Feasibility Analysis
ETest Automation
Unique requirements
identifiedProof Of
Concept for ETest
Automation
Complete Automation
Implementation in ETest
Rollout of Automation Testing for
ETest
- 19 -
Executive Summary
Current State Post Implementation Benefits
Quality
• Test steps scoped based on available effort
• End to End Business scenario validated in base state
• Volume test steps execution was increased by 3 times
• End to End Business scenario implemented in multiple states
• 200% improvement across Tests and ROCs
• Improved coverage at backend layer• Increased and consistent validation• Multiple Regression Runs in current
window
Time• 3-week window
to execute E2E tests
• 1-week window to execute E2E tests
• 33% reduction in the execution window
• Creating window for more execution• Reduced turn around time on defect
resolution
Cost• High manual
effort to rerun failed scenarios
• Overall team is reduced by 40%
• 30% reduction in execution cost• Early validation• Reduced issues and effort during End
to End• Lessen manual effort with automation
ETest Automation - Post Implementation Analysis
- 20 -
Executive SummaryAdditional Benefits of Integrated Automation Solution
Predictability Validation at backend layer resulting in early defect detection Post implementation UI execution happens on a validated backend
layer
Portability Solution can be deployed on E-Test or other environments with minimal effort
Coverage Improve coverage by introducing more validations at all layers of validation
Modularity & Scalability
Deploy solution in packages to enable selective run Ability to run automation scripts for specific scenario category
and/or applications.
- 21 -
Questions & Answers
- 22 -
Thank you.