Embed Size (px)
Citation preview
SOA Testing – SISE Approach & Scelta Test Automation
Framework
Sathyan S NairSenior Technical Analyst
December 13,2007
Agenda
Needs & Solutions
Focus Areas & Challenges
SOA Lifecycle & SISE Approach
Scelta – The Automation Framework
Key – Benefits
Future Research
Open Discussion
Needs & Solutions
Business & Testing Needs
Business Needs► Evolve IT Solutions
► Avoid costly integration and re-engineering efforts
Testing Needs► How do we validate services for functional
requirements?► How do we manage the complexity of testing –
Integration of multiple services?► How do we validate shared services across
multiple applications, as services change and evolve?
Address the Needs
How to address the needs?► Various stages of testing - Required to address
function as well as interaction complexities► Testers to wear additional hats and follow an
evolved approach► Automation Framework for multiple test
execution cycles
Our Solution
Focus Areas & Challenges
How different is SOA Testing
Head-fewer services, without a UI
Services spread across different boundaries
Look beyond FT & PT – Integration, System & Workflow Testing
Requirements change impact – Across multiple application boundaries
Need for multiple end-to-end business process/workflow testing
Addressing Challenges
SOA Lifecycle – SISE Approach
SOA Lifecycle – SISE Approach
SISE Approach – Service/Component Testing What we do?
► Validation - Light-weight service Implementation
► Scripts Development - Assertions and Boundary Conditions
► “Good known data” based validation technique
http://www.webservicex.net/CreditCard.asmx?WSDL
Card Type – VISACard # - 7767545687679876
This Credit Card number is not valid.
Test Results
► Ensure the proper functioning of individual web-methods/services
SISE Approach – Integration Testing
What we do?
► Validation – Communication between different systems
► Scripts Development – Messaging formats (namespace, SOAP compliance etc), Communication policies (parameters, return type etc), Security policies (Authentication, WS-Security etc)
► “Good known data” based validation technique
Integration Testing
Service C
Output
Output
Output
Service A
Service B
Results A and B Input
Input
Input
Test Results
► Different types of services across multiple systems seamlessly integrate and communicate with each other
SISE Approach – System Testing
What we do?
► Validation – Communication between different systems
► Scripts Development – Messaging formats (namespace, SOAP compliance etc), Communication policies (parameters, return type etc), Security policies (Authentication, WS-Security etc)
► Mock services to handle – Asynchronous Messaging communications
► Message Checkpoints – Route messages to appropriate services
► Performance Monitors – Study the response times , uptimes etc
► Helps the team with known limitations, re-engineer and hardware recommendations
Test Results
► Different types of services across multiple systems seamlessly integrate and communicate with each other, under production env
► Overall transactions response time calculation
SISE Approach – End-To-End Testing
What we do?
► Validation – Complete business cases/workflow
► Different types of business systems - Types of protocols include HTTP, HTTPS, SMTP, SOAP, XML-RPC, IRC etc.
► Mock services , Messages checkpoints & performance monitors deployed for appropriate usage
► Service change undergoes a complete end-to-end testing cycle
► Study of response time delays and key performance/response issues
Test Results
► Complete business workflow completes without any errors
► System satisfies the requirements
Scelta – The Automation Framework
Test Automation Framework
Scelta – Input Data & Driver Components
Input Store
► XML driven Test Cases
► Business and technical users can create test cases
► Elements and schema guidelines for file creation
► Configuration file for implementation information
Workflow Manager
► Test execution - User choice (Business/Integration/security cases etc)
► Priority for test workflow/scenario/case
► Test execution – Environment configuration
Policy Manager
► Assign appropriate policy – Test cases
Scelta – Service Validation Components
Service Validation
► Validate individual services/web-methods
► Assertions for different test-conditions
► Xpath query to fetch information
Response Manager
► Manages request and directs response
► Invokes mock services - For transactions with response delay time
► Message Routers – To deliver responses appropriately
Logs & Reports
► Centralized logs and reports collector
► Customized reports based on customer requirements
Scelta – Performance Monitors & Metadata
Distributed Monitoring
► Service level monitors and agents
► Customized agent for collecting transactional information
Load Manager
► Workflow driven load testing
► Configurable load – Monitor performance of service
► Run periodically - Ensure Web service stability
Service Metadata
► Collects Historical information - Measure service reliability and quality
► Study service behavior (up/down time, time since last failure, service reliability history etc)
Key Benefits – Approach & Framework
Multi-level validation–Defects fixes-Minimal effort & cost
Test Coverage - Different requirements validation
Minimal Defect leakage in production
Asynchronous test script execution
Service response facts , reliability and quality reports
Re-usable across ‘n’ # of test cycles
Future Research
Support security related aspects
Finding services – Addressing testing pain-points
Open Discussion
