Embed Size (px)
Citation preview
Mercury SOA Testing: Products Overview and Demo
James Eich, Senior Systems Engineer
2Confidential & Proprietary
Testing challenges with SOA
Mercury SOA testing solution
Agenda
Functionality walkthrough
Summary
3Confidential & Proprietary
Mercury Addresses Key Challenges
• Validate functional quality• Optimize performance and ensure services will scale in production• Manage the complexity of testing multiple services and provide traceability
and impact analysis
SOATesting
• Increase visibility into service levels and business impact of outages• Improve problem detection, notification and mean time to repair• Control the risk of changes in web services, SOA components
and underlying infrastructure
SOA Management
• Provide complete visibility for discovering and understanding services• Ensure predictability and transparency between consumers and providers• Control the lifecycle of services and effectively manage change
SOA Governance
4Confidential & Proprietary
Increased Complexity
Small Number of Services Increases the SOA Testing Complexity
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service3
Operation 1
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service8
Operation 1
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service4
Operation 1
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service1
Operation 1
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service2
Operation 1
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service5
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service6
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service7
Operation 1
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service9
Operation 1
1
2
3
4
5
6
7
8
9
10
Operation 10
1
2
3
4
5
6
7
8
9
10
Operation 9
1
2
3
4
5
6
7
8
9
10
Operation 8
1
2
3
4
5
6
7
8
9
10
Operation 6
1
2
3
4
5
6
7
8
9
10
Operation 7
1
2
3
4
5
6
7
8
9
10
Operation 5
1
2
3
4
5
6
7
8
9
10
Operation 4
1
2
3
4
5
6
7
8
9
10
Operation 3
1
2
3
4
5
6
7
8
9
10
Operation 2
1
2
3
4
5
6
7
8
9
10Service10
Operation 1
1
2
3
4
5
6
7
8
9
10
10 services x 10 web operations x 10 test data points = 1,000 test permutations
What will happen when changes are introduced?What will be the number of test permutations?
How will the users know what to test?
5Confidential & Proprietary
SOA Quality Challenges
SOA Quality Management
Huge amount of services and systemsTest services stand-alone and as shared entitiesInitial and regression testingStructure of quality processVisibility into progress and quality status
SOA Testing
Holistic coverage of all service aspectsGUI-less testingComplex service orchestrationsMultiple transports and protocols supportTesting incomplete environmentsConstantly emerging technologies
SOA Ecosystem Integration
SOA governance systems– Service assets– Change– Policy management
SOA runtime and execution engines– Toolkits– Application servers and ESBs
Diagnostics and monitors
6Confidential & Proprietary
SOA TestingSOA Ecosystem Integration
Mercury’s SOA Testing SolutionSOA Quality Management
System description (architecture, topology, business processes & data)SOA aspects to test and methodologyRequirements coverage and traceabilityManage application change lifecycle Integrate with Mercury Dashboard
Service Test Management module for QC
Integration with Quality CenterMultiple testing aspects in a single tool:
– Functional (positive and negative/boundary)– Load and performance– Interoperability– Standards compliance– Security
SOAP, JMS and many other protocols supportTrue Java (Axis) and .NET toolkits usageAsync testing and WS-Addressing supportClient and server-side recording Service emulation
SOAmetadata SOA
management
Mercury Service Test / LoadRunner
Connectivity to UDDI v2-3 and Systinet’s registries/repositoriesCertification and connectivity with leading Application Servers and ESBsMercury monitoring and diagnostics Mercury Business Availability Center support for SOA
SOA Ecosystem Integration
Runtime information
7Confidential & Proprietary
ServiceTest
LoadRunner
WebServices
Component
UI
Web Services
API
FINANCE APPLICATION
Mercury’s Integrated SOA Testing
Web Services
EIS
RMI APICustom
App
Legacy API
MainframeLogic
SOA Abstraction Layer
Enterprise Services
LEGACY C/S ERP/CRM WEB
Web Services
Third party
ServiceTest
LoadRunner
API3
SOA SharedServices4
QuickTest Prowith
Web Svcs Add-in
LoadRunner
UI5Quality Centerwith
Service TestManagement
Test Management2
Systinet SOA Governance 101011001
Functionality Walkthrough
9Confidential & Proprietary
Mercury Service Test™(page I)
Fully integrated with VUGen and Mercury LoadRunner®
Single tool for all SOA testing needs – Functional testing– Load testing
Foundations– SOAP over HTTP and JMS transports– Extensive coverage
- True Java (Axis) and .NET toolkit usage/support- Web, databases and legacy systems protocols- Microsoft, IBM, SAP, Oracle and other platforms certifications
– Web service call configuration- Web services security (HTTPS, XML Digital Signature, XML-Encryption, X.509,
Kerberos, SAML)- Attachments (MIME, DIME)
– Request/response views
10Confidential & Proprietary
Service TestFoundations
11Confidential & Proprietary
Service TestFoundations
12Confidential & Proprietary
Mercury Service Test (page II)
Fully integrated with VUGen and Mercury LoadRunner
Services managementTests generation wizard
– Positive testing– Boundary testing– Standards compliance testing– Interoperability testing– Security/penetration testing
XML checkpointsTests pass/fail reportsXML parameterization
13Confidential & Proprietary
Service TestServices Management
14Confidential & Proprietary
Service TestTests Generation Wizard
15Confidential & Proprietary
Service TestXML Checkpoints
16Confidential & Proprietary
Service TestPass/Fail Reports
17Confidential & Proprietary
Service TestXML Parameterization
18Confidential & Proprietary
Mercury Service Test (page III)
Fully integrated with VUGen and Mercury LoadRunner
Integration with Mercury Quality Center– Script repository – Service sharing – Test execution – Test results viewing– Configurable runtime settings and data for each run
Script conversion QuickTest Professional Mercury Service Test
19Confidential & Proprietary
Service TestService Test Quality Center Integration
20Confidential & Proprietary
Mercury Service Test (page IV)
Fully integrated with VUGen and Mercury LoadRunner
Advanced standards compliance testing – Run WS-I compliance reports– Analysis of WSDLs, XMLs and SOAP messages
General purpose asynchronous testing platform– WS-Addressing support
Client and server-side recording
21Confidential & Proprietary
Service TestAdvanced Standards Compliance Testing
22Confidential & Proprietary
Service TestAdvanced Standards Compliance Testing
23Confidential & Proprietary
Service TestClient and Server-Side Recording
24Confidential & Proprietary
Service TestClient and Server-Side Recording
25Confidential & Proprietary
Mercury Service Test (page V)
Fully integrated with VUGen and LoadRunner
Service emulation – Provides
- Early time to test- QA incomplete environments- Isolate problems
– Automatic creation and hosting of web service stubs- Based on WSDLs- Based on recorded web service traffic
– Define default output and response times– Rule-based model to match service inputs outputs
26Confidential & Proprietary
Service TestService Emulation
27Confidential & Proprietary
Service TestService Emulation
28Confidential & Proprietary
Service TestService Emulation
29Confidential & Proprietary
Mercury Service Test Management™A complete SOA test management solution built on top of Mercury Quality Center’s proven foundations
Mercury Quality Center™Dashboard
TestDirectorRequirementsManagement Test Plan Test Lab Defect
Management
Functional TestingQuickTest
Professional WinRunner
Business Process Testing
Accelerators for SAP, Oracle, Security
Delivery Options
Mercury Managed Services In-house Deployment Combination
Shared Data Repository Central Administration Workflows Open APIs
Foundation
Optional ExtensionsSAP, SOA
30Confidential & Proprietary
Service Test ManagementFoundations
31Confidential & Proprietary
Mercury Service Test Management (page II)
SOA environment visibility– Define services
- Existing services import- Connectivity with UDDI and Systinet registries/repositories- Manual services definition
– Service detail views- Ownerships- Description- Technical data including operations
Fully integrated with Mercury Quality Center
32Confidential & Proprietary
Service Test ManagementSOA Environment Visibility
33Confidential & Proprietary
Service Test ManagementDefine Services
34Confidential & Proprietary
Service Test ManagementService Detail Views
35Confidential & Proprietary
Mercury Service Test Management (page III)
Group services in various waysLink services and service groups
– Requirements– Defects – Tests
Fully integrated with Mercury Quality Center
36Confidential & Proprietary
Service Test ManagementGroup Services in Various Ways
37Confidential & Proprietary
Service Test ManagementLink Services and Service Groups
38Confidential & Proprietary
Mercury Service Test Management (page IV)
Testing methodology environment requirements testing assets coverage
Change impact testing– Automatic update for services change– Modification of requirements based on changes– Create updated test assets to validate services change
Integration with Service TestReports and dashboard integration
Fully integrated with Quality Center
39Confidential & Proprietary
Service Test ManagementTesting Methodology
40Confidential & Proprietary
Service Test ManagementChange Impact Testing
Mercury LoadRunnerfor SOA Testing
42Confidential & Proprietary
Performance Testing Monitors(Lifecycle)
SiteScope SOA capabilities:A) Web services monitor
– Builds a synthetic web service from the actual WSDL file used by the web service
– The synthetic web service initiates an SOAP request and monitors the response
– Supports web service providers that use Basic, Digest and NTLM authentication
– Measures availability, response time and these states
B) Web services infrastructure monitors– App servers: WebLogic, WebSphere, Oracle– MOM: MQ Series– Enterprise apps: Siebel, SAP
OK unknown host name unable to reach server unable to connect to server timed out reading content match error document moved unauthorized forbidden not found proxy authentication required server error not implemented server busy
SiteScope Web ServiceMonitor State Values
43Confidential & Proprietary
Web Services Diagnostics Dashboard
Mercury QuickTest Professional for Web Services Testing
45Confidential & Proprietary
Mercury QuickTest ProfessionalWeb Services Testing Combine UI and WS testing :
– WS check pointWSDL import XML checkpointWeb services over HTTP
46Confidential & Proprietary
Summary
1. SOA quality is more than just testing the service !
2. Mercury has the only comprehensive solution in the market that will:
– Test the service with Mercury Service Test and LoadRunner
– Manage SOA testing quality with Mercury Service Test Management
– Integrate with the SOA ecosystem with Mercury SOA lifecycle
3. SOA testing solutions are based on the Mercury products leveraging existing assets and knowledge
