Upload
nguyentu
View
223
Download
0
Embed Size (px)
Citation preview
© CGI Group Inc.
iT-Toolbox Testing Module Automated tests for inter-application data flows
September 2017
Overview
2
Problems and requirements 1
Solution 2
Concepts 3
Usage 4
Roadmap 5
Problems and requirements
What test-related difficulties are encountered on exchange platforms?
Flawless quality for critical data exchanges
• Major business impact in case of error (billing, logistics, …)
• Conformance / regression tests required before deployment
Complexity when testing exchanges
• Manual execution is slow, risky, costly
• Deep understanding of the platform and flows is required
Multiple and heterogenous technologies
• Transport and data formats, ESB, application-specific, …
• Dependency with an ever-changing application environment
Lack of tools dedicated to data flow test automation
3
Problems and requirements
What are the main tooling requirements?
Ensuring the absence of regression for all kinds of data flows
• During technical migrations (e.g. new exchange platform)
• During application upgrades impacting data flows
• During framework updates
• During canonical data format updates
Building a realistic test case repository
• Centralized storage for test cases (avoiding mail exchanges, local files, …)
• Keeping test cases up-to-date
Automating tests in a controlled environment
• Testing often, easily, quickly to increase quality and reactivity
• Independence from external systems when running exchange tests
• Reducing the cost of tests (configuration and execution)
4
Solution
iT-Toolbox Testing Module: a tool to automate data flow tests
Black-box testing
• Injection of input data to launch the data flow
• Collection of actual data from the output
Regression checks
• Comparison of actual and expected data
• Support for data format specifics
• Precise problem identification (line, field name, difference, …)
• Report generation
Web interface to define and execute tests with a REST API
Based on popular open-source solutions (Maven, JUnit, …) and compatible with continuous integration servers (Jenkins, …)
5
Concepts – Overview
Hierarchical test repository
• Test project: container for test campaigns (e.g. business process)
• Test campaign: container for test suites (e.g. data exchange)
• Test suite: container for test cases (e.g. flow or half-flow)
• Test case: comparison of actual/expected data (e.g. nominal, errors)
Connectors concept
• Launch flows by injecting data and collecting output
• Many transports supported (FTP, HTTP, SOAP, JMS, JDBC, etc.)
Formats concept
• Define how actual/expected data comparison is executed
• Individual fields can be named or ignored during comparison
• Many formats supported (delimited/fixed flat file, XML, JSON, etc.)
6
Concepts – Hierarchical repository
3 levels of configuration: project, campaign and test suite
• Modular configuration of connectors / formats
• Parameters can be inherited for increased reuse
• Parameters can reference properties (key/values)
7
Project Campaign
Test suite
Connectors / Formats / Properties
Connectors / Formats / Properties
Connectors / Formats / Properties
Test case Test case Test case
Concepts – Test cases
Test case execution steps:
1. Setting up the environment
2. Reading input data
3. Launching the flow
4. Collecting output data
5. Comparing actual / expected data
6. Generating difference report
7. Cleaning up the environment
Test types:
• Comparison (default)
• Error (message / exception)
• Performance (time limit)
8
Test case
Pre-action
Actual data Expected data
Format
Input data
Post-action
WS
wM
JMS
FTP
BDD Connectors
Differences
Usage – Overview
9
Definition
• Configuration of connectors and formats
• Creation of test campaigns, suites and cases
Execution
• Criteria-based definition of the test perimeter
• Launching tests
Reports
• Results analysis
• Relevant errors
• History of previous results
Usage – Definition
Web interface for test definition
• Project / campaign / suite / test case hierarchy
• Properties / connectors / formats configuration
• Test data management (upload / view / download)
10
Usage – Execution
Execution from the test repository web interface
• Selection of campaigns / suites / tests by criteria
• On-demand execution without leaving the interface
• Reports generation and history
Execution by a continuous integration server (e.g. Jenkins)
• Project downloaded and executed by CI server
• Reports generation and history
• On-demand, scheduled or triggered execution
• Notification (e.g. by mail) when a test failure occurs
Execution in standalone mode
• For use by developers / testers
• Local execution by script
• Reports generation
• Native integration with IDE such as Eclipse
11
Usage – Reports
Identification of relevant errors
• Name and path of the difference
• Detection of moved blocks between actual / expected data
• Option to ignore extra data or specific fields
12
Roadmap
Development of new features
• Productivity and user-friendliness improvements
• Fine-grained permission management in the web interface
• Easy search and browsing of test case data
• Deeper integration with Jenkins CI server
• New connectors and comparators according to user needs
Integration of a capture / mock module
• Creation of tests cases from captures
• Activation of mocks before executing tests
• webMethods-specific plugin in the works
13
Roadmap
Discover more SOA solutions by CGI
• Web site: https://it-toolbox.fr
• Email: [email protected]
14
Our platforms
15
Thank you for your attention
Our commitment to you We approach every engagement with one
objective in mind: to help clients succeed