Embed Size (px)
Citation preview
10/3/2012
1
1
Instituting Good Practices for Data Testing Best Practices for Data Integration / ETL Testing Series
David Loshin, Industry Analyst
Senthil Kanakarajan, Wells Fargo
Ash Parikh, Informatica
Next-Generation Data Integration Series
30 Minutes with Industry Experts
10/3/2012
2
2
10/3/2012
3
3
10/3/2012
4
4
SQL scripts?
Spreadsheets?
Hand coding?
Testing tool?
Not enough?
10/3/2012
5
5
Long time
Expensive
Error-prone
Manual
Not thorough
No reuse
No audit trail
No methodology
10/3/2012
6
6
Managing the Usability of Test Data
Production Data vs. Test Data
Reusable Logic / Repeatable Process
Maintaining Same Test Data Set
Other…
Meeting Project Deadlines
Improving Test Data Over Time
10/3/2012
7
7
definition of business rules introduction of data controls automated testing, validation & reporting
10/3/2012
8
8
A way to identify ERRORS in data sets that have
been MOVED or TRANSFORMED to ensure
they are COMPLETE and ACCURATE and
meet EXPECTATIONS or REQUIREMENTS.
10/3/2012
9
9
AUTOMATION
REPEATABILITY
AUDITABILITY
10/3/2012
10
10
Data Being Transformed
• ETL Reconciliation
• Data Masking
• ETL Testing
• Application Migration
Data is Identical
• ETL version upgrade
• ETL Migration
• Database migration
• Application Retirement
10/3/2012
11
11
Production Reconciliation
Protect the integrity of data
that is loaded into
production systems.
Erroneous data due to failed
loads, faulty logic or operational
issues is caught in a proactive
automated manner and can be
addressed as needed
Development & Test
Provide automation for unit and
regression testing
of integration logic.
Ensure that data produced by DI
code meets requirements and
expectations
Informatica’s Data Validation Solution (DVO)
Ensures the integrity of data as it moves through the IT environment...
10/3/2012
12
12
Production Reconciliation
Protect the integrity of data
that is loaded into
production systems.
Erroneous data due to failed
loads, faulty logic or operational
issues is caught in a proactive
automated manner and can be
addressed as needed
Development & Test
Provide automation for unit and
regression testing
of integration logic.
Ensure that data produced by DI
code meets requirements and
expectations
Informatica’s Data Validation Solution (DVO)
Ensures the integrity of data as it moves through the IT environment...
10/3/2012
13
13
Production Reconciliation
Protect the integrity of data
that is loaded into
production systems.
Erroneous data due to failed
loads, faulty logic or operational
issues is caught in a proactive
automated manner and can be
addressed as needed
Development & Test
Provide automation for unit and
regression testing
of integration logic.
Ensure that data produced by DI
code meets requirements and
expectations
Informatica’s Data Validation Solution (DVO)
Ensures the integrity of data as it moves through the IT environment...
10/3/2012
14
14
Benefits
REQUIREMENTS MANUAL
TESTING INFORMATICA
“We used Informatica to compare 14 tables & appx. 30 million rows in less than 5 hours. The largest of the tables was 94 columns.
When I asked our QA people how long it would take them to run the scripts and test this amount of data, they mentioned months…”
- Customer
10/3/2012
15
15
Informatica.com > Products > Enterprise Data Integration >
PowerCenter > Options
PowerCenter
Productivity Pack*
PowerCenter Data Validation
Option (DVO)
Proactive Monitoring
For PC Operations
Proactive Monitoring
for PC Governance
NEW
*Single Package at an Attractive Price
NEXT UP IN THE SERIES:
Using Data Integration Testing for Reconciling Production Data Assets
PART 2
On-Demand
Product Desk Webinar:
Best Practices for DI/ETL
Testing Presentation & Demo
Click on Link in Browser
