View
53
Download
1
Category
Tags:
Preview:
DESCRIPTION
iWay Data Profiler. Product Highlights. 5.0 GA. Anand Maheshwari. What is iWay Data Profiler. - PowerPoint PPT Presentation
Citation preview
Anand Maheshwari
Product Highlights
5.0 GA
iWay Data Profiler
1
What is iWay Data Profiler
2
iWay Data Profiler provides the ability to look at data quality from a business user perspective with dashboard templates to view, monitor, compare, and report by leveraging the turnkey business intelligence technology through semantic analysis, complex business rules and data drill-down.
Features
1. Ability to profile data against a set of standard (pre-defined) data quality rules and metrics
2. Ability to create custom organization specific data quality indicators3. Advanced comparison of data profiling and quality indicators4. Ability to create and execute data quality projects against varying payloads5. Ability to rollup data with drill through6. Advanced administration capability7. Advanced auditing capability8. Easier migration from earlier versions9. Ability to track background processes for faster troubleshooting and status10. Use of current standards like JQuery for enhanced user experience11. Support for all leading database platforms12. Support for wide variety of OS platforms13. Support for German, French, Spanish and English locales14. Increased performance through several tuning efforts
3
4
Features 4.1 5.0
Create profile with drill-through
Export profile report to: AHTML and PDF format AHTML
XLS ×
Email profile using SSL
Create profile report for: Data Analyses
Business Rules
Primary Key
Foreign Key
Summary
Hierarchical folders ×
Uploading profile in background ×
View process result ×
Rollup profile with drill-through ×
Profile file size 100MB 300MB
1. Ability to profile data against a set of standard (pre-defined) data quality rules and metrics - Comparison
1. Ability to profile data against a set of standard (pre-defined) data quality rules and metrics – New Features
5
• Uploading profile in background
• View result to display uploading progress
• Export profile report in XLS format
6
2. Ability to create custom organization specific data quality indicators – Comparison
Features 4.1 5.0
Create and view DQIs report
Export PDF report
Folder Hierarchical division ×
Comparison Across Expression ×
Visualization for success and failure counts ×
More user interaction using AHTML Format ×
2. Ability to create custom organization specific data quality indicators – New Features
7
Easy to understand the data quality indicators through multiple reports
Flexibility to compare the expressions View Result to display progress for the background processes
2. Ability to create custom organization specific data quality indicators – New Features
8
Easy to understand the data quality indicators through multiple reports
2. Ability to create custom organization specific data quality indicators – New Features
9
Flexibility to compare the expressions
2. Ability to create custom organization specific data quality indicators – New Features
10
View Result to display uploading progress
11
Features 4.1 5.0
Quick compare of profiles/DQIs
Create Dataset & Template
Graphical representation of profile/DQI compare
Default Graph Line 3D Bar
Displaying table report with visualization ×
Hierarchical folders division ×
3. Advanced comparison of data profiling and quality indicators - Comparison
3. Advanced comparison of data profiling and quality indicators – New Features
12
Additional table report and chart type
• Display of both success and failure count and percentages
• Informative table report display with charts
4. Advanced comparison of data profiling and quality indicators – New Features
13
New design of data displayed in dataset table, for Profile & DQI
Modified default report as 3D Bar
4. Ability to create and execute data quality projects against varying payloads – New Features
14
• Data Quality Projects for Profiles/DQIs enables running a DQC plan for generating a Data Profile/DQI and loading into database repository• Create data quality project• Manage data quality project
• Reusability of plan - The same plan can be executed against different inputs• View Result for different level of progress
4. Ability to create and execute data quality projects against varying payloads – New Features
15
Create project – option to upload Profiling plans and DQI plans
Plan can be with drill-through and also with Rollups
4. Ability to create and execute data quality projects against varying payloads – New Features
16
Manage project
4. Ability to create and execute data quality projects against varying payloads – New Features
17
Reusability of plan : The same plan can be executed against different inputs
4. Ability to create and execute data quality projects against varying payloads – New Features
18
Note:
Plan is supported with TXT and CSV Input, Other Input in DQC like XML, Database is not supported in the current version
Output can only be profile steps (or DQI steps depending on the type of DQ project)
The following input settings of the input file can be modified without DQC GUI:• Number Of Lines In Header • Field Separator • Line Separator • String Qualifier • Encoding
4. Ability to create and execute data quality projects against varying payloads – New Features
19
View Result for different level of progress
5. Ability to rollup data with drill through – New Features
20
Roll up is a part of iWay Data Profiler that enables a user to profile view a specific subset of a data.
Rollups for: Data Analysis Primary Key Business Rules.
5. Ability to rollup data with drill through – New Features
21
Enabled drill-through for the subset of data
6. Advanced administration capability – Comparison
22
Features 4.1 5.0
Create Roles, Groups and Users
LDAP User Authentication and LDAP Role Authorization
Option to create, delete and manage folders ×
Group and Role details can be viewed while creating a user ×
Groups have access to one or more Virtual Folders ×
Base Role : Admin, Power, Technical and Analytical with different privileges
×
iSM Tracking privilege ×
DQ Project privilege ×
Rollup privilege ×
6. Advanced administration capability – New Features
23
• Hierarchical folder structure to manage profiles, DQIs, datasets, templates Grouping related things together based on the user choice Sharing and easy to compare Created a group called “Admin” and linked all folders to it Folder structure for a user
o Private o My Groupo Customo Public
• Created default base roles: admin, power, technical and analytical with varying authorizations
6. Advanced administration capability – New Features
24
Manage Folders
6. Advanced administration capability – New Features
25
Manage Roles
6. Advanced administration capability – New Features
26
Manage Groups
6. Advanced administration capability – New Features
27
Manage Users
7. Advanced auditing capability – New Features
28
Audit (Tracking user name, IP address, login time and logout time) with different databases
• Enhanced user security
• Filter audit information
• Track log in and log out time in iDP with IP Address
• Track invalid login & iDP server manager login
7. Advanced auditing capability – New Features
29
Audit information for a user
7. Advanced auditing capability – New Features
30
Filter audit information
8. Easier migration from earlier versions – New Features
31
IncludesUpgrade script is for migrating from the older version to newer version
4.1 5.0Available for:
MySql MSSQL Oracle
There is no loss of data exists in database “idp_data”
Does not includeSynonym creationDrill-through adapter creation in WebFOCUS
9. Ability to track background processes for faster troubleshooting and status – New Feature
32
Interface for the user to monitor the back-end process Multilingual support for view process result – Available for Profile, DQI, DQ
Project and Custom Reports
iWay Data Profiler : Architecture
33
iWay Data Profiler - Component Architecture
34
Benefits
Web enabled thereby single deployment and access anywhere Aids in the identification of data quality issues Expertise and support of a mature business intelligence product –
WebFOCUS Data quality comparison over time Documents and audits quality-control processes for review Streamlines offline maintenance activities to ensure optimum performance Share the reports using a secured connection Mobile support
35
Anand Maheshwari
Thank You !
5.0 GA
iWay Data Profiler
36
Appendix – Additional Features
37
Multilingual support
38
iWay Data Profiler supports multi-languages English German Spanish French
Note: ISM Tracking Edit page language will change corresponding to the language choose in ISM configured not based on the IDP chosen language.
Performance tuning
39
JAVA code has been restructured for better performance
• Complete recoding using XPATH and transaction based multiple coding being undertaken as processing of complicated Rollups found to crash the JVM
• The maximum size of the Profile XML file, which can be uploaded into DP, has been increased from 1MB to 300 MB
Revamped iWay Data Profiler with JQuery
40
JQuery Functionalities• Enhance look and feel• Styling• Cross browser compatibility• Support all mobile devices• Grouping related things together• Sharing• Easy to compare• Fast loading• Alerts and alignment • Expand All/ Collapse All functionality• More/ Less functionality• Pagination• Homepage carousel – to show case different graphs of iDP• Landing page accordion layout – to display different iDP modules
Modified both Windows and Linux installer
41
Updated both Windows and Linux Installer scripts for DQ Project to get the Drill-through DB credential from the installer
Installer is available in English, German, French, Spanish Same Linux build is supported for AIX platform
Backward Compatibility with iWay Data Quality Center
42
Modified iDP for Profiling DQI
Support across versions DQC 7.0.1 & higher Input data Reports
Recommended