1

A Workable Solution for Basic Metadata

January 9, 2006

2

Contents

What is Metadata? CPD Decision Support Environment (DSE) Metadata Preparation Metadata Presentation – Navigation Part 1 Metadata Presentation - Navigation Part 2 Metadata Presentation - Navigation Part 3 Metadata Presentation - Data Depth Value of This Metadata Current Issues & Next Steps

3

What is MetaData ?

DEFINITION:Metadata is information about the data which is managed by an enterprise to conduct its business. It includes:

what the data means, where it is used, aliases, valid values Physical data structures and designs Sourcing and transformation rules, information on programs used to move data Descriptive information such as quality and condition of the data

EXAMPLES

BUSINESS METADATA• business definitions• business reference data• report descriptions • metrics• logical data models• Data sourcing and transformation rules

TECHNICAL METADATA

• physical data models• XML Message formats• interface file layouts• ETL programs• ETL data mappings • Data lineage diagrams

OPERATIONAL METADATA

•ETL load statistics and load times

4

CPD Decision Support Environment (DSE)

DECISION SUPPORT DATA BASEFOR MARKETING

DM2

Lan

din

g Z

one

Cognos

MarketingUsers

DECISION SUPPORT DATASTORAGE

DM1

AS400

Confirmed Fraud

Fraud Disputes

Past DisputedTransactions

TSYS

PostedTransactions

CapstoneApplications -

ApprovedApplications

Falcon

Transactions

CPD - Decision Support Environment

USERS

MarketingCampaignDirector

Other users

Data Structure / DataBase Environment : Models are in ModelMart/ ERWIN Physical DataBase is in Oracle ETL tool used is DataStage.

MetaData Structure / Repository Environment : Metadata is Kept in Erwin models. Repository (An Excel Spread Sheet Kept on CIBC Intranet) Contains - What Attributes means, its valid values, Sourcing and transformation rules etc... Navigation is with in the Excel Spreadsheet using VBA.

5

Meta Data Preparation

Meta DataPublishingTemplate

1.

3.

Erwin Model(ModelMart)

Model EntryModel

Change Extract

2.

*.csv Merge Meta DataDocument

Publish

1. When column/table additions are required, data modelers enter the changes into the Erwin Model. For Attribute updates, an in-house developed tool (ExcelERwin) is used.

2. For Meta Data releases, the current Meta Data is extracted from Erwin into a raw .CSV file.

3. The raw .CSV file is formatted for ease of use by merging it with a Meta Data Publishing template, which contains Lookup Macros and Drill Down menu systems.

4. The finished Meta Data document is then published to the Intranet Data Management & Decision Support website.

6

Meta Data Presentation-Navigationpart 1

The Meta Data presentation sheet offers multiple ways to find the information you are looking for:

Drill Down Menus by Table Groups for General browsing.

Search by Table andColumn for quick accessto specific information.

Go directly to native Spreadsheet

7

Meta Data Presentation-Navigationpart 2

After choosing a Group from the Drill Down menu, you jump to the Tables menu.

The cursor is placed on the Table Group selected.

The Tables are organizedby group, and colour codedwith the colour of the groupbutton.

8

Meta Data Presentation-Navigationpart 3

Clicking on a Table brings up the sheet containing the Meta Data

The cursor is placed onthe first listing of the table’scolumns.

Once you are done, youcan jump back to the Groupmenu with this button.

9

Meta Data Presentation-Data Depth

The following are the column headings in the Meta Data spreadsheet.They Illustrate the depth of data available in the current format.

10

Value of this Metadata

- It contains the basic metadata – which is very important.- Basic Metadata like: attribute definitions, size, valid values…...- Other important contents are the sourcing information.- Repository is simple and easy to access – on Excel spreadsheets. - Navigation is within Excel spreadsheet using VBA code. - Distribution is through Intranet, which is integrated within our site.- Excel Templates are available to down load metadata for offline study - Can use all Excel functionality.- Metadata releases are aligned with DSE releases.- Users are dependant on this single source of Metadata.- Resource usage – it is totally developed by co-op students and all regular releases are done by them.

- This repository is also supported by jpeg images of the physical model data structure separately for DM1 & DM2 on Intranet.

11

Current Issues and Next Steps

Current Issues: - Major issue is the Metadata quality, which is poor – it requires upgrades. - Another issue is the completeness of contents, especially for the old data elements – it also requires upgrades.

Next Steps:

- Besides the above upgrades, we plan to enhance each Table’s base definitions and its characteristics like granularity, types of attributes, source of data, frequency of creation, number of records & growth, a brief process of creation and archive & backup policies ……