Click here to load reader

Informatica PowerExchange 9.X

  • View
    223

  • Download
    13

Embed Size (px)

DESCRIPTION

Informatica PowerExchange 9.X. Training | CIF CONSULT | Redouane BELBAHRI. Agenda. Objectives What is PowerExchange How Does the Team typically use PWX Power Exchange Batch Architecture Real Examples. Agenda. Objectives. 1. Agenda. - PowerPoint PPT Presentation

Text of Informatica PowerExchange 9.X

  • AgendaObjectivesWhat is PowerExchangeHow Does the Team typically use PWXPower Exchange Batch ArchitectureReal Examples

  • Agenda1Objectives

  • AgendaThis course explains what PowerExchange is and how it is used. We will learn the Architetue (batch) of PowerEXchange, and the key components Undestand the PowerExchange Architecture Describe How PWX Datamaps are used and where they are storedCreate a basic Datamap in the PWX Navigator GUI

  • Agenda2What is PowerExchange

  • AgendaPower Exchange (PWX) is a tool we use to parse complex files and pull them into Designer.Power Exchange (PWX) is comprised of 3 main componentsChange Data Capture Allows you to monitor and retrieve changes that have occurred to data in which you have expressed an interest.Data Maps Used to access data sources like IMS databases, VSAM files and flat files.Personal Metadata - Allows you to browse meta data from remote databases.Remark: Change Data Capture (CDC) and Personal Metadata are out of scope in this presentation

  • PowerExchange Batch Platforms

  • PowerExchange Change Capture (CDC)

  • Agenda3How Does the Team typically use PWX

  • How Does the Team typically use PWXComplex Flat File ParsingComplex copybooks with multiple record typesCreating multiple records out of occurs depending on or blowing them out to the full range of the occursVerifying that the layout is correct with a row testData from the row test can be copied into a text file and then imported into Excel as a comma delimited file.Making sure that the input data can be seen correctly by PWX Interaction with Power CenterPulling Sources and Targets set up with PWX into Informatica Designer

  • When to Use PowerCenterPowerCenter supports only fixed width files,Informatica VSAM reader does not support variable length file. This file is read using a PowerCenter Normalizer transformation. PowerCenter does not support the Occurs Depending On clause.PowerCenter supports comp, comp-3, comp-6 and comp-x fields.

    When to use PWXWhen to Use PowerExchangeHowever, PowerExchange has a capability of reading the variable length VSAM file. In PowerExchange create a data map for the VSAM file by importing the COBOL copybook.

  • Agenda4Power Exchange Batch Architecture How Does PWX work?

  • Where PWX is installed ?

  • Informatica PowerExchangeDesign Time - Batch Architecture

  • PowerExchange CDC ArchitectureFor Oracle (option available at AXA)PowerExchange Change Data Capture (CDC) works in conjunction with PowerCenter to capture changes to data in source tables and replicate those changes to target tables and files.

  • Traditional Methods for Moving DataMultiple manual steps for each source database!

  • PowerExchange ApproachNo Programming, No Intermediate FilesData extracted with SQL, converted and filteredLoaded to target in memoryRestart and Recovery integration with PowerCenter

  • A Datamap defines metadataFor non-relational databases, need to define metadataContains all information regarding the underlying datasetCreated from existing informationVSAM copybooksIMS DBDs, then overlay segment copybooksADABAS FDTIDMS schemasFlattens out hierarchical or multi-record type dataStored on the same platform as the dataNot needed for relational sourcesProvides the information to convert an SQL statement to actual data base requestsWhat is a Datamap ?

  • Where is the Datamap stored?Source version of datamap is stored on windows machine where createdDatamaps folder, (dmp file on windows)Run time version is stored with the PWX software on the source data machineDatamaps folder, (dmp file type on windows)Install.lib.DATAMAPS sequential file on MVSStdatamaps lib on AS400Run time version can be imported into any windows session of PWX Navigator to pull in most recent copyRefer to the Navigator User Guide for more information

  • Different types of data and where they are storedThe 'compiled' version of the data map, in this case test.EBCDIC.dmx' is stored in the 'datamaps' folder directly under the product install directory on the system where it will be used by the product to access the data. It is sent when you see a message in Navigator that says that information must be sent to the 'remote system' in order to retrieve data.

    There is also a lock file for the customer data map. It is 'customer.lck' and is also stored in the Shared folder. It is used to serialize access to the data map source.

  • Configuration FileConfiguration FileSets many defaults parameters for PowerExchangeProvide security settingsCommunication SettingsConfig File Communication Params:LISTENERNode NameTCPIP (standard parm)Port NumberNODENode nameTCPIP (standard parmHost Name or IP addressPort Number

  • What is a Listener?A PowerExchange task that accept requests for dataUses an IP address and Port NumberCan listen on multiple Port numbers at the same timeCan have multiple listeners running (on unique IP and Ports)Only needed on remote platforms for access to batch dataAlso used for connecting to remote PowerExchange Nodes for creating datamaps and Capture Registrations

  • Listener = Phone lineEasy way to rememberThe Listener is the equivalent of the phone number that defines a number, whitch is the IP address and Port, once turned on can be called from other places.

  • Node = Address Book Numbers Easy way to rememberNode statement gives all the available numbers we can call from this location

  • Listener and Node Example

  • Agenda5Real Examples

  • ExamplesFixed Length FileVariable Length FileVariable Length File with multiple record types

  • Connection to PWX NavigatorFrom the TaskbarDesktop icon11Connection to PowerExchange client toolNote: No need to use a Login and password to access2Acces HomepageDemonstration Of PWXOR

  • Example 1 Fixed length VSAM fileCreate Data Map that convert an EBCDIC file to ASCII file format:Add a Data MapFill in the Schema name and the Data Map NameInsert the Input File name and its location on the server, Select the fixed option, & change the default line to EBCDIC & IBM-1047Enter the name of the source file copybook

    Demonstration Of PWX

  • Prepare the Cobol file structureThe Cobol file structure you are being to use should not contains the header informations, if not later you will have this errorRemove this sectionDemonstration Of PWX

  • Add a data mapRight click on the data map folder to add a data map.Demonstration Of PWX

  • Fill in the Schema Name and the Data Map NameFill in the Schema Name and the Data Map NameAccess Method: SEQ (for Flat file)Click nextDemonstration Of PWX

  • Data Map PropertiesLocation and type of copybook that you want to import.Insert the file name (data file) that the map will need to read in Informatica (this file will be on the server). Select fixed or variable depending on the type of file. In the Size box, enter the record length (if not known give 1 as size, will correct later). The screen print above is for a fixed file only. Click FinishDemonstration Of PWX

  • Enter information about the location and type of metadata that you want to import.Click NextImport Copybook Source DetailsColumn Range:Enter the start and end column range for the copybook:Source:Local. The copybook is located on the PowerExchange Navigator system. Remote. The copybook is located on a remote system. Demonstration Of PWX

  • Tell PowerExchange where you have your copybook located on your PC Click next.

    Import Copybook Local Cobol DetailsDemonstration Of PWX

  • Click FinishImport Copybook Configuration DetailsDemonstration Of PWX

  • Import Copybook InformationClick OKDemonstration Of PWX

  • Record DefinitionClick OKDemonstration Of PWX

  • Add a data map Physical definition mapped to the relational definitionRecordTableClick OKDemonstration Of PWX

  • Add a data map (EBCDIC)Click CloseMake a note of the file actual lengthClick yesDemonstration Of PWX

  • Add a data map (EBCDIC)Right click on Test.EBCDIC in the resource explorer (Data Map) and select properties.Select the SEQ Access Method Tab and change the length to 350.Demonstration Of PWX

  • Check the Created Data mapEdit Resource ConfigurationA PowerExchange Navigator resource configuration defines the location of data maps, personal metadata profiles,and any locally stored capture registrations and extraction maps.

    Note: You cannot edit the resource configuration if a PowerExchange resource is open.

    Demonstration Of PWX

  • Add a data map (EBCDIC)Example with OccursDemonstration Of PWX

  • Send data map to Node See dbmover.cfg on the local machineDemonstration Of PWX

  • Perform a Row Test (1/3)

    Select the Table and Click Row Test

    In the Database Row Test dialog box select NRDB (Nonrelational data source )Enter the name of the file in the Override File Name box (file should be in the server)Click Go Demonstration Of PWX

  • Perform a Row Test (2/3)Demonstration Of PWX

  • Perform a Row TestExample with Occurs (3/3)Demonstration Of PWX

  • Created datamap on the integration ServerDemonstration Of PWX

  • Import the created dmx as datamapDemonstration Of PWX

  • Import the source into PowerCenter Designer Go into Informatica designer and open the Source Analyzer. Select Sources and then Select Import from PowerExchange.Demonstration Of PWX

  • Select the Location: nodeDEVSelect Source Type: SEQSelect Multi-Record Datamaps.Click connect.Import the source into PowerCenter Designer It doesnt really matter if the multi record box is checked unless you truly have multi-record datamaps. List of nodes on the dbmover.cfg file in the local machineDemonstration Of PWX

  • Select the data map that you want to import from PWX Click okImport the source into PowerCenter Designer Demonstration Of PWX

  • Import the source into PowerCenter Designer Demonstration Of PWX

  • Metadata ExtensionsNote here, there is a Metadata Extension created automatiqually in the source VSAM.Demonstration Of PWX

  • Use the Source in a mappingDemonstration Of PWX

  • Use the Source in a mappingWith OccursDemonstration Of PWX

  • Create new Connection for PWXIn the Workflow manager, create new Application Connection to use in the session,Choose PWX NRDB BatchClick OKDemonstration Of PWX

  • Create new Connection for PWXIn the Workflow manager, create new Application Connection to use in the session,Choose PWX NRDB BatchChoose Code Page: IBM1047Click OKDemonstration Of PWX

  • PC Session runsWhen the PC session runs, it would talk to the PWX Listener on the Integration Server, and the PWX Listener would open the file, read it, and send the data to the PowerCenter session Demonstration Of PWX

  • Creating a Data mapWhat we have done\DATAMAPS\Training.EBCDIC.dmp

  • Example 2 Variable Length VSAM fileComing soon

  • Variable Length VSAM examplesExample 1A VSAM data set consists of two record types of different lengths distinguished by the value in a field that is in the same location in both record types.

    Below is a sample COBOL copybook of the records definition:

    01VSAMVAR-REC1. 10 KEY-FIELDPIC X(2). 10 REC-TYPEPIC X. 10 CHAR-FIELDPIC X(10).

    01VSAMVAR-REC2. 10 KEY-FIELDPIC X(2). 10 REC-TYPEPIC X. 10 CHAR-FIELDPIC X(20).

    Example 2A VSAM data set consists of one record type whose last field is optional.

    Below is a sample COBOL copybook of the record definition:

    01VSAMVAR-REC1. 10 FIELD1PIC X(2). 10 FIELD2PIC X. 10 FIELD3PIC X(10).

    Example 3A VSAM data set consists of a record that contains a variable length field. The length of the field is contained in a field prior to the variable length field.

    Below is a COBOL copybook of the record definition:

    01VSAMVAR-REC1. 10 FIELD1PIC X(2). 10 FIELD2PIC 99. 10 FIELD3PIC X occurs 10 times depending on FIELD2.

  • Windows Registry

  • Upgrade or Migrate Datamaps across different platforms using PowerExchange Navigator The datamaps will show in your Navigator datamaps window and the .dmp files will be created in the Local Path. import the datamaps that you want to migrate. Copy the .dmp files to theLocal Path for your new PowerExchange Navigator environment. Save each datamap in the new Navigator environment:

  • Configure a PowerExchange Datamap to replace invalid pack data with zerosTo replace invalid pack data with zeros enable Data Checking in the PowerExchange Datamap Properties.

    Click the Data Checking tab.Select the Packed Data Field option.Select Replace according to Data Defaults from the How do you want to Handle bad Field Data list.Go to the Data Defaults tab.Select All or Single Character under Character Replacement.Enter '0' (zero) in Value. This will replace all invalid packed Data fields with zeros.

  • Configure a PowerExchange Datamap to retain record values of bad dataIs it possible to extract the records or find the record information any where which gets skipped as bad data, so that we can report it and send it to our concerned team and make sure that we dont have the bad data in the source file.

    To retain records with bad data in the source file do the following:

    Select the Data Checking option in the Datamap properties.Select "Skip row and write log file msg" under the How do you want to handle Bad Field Data? list. This option will skip the records with bad data and writes the record to the PowerExchange Listener Detail log.Clear the Suppress Data Logging to retain the record.

  • VSAMVSAM stands for Virtual Storage Access Method. It is a method of managing files that is used mainly on mainframes but also on PCs. Specifically, VSAM can speed up access to file data by using a reverse index of records appended to files. This index is called a B+ tree.

    VSAM is a kind of record-oriented file system. In this kind of dataset, information is stored as a collection of records. VSAM records can be of any length; they need not be of one set length. They are, however, organized into blocks called Control Intervals, which are measured in bytes. These Control Intervals are further organized into Control Areas, which are measured in much larger units.

  • Overriding Connection Attributes in the Parameter FileIf you use a session parameter to define a connection for a source or target, you can override the connection attributes in the parameter file. Use the $FTPConnectionName, $QueueConnectionName, $LoaderConnectionName, or $AppConnectionName session parameter.

    When you install PowerCenter, the installation program creates a template file named ConnectionParam.prm that lists the connection attributes you can override for FTP, queue, loader, and application connections. The ConnectionParam.prm file is located in the following directory: /server/bin

  • UPACKED FieldsFor the UPACKED Fields. In the output, we expect sign as F instead it was coming as C. This was an issue in the configuration settings. To resolve this we need to add the following setting in dbmover.cfg file.

    NUMERICSIGN=F

  • Many Thanks for you time today!

    *Diagram of PowerExchange ArchitectureADABAS FDT = Field description information for ADABAS files is maintained in the FDT (File Description Table) IMS DBDs = a database description (DBD), specifies characteristics of a database Datamap creation folder : C:\Informatica\PowerExchange9.0.1\datamapsDBMOVER Configuration File OverviewThe PowerExchange DBMOVER configuration file contains statements that control PowerExchange operationsand communications.The other platforms accessing the listener only needs the IP adress and Port number, the virtual name Node1 is used inlt internaly.Node statement: gives all the available numbers we can call from this locationThe Node name we are given does not have to match the name in the listener machineThe PowerExchange GUI (PowerExchange Navigator) is part of the Windows 32-bit package. I There are 2 options for data maps. You can add a data map (create a new data map) or you can pull down an existing data map that has already been exported to node1.

    Schema NamePowerExchange uses this name as the first part of the data map name, as follows: schema_name.data_map_name Maximum length is 10 characters.

    Access method and corresponding data source, into which you can import each type of copybook:Access Method: SEQ. Flat file. Import Record Definitions: Select this option to import a copybook into the data map.

    Import Record DefinitionsSelect this option to import a copybook into the data map. Clear this option to add only the data map.

    For some copybook types, you must enter additional information.Source Local. The copybook is located on the PowerExchange Navigator system. Remote. The copybook is located on a remote system. The locations displayed in the Location list are defined by NODE statements in the dbmover.cfg configuration file on the PowerExchange Navigator system

    Histirique: carte perfore (7,72)Here you can see the created datamap:A record: whitch is the physical definition, this match the source file exactlyTable: Relationbal logical view of the datamapWe have the physical definition mapped to the relational definitionMake a note of the files actual length.Click yes.

    Resource ConfigurationIf you are starting PowerExchange Navigator for the first time, theResource Configuration - Current Configuration dialog box appears. Click OK to create a resourceconfiguration named Default in the following default directory:c:/Informatica/PowerExchangev.r.m/ExamplesChoose the datamap to send and open the Data Map folderYou dont need to fill in the user & passwordCheck on the server that your datamap is created in the folder C:\Informatica\PowerExchange9.0.1\datamapsThe Row Test:Provide the ability to see if the datamap is working

    DB Type:NRDB. Nonrelational data source such as a VSAM data set or a flat file. Note: This data source type uses a three-tier naming convention in the generated SQL. Optionally enter the name of the file to override the default file in the Override File Name box. In the File Password box, enter the password for the file, if required.

    It doesnt really matter if the multi record box is checked unless you truly have multi-record datamaps.

    It doesnt really matter if the multi record box is checked unless you truly have multi-record datamaps. If the copybook has multiple layouts it will look like this when you expand it, you can then select the layout you want ,If there is only 1 layout then select the main folder as shown above in this slide.

    Without Occurs & RedefineLocation of the source or target database as specified as a node in the PowerExchange configuration file dbmover.cfg. User Name=Database user namePassword=Password for the user name=node1Then when the PC session runs, it would talk to the PWX Listener on the mainframe, and the PWX Listener would open the file, read it, and send the data to the PowerCenter session From the PowerExchange Navigatore we created a datamap, name it and specified the source of the metadata (cobol copybook)Import the copybook into PowerExchange, made any modification in the datamap, send to node and perform a row testOnce the datamap is sent in the remote node, the map is saved using the PWX softwareThe source copy of the datamap will not be executedYou should always export what is in the server , before modifying the data map