PowerExchange SQL Server Adapter Guide...1 C HAPTER 1 Overview This chapter includes the following topics: ♦ PowerExchange Batch, 1 ♦ Change Capture with PowerExchange Change and

Informatica® PowerExchange®

(Version 8.6)

SQL Server Adapter Guide

Informatica PowerExchange SQL Server Adapter Guide

Version 8.6 June 2008

Copyright (c) 1998–2008 Informatica Corporation. All rights reserved.

This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. This Software may be protected by U.S. and/or international Patents and other Patents Pending.

Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as provided in DFARS 227.7202-1(a) and 227.7702-3(a) (1995), DFARS 252.227-7013(c)(1)(ii) (OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR 52.227-14 (ALT III), as applicable.

The information in this product or documentation is subject to change without notice. If you find any problems in this product or documentation, please report them to us in writing.

Informatica, PowerCenter, PowerCenterRT, PowerCenter Connect, PowerCenter Data Analyzer, PowerExchange, PowerMart, Metadata Manager, Informatica Data Quality, Informatica Data Explorer, Informatica B2B Data Exchange and Informatica On Demand are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

Copyright, Byte Designs Ltd. All rights reserved.

This product includes ICU software which is copyright (c) 1995-2003 International Business Machines Corporation and others. All rights reserved. Permissions and limitations regarding this software are subject to terms available at http://www-306.ibm.com/software/globalization/icu/license.jsp.

The product includes the zlib library copyright (c) 1995-2005 Jean-loup Gailly and Mark Adler.

DISCLAIMER: Informatica Corporation provides this documentation “as is” without warranty of any kind, either express or implied, including, but not limited to, the implied warranties of non-infringement, merchantability, or use for a particular purpose. Informatica Corporation does not warrant that this software or documentation is error free. The information provided in this software or documentation may include technical inaccuracies or typographical errors. The information in this software and documentation is subject to change at any time without notice.

Part Number: PWX-MSQ-860-0001

Table of Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiInformatica Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Informatica Customer Portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Informatica Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Informatica Web Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Informatica Knowledge Base . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Informatica Global Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

Chapter 1: Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1PowerExchange Batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Change Capture with PowerExchange Change and PowerExchange Real-time . . . . . . . . . . . . . . 1

Phases of Change Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

PowerExchange Change Capture Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Chapter 2: Bulk Movement of Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Accessing Microsoft SQL Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Testing a Remote PowerExchange Listener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Sourcing Microsoft SQL Server Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Chapter 3: Personal Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Using Personal Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Chapter 4: Configuring SQL Server Change Data Capture . . . . . . . . . . . . . . . . . . . . . 9Configuring SQL Server for Change Data Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Configuring PowerExchange for Change Data Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Configuring the DBMOVER Configuration File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Configuring Real-time Extraction Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Configuring Batch Extraction Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Configuring Continuous Extraction Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Implementing SQL Server Change Data Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Managing Capture Registrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Adding a Capture Registration to an Existing Group . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Deleting a Registration Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Deleting a Registration Entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Viewing a Registration Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Viewing a Capture Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Extraction Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Define the Extraction Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Deleting an Extraction Group or Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Application Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Table of Contents iii

Reset to a New Start Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Migrating to Condense File Extraction Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Chapter 5: PowerExchange Condense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29PowerExchange Condense Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Making Capture Registrations Eligible for Condense Processing . . . . . . . . . . . . . . . . . . . 29

Configuring the PowerExchange Condense Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

PowerExchange Condense Operational Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

PowerExchange Condense Output Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Configuring PowerExchange Condense Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Starting and Stopping PowerExchange Condense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Starting PowerExchange Condense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Stopping PowerExchange Condense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Chapter 6: Managing Change Data Capture Definitions. . . . . . . . . . . . . . . . . . . . . . . 37Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Changing the Underlying Data Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Adding Columns to Capture Registrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Removing Columns from Capture Registrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Modifying Columns in Capture Registrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Changing Extraction Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Associating Capture Registrations with Extraction Maps . . . . . . . . . . . . . . . . . . . . . . . . . 41

Adding or Removing Columns from Extraction Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Version Indicator in Extraction Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Changing Properties of Capture Registrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Amending a Capture Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Setting to Active . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Set to History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Chapter 7: Accessing CDC Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Accessing Captures Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Creating Restart Points with DTLUAPPL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

DTLUAPPL - The Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Running DTLUAPPL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Adding an Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Modifying an Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Printing an Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Real-time Extraction Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Testing with the Navigator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Using Real-time Extraction Mode in PowerCenter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

SQL Server CDC Condense File Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Chapter 8: Stopping Change Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

iv Table of Contents

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Table of Contents v

vi Table of Contents

Preface

This manual is designed to give the required information to develop a Microsoft SQL Server data capture environment and must only be used once all the required steps from the PowerExchange Installation Guide are completed successfully. It is assumed that the PowerExchange Listener is active on the target platform and that it has been tested by use of the Ping utility.

The steps required in this manual depends on the functions of the product will be used at this installation. There are two strategies available for refreshing data in a target database:

♦ Bulk update using PowerCenter

♦ Frequent or near real-time update using data capture techniques

PowerCenter or other third-party software can be used to move the data to the target.

Informatica Resources

Informatica Customer PortalAs an Informatica customer, you can access the Informatica Customer Portal site at http://my.informatica.com. The site contains product information, user group information, newsletters, access to the Informatica customer support case management system (ATLAS), the Informatica Knowledge Base, Informatica Documentation Center, and access to the Informatica user community.

Informatica DocumentationThe Informatica Documentation team takes every effort to create accurate, usable documentation. If you have questions, comments, or ideas about this documentation, contact the Informatica Documentation team through email at [email protected]. We will use your feedback to improve our documentation. Let us know if we can contact you regarding your comments.

Informatica Web SiteYou can access the Informatica corporate web site at http://www.informatica.com. The site contains information about Informatica, its background, upcoming events, and sales offices. You will also find product and partner information. The services area of the site includes important information about technical support, training and education, and implementation services.

vii

Informatica Knowledge BaseAs an Informatica customer, you can access the Informatica Knowledge Base at http://my.informatica.com. Use the Knowledge Base to search for documented solutions to known technical issues about Informatica products. You can also find answers to frequently asked questions, technical white papers, and technical tips.

Informatica Global Customer SupportThere are many ways to access Informatica Global Customer Support. You can contact a Customer Support Center through telephone, email, or the WebSupport Service.

Use the following email addresses to contact Informatica Global Customer Support:

♦ [email protected] for technical inquiries

♦ [email protected] for general customer service requests

WebSupport requires a user name and password. You can request a user name and password at http://my.informatica.com.

Use the following telephone numbers to contact Informatica Global Customer Support:

North America / South America Europe / Middle East / Africa Asia / Australia

Informatica Corporation Headquarters100 Cardinal WayRedwood City, California 94063United States

Toll Free +1 877 463 2435

Standard RateBrazil: +55 11 3523 7761 Mexico: +52 55 1168 9763 United States: +1 650 385 5800

Informatica Software Ltd.6 Waltham ParkWaltham Road, White WalthamMaidenhead, BerkshireSL6 3TNUnited Kingdom

Toll Free 00 800 4632 4357

Standard RateBelgium: +32 15 281 702France: +33 1 41 38 92 26Germany: +49 1805 702 702Netherlands: +31 306 022 797United Kingdom: +44 1628 511 445

Informatica Business Solutions Pvt. Ltd.Diamond DistrictTower B, 3rd Floor150 Airport RoadBangalore 560 008India

Toll Free Australia: 1 800 151 830Singapore: 001 800 4632 4357

Standard RateIndia: +91 80 4112 5738

viii Preface

C H A P T E R 1

Overview

This chapter includes the following topics:

♦ PowerExchange Batch, 1

♦ Change Capture with PowerExchange Change and PowerExchange Real-time, 1

♦ Phases of Change Propagation, 2

♦ Software Requirements, 3

PowerExchange BatchPowerExchange Batch allows you to perform bulk moves of data from data source to target. You can move data from and to UNIX or Linux, Windows, z/OS, and i5/OS.

Change Capture with PowerExchange Change and PowerExchange Real-time

Continually repeating the load process through PowerExchange Batch may be costly in terms of space, time, and human resources. What is required is the ability to capture additions, amendments, and deletions to the source data structures and the data itself. In addition, there is a need to ensure that if several changes have taken place, only the latest change is applied.

A number of rules are required to ensure that the start point for the changes and the changes themselves can be clearly identified. The change management process must impact on operational systems and procedures. The process needs to be integrated within them to ensure continuous identification and capture of changes.

The logical extension of the initial PowerExchange Batch process is PowerExchange Change that retrieves updates at a periodicity set by the user, and the near real-time version PowerExchange Real-time. A variety of components work together to propagate changed data from a source to a target. This section provides a diagram and descriptions of the data-flow and control-flow components of PowerExchange Change. PowerExchange Change and PowerExchange Real-time are both briefly covered here for clarity, however, for MS SQL Server capture, PowerExchange Real-time is the only option.

PowerExchange Change is enabled by the CAPX access method from condensed files.

PowerExchange Real-time is enabled by the CAPXRT access method direct against the PowerExchange Logger.

1

PowerExchange Change and PowerExchange Real-time make data available to be applied to target databases but does not apply the data directly. In these instances, the apply process can be carried out with PowerCenter or a partner tool.

The following figure shows how the CAPXRT or CAPX access methods make changes available to PowerCenter so that PowerCenter can apply the changes to the target databases:

Phases of Change PropagationChange capture is the process of capturing changes made to the source database and creating an inventory of those changes in an internal log.

PowerExchange Change Capture ComponentsPowerExchange Microsoft SQL Server Capture sources data from Microsoft SQL Server publication technology.

Figure 1-1 depicts the PowerExchange Change Capture architecture for a Microsoft SQL Server Capture source that can be applied to a target database using either the PowerCenter Apply feature or a partner product:

Figure 1-1. PowerExchange Change Capture Architecture for a Local Microsoft SQL Server Target

PowerExchange Listener

CONDENSEDCHANGES

LISTENERCAPX PowerCenter

PowerExchangeCCT

CaptureRegistrations

PowerExchangeChange

UOWCLEANSER

CONDENSERTASK

PowerExchange Real-timeAdapters

AdabasDatacom

DB2 for z/OSDB2 for i5/OSDB2 for LUW

IDMSIMS

OracleSQL Server

VSAM

PowerExchange ChangeAdapters

AdabasDatacom

DB2 for z/OSDB2 for i5/OS

IDMSIMS

VSAM

EXTRACTIONMAPS

CAPTUREREGISTRATIONS

PowerExchange Navigator

PowerCenterUOW

CLEANSER

LISTENERCAPXRT

TCP/IP or MQ

Push or Pull

PowerExchangeReal-time

PowerExchangeApply

Third PartyChange Data

Consumer

WINDOWS

User Application

1

SQL ServerReplication

4

PowerExchange

5

PowerExchange REGISTRATIONDEFINITIONS

B

PowerExchange NAVIGATOR

A

SQL ServerLog

3

SQL ServerDatabase

2

Key

CONTROL FLOWCOMPONENT

DATA FLOWCOMPONENT

2 Chapter 1: Overview

The components that control the flow of data appear as elliptical shapes with alphabetic labels. These components handle the changed data as it progresses through the propagation process. Some of these components, such as the user application and the source and target databases, are not PowerExchange Change Capture components.

Table 1-1 describes the data flow components:

To create a data capture environment, complete the following steps:

1. Configure DBMOVER.cfg parameters.

2. Configure Microsoft Replication.

3. Register a data source.

4. Complete Microsoft SQL Server prerequisites.

5. Materialize tables using the Bulk Move facility.

6. Register a restart token.

7. Access the data.

For more information, see the PowerExchange Reference Manual.

Software RequirementsSoftware requirements for PowerExchange Change Capture and Microsoft SQL Server Capture are Microsoft SQL Server 2000 or greater in the following versions:

♦ Enterprise Edition

♦ Standard Edition

♦ Development Edition

Change capture is not compatible with Personal Edition.

Table 1-1. Change Propagation Components

Component Description

User Application Any software that updates the source database on an ongoing basis.

SQL Database The Microsoft SQL Server database where the source data resides. PowerExchange Change Capture can capture changes from more than one source database or data file. Microsoft SQL Server data is captured by utilizing existing Microsoft Publication technology.

SQL Server Log Microsoft SQL Server uses the logs to source the replication transactions.

Replication PowerExchange works with Microsoft publication technology to make changes available for extract.

PowerExchange PowerExchange can direct data returned to a number of targets. These may involve passing data to a partner tool to apply the changes to the target, or to use PowerCenter software to apply the data changes directly to the target.

PowerExchange Navigator PowerExchange Navigator is used to specify the PowerExchange registration, locations, users, and user groups.

PowerExchange Registration Definitions

The central repository holds information on all the data sources and locations. The repository can be created on whatever platform is deemed preferable.

Software Requirements 3

4 Chapter 1: Overview

C H A P T E R 2

Bulk Movement of Data


♦ Accessing Microsoft SQL Server, 5

♦ Testing a Remote PowerExchange Listener, 5

♦ Sourcing Microsoft SQL Server Data, 5

Accessing Microsoft SQL ServerBefore you access Microsoft SQL Server, complete the PowerExchange pre-installation checklist.

♦ The name of the distribution server either already in existence or created for PowerExchange capture.

♦ The name of the distribution database either already in existence or created for PowerExchange capture.

♦ The tables to be sourced and whether they are grouped under a particular owner.

♦ The user name and password to access these tables.

Testing a Remote PowerExchange ListenerTo use a remote PowerExchange Listener, it must first be started. To determine if it is started and to check your connectivity, enter the following command at a command prompt:

dtlrexe loc=<node> prog=ping

In this command, <node> is the Node name in the dbmover.cfg file that points to the remote platform.

Sourcing Microsoft SQL Server DataThis document covers the following topics, allowing you to capture data, then access that data with PowerExchange tools. The captured data is also available for use by other products, such as ETL tools.

To source Microsoft SQL Server data:

♦ Obtain the valid user ID and password for the source database.

5

♦ Obtain any database specific information, such as Microsoft SQL Server database and server names.

♦ Ping the IP address of the source machine to check it is visible.

♦ Add a node name for the location in the dbmover.cfg file.

♦ Use the command “dtlrexe loc=<node> prog=ping uid=<userid> pwd=<pwd>” to test that the remote PowerExchange Listener is started, where <node> is the Node name from the dbmover.cfg file.

♦ Create the Microsoft SQL Server replication environment if not already used.

♦ Use PowerExchange Navigator to create a Personal Metadata Profile for the Microsoft SQL Server source database and view some real data, having checked that the metadata is returned successfully. For more information, see the PowerExchange Navigator User Guide.

♦ Use PowerCenter or similar software product to execute the request to move data.

6 Chapter 2: Bulk Movement of Data

C H A P T E R 3

Personal Metadata

This chapter includes the following topic:

♦ Using Personal Metadata, 7

Using Personal MetadataYou can view Microsoft SQL Server tables and data from the PowerExchange Navigator.

To view Personal Metadata, make sure the PowerExchange Navigator can see the data server by adding a new entry to the dbmover.cfg file. By default, the first lines of the file appears as follows:

LISTENER=(node1,TCPIP,2480)NODE=(local,TCPIP,127.0.0.1,2480)NODE=(node1,TCPIP,127.0.0.1,2480)NODE=(default,TCPIP,x,2480)

If Microsoft SQL Server resides on a machine remote from the PowerExchange Navigator, add a line that specifies the IP address and the PowerExchange Listener started on that machine.

To create Personal Metadata to view Microsoft SQL Server data:

1. In the PowerExchange Navigator Resource Explorer, right-click Personal Metadata and click Add Personal Metadata, or click the Add Personal Metadata icon. The Personal Metadata - Name dialog box appears.

2. Enter the following information:

Field Description

Name Name for the Personal Metadata.

Location Location of Microsoft SQL Server data. This location must also be specified in the dbmover.cfg configuration file.

Type Type of data. Select MSSQL.

UserID UserID may not be required. The security requirements are dependant upon the settings of the SECURITY dbmover parameter. For more information about the SECURITY parameter, see the PowerExchange Reference Manual.

Password Password may not be required. The security requirements are dependant upon the settings of the SECURITY dbmover parameter. If the password is left blank but is required then a pop-up password box will appear enabling the password to be given. Leaving the password blank prevents the password being stored by PowerExchange and potentially causing password violations if the Personal Metadata is read after the password for the user has changed.The SECURITY parameter is documented in the PowerExchange Reference Manual.

7

3. Click Next.

4. To reduce the number of tables returned, enter the Owner or Table name. Qualify the selection if required, and click Finish.

If you leave Owner and Table blank, the Navigator returns all tables in the database.

The number of tables returned for selection can be narrowed down. If both the Owner and Table are left blank then all tables within the database are returned. Qualify the selection if required and click Finish.

The Navigator displays all tables meeting the filter requirements.

5. Click the table you want to use.

The columns and column attributes appear.

6. Click the Row Test icon .

7. Review the database information. By default, the row test returns 10 rows. Optionally, change the number of rows returned by the row test, and then click Go.

The Navigator returns the specified number of rows from the table.

DSN Data Source Name. The name of the Microsoft SQL Server.

Database Name of the Microsoft SQL Server database. This field is required.

Description An optional description of the metadata.

Field Description

8 Chapter 3: Personal Metadata

C H A P T E R 4

Configuring SQL Server Change Data Capture


♦ Configuring SQL Server for Change Data Capture, 9

♦ Configuring PowerExchange for Change Data Capture, 10

♦ Implementing SQL Server Change Data Capture, 16

♦ Managing Capture Registrations, 17

♦ Extraction Maps, 20

♦ Application Group, 24

♦ Migrating to Condense File Extraction Modes, 26

Configuring SQL Server for Change Data Capture

Microsoft SQL Server Prerequisites♦ If the PowerExchange Navigator does not reside on the same box as the Microsoft SQL Server software, then

Microsoft SQL Server client software must be installed on the Navigator machine.

♦ Microsoft SQL Server must be Microsoft SQL Server 2000 or greater.

♦ Microsoft SQL Server Replication must be enabled using the Microsoft SQL Server Publication technology.

Required AuthorityIn the absence of a user name and password both the PowerExchange Navigator and capture extract uses Windows authorization to connect to Microsoft SQL Server. Registering items for capture requires system administrator or db_owner authority. For data extract tasks the user must have read access to the distribution database.

You require read access to the distribution database for data extraction tasks.

Microsoft SQL Server PublicationsPowerExchange requires that Microsoft SQL Server distribution is enabled on the server from which data is to be captured. To enable distribution through the Microsoft SQL Server Management console, click Tools > Replication > Configure Publishing, Subscribers, and Distribution:

9

Alternatively, use the following Microsoft SQL Server supplied stored procedures:

♦ sp_adddistributor

♦ sp_adddistributiondb

♦ sp_adddistpublisher

For more information about Microsoft SQL Server Replication technology, see the Microsoft MSDN library on the Microsoft web site.

To capture a large number of transaction updates, use a distributed server as the host of the distribution database. Using a dedicated distribution server can avoid competition for CPU and disk storage with a production instance.

The default retention period for transactions at the distributor is 72 hours. Informatica recommends that this is increased to 14 days, although the need for resilience will need to be balanced against practical space issues when there is a very high frequency of change.

RestrictionsThe following restrictions apply to SQL Server capture:

♦ Capture tables must have a Primary Key.

♦ System tables can not be captured.

♦ Long columns are not captured in before images.

♦ User ID is not captured.

♦ Timestamp is that of capture, and not of the original transaction.

♦ Tables that include column types of sql_variant cannot be registered for capture.

Configuring PowerExchange for Change Data CapturePowerExchange uses the Microsoft SQL Server distribution database to extract captured changes and provides the following extraction options:

♦ Real-time extraction mode. When you use real-time extraction mode, PowerExchange reads the changes from the distribution database and continues reading change data until you stop the extraction.

Real-time extraction provides the lowest latency for change data extraction.

♦ Batch extraction mode from condense files. When you use batch extraction mode from condense files, PowerExchange reads the changes from condense files created by PowerExchange Condense. PowerExchange reads only the closed condense files that existed when the extraction starts and ends the extraction when these condense files have been processed.

Batch extraction mode from condense files provides the highest latency for change data extraction.

♦ Continuous extraction mode from condense files. When you use continuous extraction mode from condense files, PowerExchange reads the changes from condense files created by PowerExchange Condense. PowerExchange reads data from condense files as it is being written by the condense task, even if the condense file is not yet closed. PowerExchange continues reading changes from condense files until you stop the extraction.

Using continuous extraction mode from condense files provides near real-time access to change data.

For more information, see “PowerExchange Condense” on page 29.

You can configure PowerExchange for all three extraction modes providing flexibility when choosing an extraction mode for a specific application or set of tables.

10 Chapter 4: Configuring SQL Server Change Data Capture

Configuring the DBMOVER Configuration FilePowerExchange uses the specifications provided in the DBMOVER configuration file, dbmover.cfg, to configure the operation of various PowerExchange tasks as well as their communication with other PowerExchange tasks. For a complete list of all DBMOVER configuration file parameters, see the PowerExchange Reference Manual.

PowerExchange CDC and the PowerExchange Listener use parameters specified in the dbmover.cfg configuration file to:

♦ Connect to Microsoft SQL Server databases and tables.

♦ Determine the directories in which to store capture registrations, extraction maps, capture files, and condense files.

♦ Get connection information in order to extract changes from SQL Server distribution databases and from condense files.

PowerExchange Condense has an additional configuration file, dtlca.cfg. For more information, see “Configuring PowerExchange Condense Parameters” on page 32.

To use change data capture, configure the following DBMOVER configuration parameters:

CAPT_PATH and CAPT_XTRA StatementsBy default, PowerExchange creates all capture files, registrations, and extraction maps in the PowerExchange install directory. Use the following parameters in the PowerExchange DBMOVER configuration file to specify different directories:

Parameter Required Description

CAPT_PATH No PowerExchange stores registrations, capture files such as the CCT and CDCT, condense files, and Condense task checkpoint files in the specified directory. Default is the PowerExchange install directory. See “CAPT_PATH and CAPT_XTRA Statements” on page 11.

CAPT_XRTA No PowerExchange stores capture extraction maps in the specified directory. Default is the PowerExchange install directory. See “CAPT_PATH and CAPT_XTRA Statements” on page 11.

CAPI_SRC_DFLT Yes Specifies the default CAPI_CONNECTION PowerExchange uses when none is supplied. Using CAPI_SRC_DFLT, you can specify default CAPI_CONNECTION statements for different source types in a single configuration file.See “CAPI_SRC_DFLT Statement” on page 12.

CAPI_CONNECTION Yes PowerExchange requires CAPI_CONNECTION statements for real-time extraction mode, continuous extraction mode from condense files, and the Condense task.See “Required CAPI_CONNECTION Statements” on page 12 and “CAPI_CONNECTION Statement for Continuous Extraction Mode” on page 13.

Parameter Description

CAPT_PATH PowerExchange stores registrations, capture files such as the CCT and CDCT, condense files, and Condense task checkpoint files in the specified directory. Create this directory, if it does not exist, prior to using CDC. Default is the PowerExchange install directory.

CAPT_XTRA PowerExchange stores capture extraction maps in the specified directory. Create this directory, if it does not exist, prior to using CDC.Default is the PowerExchange install directory.

Configuring PowerExchange for Change Data Capture 11

Specify these parameters with unique directories to separate CDC objects and files from the PowerExchange code. Separating these data objects and files from the PowerExchange code can ease the software migration process since data objects and files are not inter-mingled with the PowerExchange code.

CAPI_SRC_DFLT StatementYou can code multiple CAPI connection statements in a single DBMOVER configuration file. Use the CAPI_SRC_DFLT to specify the default CAPI_CONNECTION statement for a specific source type. PowerExchange uses the CAPI_SRC_DFLT statement for a specific source type when no CAPI_CONNECTION override is supplied. The following example shows CAPI_SRC_DFLT statements for multiple source types in a single DBMOVER configuration file:

CAPI_SRC_DFLT=(MSQL,sql_name)CAPI_SRC_DFLT=(UDB,db2_name)CAPI_SRC_DFLT=(ORA,uowc_name)

To use continuous extraction mode for condense files, the extraction must use a CAPX CAPI_CONNECTION statement. If the CAPI_SRC_DFLT statement points to a source CAPI_CONNECTION, specify the CAPI connection override for the extraction.

PowerExchange Condense uses source CAPI_CONNECTION statements. If the CAPI_SRC_DFLT statement points to a CAPX CAPI_CONNECTION, specify the appropriate source CAPI_CONNECTION name in the CONN_OVR parameter in the PowerExchange Condense configuration file.

Required CAPI_CONNECTION StatementsPowerExchange uses CAPI connection statements to configure CDC processing. SQL Server CDC requires the following CAPI_CONNECTION statements:

♦ MSQL CAPI_CONNECTION for real-time extraction mode and for PowerExchange Condense

Add the MSQL CAPI_CONNECTION statement to the PowerExchange DBMOVER configuration file, dbmover.cfg, on the machine running SQL Server CDC. For example:

CAPI_CONNECTION=(NAME=sql_name,TYPE=(MSQL,DISTDB=distdb,DISTSRV=distsrv))

Note: PowerExchange does not require CAPI_CONNECTION statements on the PowerExchange Navigator machine. You can register tables for CDC and perform database row test functions in Navigator using the remote PowerExchange Listener running on the machine capturing the changes.

MSQL CAPI_CONNECTION ParametersUse the MSQL CAPI_CONNECTION statement to specify parameters for Microsoft SQL Server CDC.

The MSQL statement has the following syntax:

CAPI_CONNECTION=( NAME=name, [DLLTRACE=trace_id], [TRACE=trace], TYPE=(MSQL, DISTDB=distribution_db, DISTSRV=distrib_srv, [DWFLAGS=(flag1,flag2,flag3)], [EOF=Y|N], [MEMCACHE=cache_value], [POLWAIT=ssss], [RSTRADV=sssss] ))

Parameter Required Default Description

DISTDB Yes none The name of the distribution database.

DISTSRV Yes none The network name of the distribution server.


CAPI_CONNECTION Statement for Continuous Extraction ModeFor continuous extraction mode from condense files, configure CAPX CAPI_CONNECTION statements. For example:

DLLTRACE No none Specifies the name of the TRACE= statement that activates internal DLL tracing for this specific CAPI. Only specify this parameter when instructed to do so by Informatica Global Customer Support.

DWFLAGS No none A series of three positional flags with values of Y or N:Flag1 - Warn on data truncation. Y Forces processing to continue if data of an unexpected length is received. The default is N and to stop processing. Flag2 - Warn on schema change. Y Forces processing to continue if schema changes are detected. The default is N and to stop processing. Flag3 - Warn on change data lost. Y Forces processing to continue if the requested start sequence is not found in the transaction log. The default is N and to stop processing. Only use this parameter when instructed to do so by Informatica Global Customer Support.

EOF No N Specifies whether PowerExchange ends extractions when end-of-log is reached (EOF=Y) or not (EOF=N). Since this parameter affects all users of the CAPI_CONNECTION, use one of the following methods instead:Specify Idle Time=0 on the PWXPC Real Time application connection. - Specify WAITTIME=0 in the ODBC data source.- Valid values are Y or N.

MEMCACHE No 248K Number of Kilobytes to allocate for caching a single row update. Valid values are 1 to 519720.

NAME Yes none Specifies a unique name for the CAPI_CONNECTION statement.Maximum is 8 characters.

POLWAIT No 10 Number of seconds to wait between polling for new data after end of current data has been achieved. Valid values are 1 to 2147483647.

RSTRADV No none Specifies the number of seconds PowerExchanger waits before advancing the restart tokens by returning an empty Unit of Work (UOW). Empty UOWs contain no data, only restart tokens. The RSTRADV wait period for a source starts after a UOW for that source is processed and is reset when the next UOW is received or when an empty UOW is returned because the wait period expires, whichever comes first.If RSTRADV is not specified, then PowerExchange does not return empty units of work to help advance the restart point. For example, if you set RSTRADV=5 and a source in the extraction has not had any changes for ten seconds, an empty UOW will be returned for that source after five seconds. This advances the restart point of that source. Warning: Specifying RSTRADV=0 adversely affects performance. With RSTRADV=0, PowerExchange returns empty UOWs after every UOW processed.Valid values are any number of seconds between 0 and 86400.

TRACE No none Specifies the name of the TRACE= statement that activates the common CAPI tracing. Only specify this parameter when instructed to do so by Informatica Global Customer Support.

TYPE Yes none Specify MSQL.



CAPI_CONNECTION=(NAME=capx_name,TYPE=(CAPX,DFLTINST=collection_id))

Collection_id must match the instance or database name in the PowerExchange Navigator registration group containing the capture registrations.

When using continuous extraction mode for condense files, configure the CAPI_SRC_DFLT statement to point to the MSQL CAPI_CONNECTION statement. PowerExchange Condense does not allow CAPI_CONNECTION overrides so the MSQL CAPI connection statement must be the default CAPI connection statement.

Note: PowerExchange does not require CAPI_CONNECTION statements on the PowerExchange Navigator machine. You can register tables for CDC and perform database row test functions in Navigator using the remote PowerExchange Listener running on the machine capturing the changes.

CAPX CAPI_CONNECTION ParametersThe CAPX CAPI_CONNECTION statement specifies the parameters used for continuous extraction from condense files. Using continuous extraction mode from condense files, extractions run near real-time using condense files as the change stream.

You can use continuous extraction mode from condense files with DB2 for Linux, UNIX, and Windows CDC, Oracle CDC, and SQL Server CDC.

The CAPX statement has the following syntax:

CAPI_CONNECTION=( NAME=name, [DLLTRACE=trace_id], [TRACE=trace], TYPE=(CAPX, DFLTINST=collection_id, [FILEWAIT=nnnnn], [RSTRADV=sssss] ))


DFLTINST Yes none Specifies the PowerExchange instance to process. For DB2 for Linux, UNIX, and Windows, specify the database from the registration group.For Oracle, specify the collection identifier from the ORACLEID statement or the instance from the registration group. For Microsoft SQL Server, specify the instance from the registration group.

DLLTRACE No none Specifies the name of the TRACE= statement that activates internal DLL tracing for this specific CAPI. Only specify this parameter when instructed to do so by Informatica Global Customer Support.

FILEWAIT No 1 Specifies the number of seconds to wait before checking for new condense files.Minimum is 1. Maximum is 86400.

NAME Yes none Specifies a unique name for the CAPI_CONNECTION statement.Minimum is 1 character. Maximum is 8 characters.


Configuring Real-time Extraction ModeComplete the following steps to configure PowerExchange real-time extraction mode:

1. Configure SQL Server. See “Configuring SQL Server for Change Data Capture” on page 9.

2. Configure PowerExchange for CDC. For more information about the statements that you need to define, see the following topics:

♦ “CAPT_PATH and CAPT_XTRA Statements” on page 11

♦ “CAPI_SRC_DFLT Statement” on page 12

♦ “Required CAPI_CONNECTION Statements” on page 12

3. Register tables for SQL Server CDC. For more information, see “Managing Capture Registrations” on page 17.

4. Test the configuration.

5. Configure the PowerCenter extraction by creating mappings and workflows. For more information, see PowerExchange Interfaces for PowerCenter.

6. Establish a starting point in the change stream for the extraction. See “Creating Restart Points with DTLUAPPL” on page 81.

Configuring Batch Extraction ModeComplete the following steps to configure PowerExchange batch extraction mode for condense files:

1. Configure SQL Server. See “Configuring SQL Server for Change Data Capture” on page 9.





♦ “Configuring the PowerExchange Condense Process” on page 30

3. Register tables for SQL Server CDC selecting Part in the Condense option. Delete and recreate any existing capture registrations. For more information, see “Managing Capture Registrations” on page 17.

RSTRADV No none Specifies the number of seconds to wait after receiving a Unit of Work (UOW) for a source in the extraction before advancing the restart tokens by returning an “empty” UOW.Use this parameter to reduce the amount of data which is reprocessed when restarting the extraction.Default is to advance the restart tokens for a source in the extraction when a UOW for that source is read.Minimum is 0. Maximum is 86400.RSTRADV=0 returns empty UOWs regardless of whether the overall restart point has changed. It advances the restart point for every UOW encountered, regardless of whether the overall restart point for the extraction is advanced. Warning: Specifying RSTRADV=0 adversely affects performance.

TRACE No none Specifies the name of the TRACE= statement that activates the common CAPI tracing. Only specify this parameter when instructed to do so by Informatica Global Customer Support.

TYPE Yes none Specify CAPX.



4. Start PowerExchange Condense. For more information, see “Starting and Stopping PowerExchange Condense” on page 35


6. If migrating from real-time extraction mode, see “Migrating to Condense File Extraction Modes” on page 26.


8. Establish a starting point in the change stream for the extraction. For more information, see “Creating Restart Points with DTLUAPPL” on page 81

Configuring Continuous Extraction ModeComplete the following steps to configure PowerExchange continuous extraction mode:

1. Configure SQL Server. For more information, see “Configuring SQL Server for Change Data Capture” on page 9.





♦ “CAPI_CONNECTION Statement for Continuous Extraction Mode” on page 13

♦ “Configuring the PowerExchange Condense Process” on page 30

3. Register tables for SQL Server CDC selecting Part in the Condense option. Delete and recreate any existing capture registrations. For more information, see “Managing Capture Registrations” on page 17.

4. Start PowerExchange Condense. For more information, see “Starting and Stopping PowerExchange Condense” on page 35


6. If migrating from real-time extraction mode, see “Migrating to Condense File Extraction Modes” on page 26.


8. Establish a starting point in the change stream for the extraction. For more information, see “Creating Restart Points with DTLUAPPL” on page 81

Implementing SQL Server Change Data Capture Use the following procedure to implement SQL Server change data capture:

1. Amend the dbmover.cfg file on the registration machine. For more information, see “Configuring PowerExchange for Change Data Capture” on page 10.

2. Configure Microsoft SQL Server Replication. For more information, see “Configuring SQL Server for Change Data Capture” on page 9.

3. Make sure the Microsoft SQL Server Agent service is running.

4. Register the source using the PowerExchange Navigator. For more information, see “Managing Capture Registrations” on page 17.


5. Establish a capture start point. For more information, see “Creating Restart Points with DTLUAPPL” on page 45.

6. Set the PowerExchange capture registration to Active.

Managing Capture RegistrationsTo enable PowerExchange to select the correct changes, you must specify the tables for which the capture collects changes. The registration process consists of a number of tasks, some using the PowerExchange Navigator and others using standard Microsoft SQL Server utilities or Stored Procedures.

Creating a Registration GroupThe registration of the source table is performed using the PowerExchange Navigator. The capture registrations are defined under the Registration Group under Resources on the Resource Explorer panel. A Registration Group is a user-defined group of individual capture registrations.

To add a registration group:

1. Use the toolbar Add Registration Group icon or right-click Registration Group and choose Add Registration Group. The Add Registration Group dialog box appears.


3. Click Next.

4. Enter the following information in the Add Capture Registration - Name and Table Filter dialog box.

Use the Table Filter fields to reduce the number of tables returned. If you do not use a table filter, all tables available within the selected database are returned. Qualify the selection if required.

Field Description

Name The name of the Registration Group. A Registration Group can have multiple capture registrations.

Location The location of the Microsoft SQL Server Distribution server. Pick a location already defined within the Navigator.

Type Select MSSQL.

UserID/Password The user name and password to be used when accessing the Location and the Instance. The Microsoft SQL Server user name must have access to the Microsoft SQL Server location through the Navigator. If the password is left blank the Navigator prompts for the password to be entered as required.

Registration Server Database Server

The name of the server in which the database from which changes will be captured resides. Select from the list.

Registration Server Database Name

The name of the database for which changes are captured. Select from the list.

Field Description

Name Name of this registration within the Registration Group. This is limited to 8 alphanumeric characters.

Table Filter The filter criteria will minimize the actual selection list retrieved from the specified Location and Instance.

Owner Filter on the table owner, table name or no filter. Filter where possible where the database is known to include many tables otherwise a large amount of information will be returned.

Managing Capture Registrations 17

5. Click Next. A list of source tables matching the filter conditions appears.

6. Double-click the table you want to register for data capture.

There can only be one capture registration for a given source.

The columns in the selected table appear. Columns with a key icon are the index columns for that table. To be eligible for data capture, tables must have a primary key.

7. Check Select Column for each column you want to include for capture.

When selecting columns for capture, be sure to include all columns you might want to use. An extraction process can select a subset of the captured columns, but it cannot select data that has not been captured.

Note: Changes to the table structure after registering the columns require updating the capture registration through versioning. For more information, see “Version Indicator in Extraction Definitions” on page 42.

The following table describes the Tables fields that appear in this dialog box:

The following table describes the Columns fields:

8. Click Next. A capture registration can only contain one table.

In the Add Capture Registration - Type dialog box, define the type and characteristics of the capture registration.

9. Enter the following information to define the capture registration:

10. Click Finish. In this example, the registration has been added as Inactive.

If the registration had been created as active, data would be captured from that point onwards.

Table The required table if known.

Escape Character

The escape character that can be used in table names.

Tables Field Description

Creator Name Table creator

Name Table name

Type Database object type

Columns Field Description

Select Columns Indicates whether a column is selected for capture.

NameTC1 Column name

Field Description

Condense Specifies the type of condense processing to do for the capture registration:- None. Indicates that no condense processing is done. - Part. Indicates that condense processing is done and keeps all committed changes for

each source row.

Status Status of the capture registration. This can have one of two values:- Inactive. The registration is to be defined but is not to be used by the Collection Agent.

This may be done so that a registration is prepared ready for starting capture before the target is materialised (initial data load), and ready to switch to active at the appropriate time.

- Active. The registration is effective immediately.

Field Description


For a more controlled method, create the registration as inactive, perform an initial load of target tables, then change the registration to active at the appropriate time.

Each creation of a Registration Group automatically creates a matching extraction group and Application Group with the same name.

When defining an actual capture registration for a source the system automatically defines an extraction process for that registration. These default extract definitions can be modified by the user.

Once a table is registered and the status of the registration is set to Active, the registration cannot be changed. This is to ensure that the data already collected is not invalidated. To change an existing registration, a new version for the table has to be created and activated. Changes required as a result of changes to the source table are handled by versioning of the registrations. For more information, see “Version Indicator in Extraction Definitions” on page 42.

To change the capture registration status:

1. In the Navigator, select the capture registration.

2. In the Resource Inspector, change the Status as necessary. A small red square appears next to Status.

3. To accept the change, click Apply. The Status changes. The red square clears and an icon next to the registration turns red.

4. When exiting the registration, a prompt appears to save your changes. Click Yes.

The Registration saved to the Capture Registration Repository file on the source platform.

5. After the registration is created, you can view it in the Microsoft SQL Server Enterprise Manager.

Adding a Capture Registration to an Existing GroupTo add a capture registration to an existing group:

1. Right click the required Registration Group and click Open.

2. Use the toolbar Add Capture Registration icon. The Add Capture Registration - Name and Table Filter panel appears.

3. Follow Steps 2 to 4 in “Managing Capture Registrations” on page 17.

Deleting a Registration GroupDeleting a Registration Group does not delete any Registration Entries. Only group information held locally on the workstation is deleted. You can delete a registration group if there are no active registrations outstanding for the group.

To delete a registration group:

1. Select the appropriate Registration Group.

2. Click Delete and click Yes to confirm deletion.

3. Click Yes.

Deleting a Registration EntryTo delete a registration entry, select the registration entry and click Delete. If the registration is not active you will be asked to confirm the deletion, otherwise you will be prompted to make the registration history before deleting. This will delete the registration entry on the source platform. Data will no longer be picked up by the extraction process.

Managing Capture Registrations 19

Viewing a Registration GroupSelect a group in the Resource Explorer. This displays group details in the Resource Inspector. A complete list of registrations displays under the Registration Group in the Resource Explorer.

Viewing a Capture RegistrationTo view capture registration details:

1. Double-click a Capture Definition Group from the Resource Explorer Resource tab.

2. Select a capture registration from the group list and then double-click on the Registration Name that you want to view. The following information displays:

Extraction MapsAn extraction process is a process of extracting changes from PowerExchange Change Capture. Captured data can be extracted directly from the log using the CAPXRT access. As part of the capture registration process, an extraction group with the same name is automatically defined. You can modify these default definitions.

Note: A separate Application is created for each extraction. If you use multiple extractions in a single Application, run only one extraction at a time. If you need additional extraction processes, use the Add Extract Definition wizard to define them. When extracting changes for a source the user (PowerCenter or third-party software) specifies a standard SQL request in the format:

Select * from DataMap.Name

This is the same as any other SQL request given to the PowerExchange API. When using the default data map that is being created as part of the capture registration process the request would have the following format:

Area Description

Resource Explorer Displays a list of capture registration groups and the capture registrations within each group. To select and open a capture registration double click on the capture registration name. All the registration details are displayed in the Capture Registration details panels.

Capture Registration Details

Displays the column information of the table referenced by the capture registration. This only displays details for those columns that are selected for capture. This includes:- Name- Table Name- Column No - Type- Length- Scale- KeyTo make changes to an existing list of columns (for a given source) right-click on the registration name.

Resource Inspector Displays the registration properties of the capture registration. For example, Type will be Synchronous and the Tag contents will be used when writing changes to the PowerExchange Logger. If you make any changes they must be confirmed by clicking on the Apply button. Once confirmed, the Registration in the Explorer changes color. This is an indication that the source requires to be saved.


Select * from Dxcollectionid.creator_tablename

Define the Extraction ProcessThe extractions are defined under the extraction group under Resources in the Resource Explorer. By default, when a Registration Group is defined an extraction group with the same name is automatically created. There is normally no need to define extraction groups manually.

To add an extraction group:

1. In the Resource Explorer, right-click Extraction Group and click Add Extraction Group. Or use the Add Extraction Group icon.


3. Click Finish.

To add an extraction definition to an extraction group:

1. In the Resource Explorer, right-click the new extraction group and select Open.

2. Right-click the extraction group name and click Add Extraction Definition.

3. Enter the following information.

Variable Description

D Default entry, all user-defined data maps will start with U.

x 10 for Microsoft SQL Server.

collectionid Collection ID specified when creating the registration group.

Field Description

Name The name of the extraction group.

Location Specifies the name as defined in the dbmover.cfg file where the PowerExchange Listener is located.

Type The type of data to be extracted.

UserID/Password User name and password to identify the user and check their authorizations.

Registration Server Database Server

Select the Microsoft SQL Server database server.

Registration Server Database Name

Select the required Microsoft SQL Server database for capture.

Field Description

Schema Name Name of the schema based on the following naming convention: U_<DB_Type>_<DB_Instance>- U. User Defined.- DB_Type. Database type. Use 10 for Microsoft SQL Server.- DB_Instance. Database instance, for example, docd000.The schema name is unique within a specific PowerExchange Listener, database instance, and database type.

Map Name A name given to that specific extraction definition. The default map name contains the registration name. You can create a map name up to 8 characters long.

Table Name A name given to the table part of the data map name. The default name contains the actual table name. You can create a table name up to 20 characters long.

Extraction Maps 21

The constructed data map name has the format: U10<dbid>.<Mapname_TableName>. Add Extraction Definitions - Capture Registrations.

You can add a version of a capture registration to the extraction process definition using the Add Extract Definitions - Capture Registrations dialog box.

The Available Capture Registrations list displays the capture registrations that are available for selection. The Current Capture Registrations list displays the name of the capture registrations that have been selected for the extraction definition.

In this dialog box, you can manage the versions of registrations associated with the extraction process. For example, you could remove a version 1 registration and then add the next version.

Data Delivery FormatPowerExchange captures data and saves it as Before and After Images in the PowerExchange Logger.

The way this captured data is delivered is based on the Image Type defined in:

♦ CAPXImageType through ODBC

♦ SQL Escape Sequence DTLIMTYPE

The following table describes how captured update data can be delivered:

Note: Inserts and deletes are delivered without before images.

Table 4-1. Available Capture Registrations Fields

Field Description

Name Displays the name of the capture registrations that are available for selection.

Version Displays the version number of the capture registration.

Condense Displays the condense state of the capture registration.

Status Displays the status of the capture registration. This can have one of two values:- I. Inactive. The registration is to be defined but not used by the Collection Agent.- A. Active. The registration is to be defined and used by the Collection Agent.

Table 4-2. Current Capture Registration Fields

Field Description

Name The name of the capture registration that has been selected.

Version The version number of the capture registration.

Condense The condense state of the capture registration.

Status The status of the capture registration.

Image Type Description

BA Delivers the Before Image (before the latest change) and the After Image (the latest change). The Before Image is shipped with an action character forcing the deletion of the record. The After Image (which includes the changes) is in the format of an Insert.

AI Delivers only the After Image data for the update.

TU Delivers the data before and after images as they occur. The Before Image is shipped to enable the application to ensure nothing has changed in that record since the request. The After Image comes through as a change (or Update) record.The update is still a full image but the application picking this up will be able to change the target record rather than Delete it and Insert the new one as when using BA.


Specify Extraction CriteriaThere are a variety of values that can be specified when running an extraction process. When using the ODBC interface to PowerExchange, you define them as part of the ODBC definition. ODBC allows various overrides using escape sequences:

♦ DTLXTYPE=

♦ DTLAPP=

♦ DTLDSN=

Note: The overrides below are fully detailed in the PowerExchange Reference Manual.

Deleting an Extraction Group or DefinitionTo delete an extraction group:

1. Select the appropriate extraction group.

2. Click Delete, and click Yes to confirm the deletion.

Deleting the extraction group does not delete any extraction definitions. It only removes the group-related information.

To delete an extraction definition:

Select the Extract Definition and then press the Delete button.

This deletes the Extract Definition entry. Once the entry is deleted it is not possible to extract any more changes using that extraction definition. Care has to be taken to ensure that the back end application is changed in correspondence. This will invalidate the source definition in, for example, the ETL tools.

Viewing an Extraction Definition

To view the extraction definition details:

1. On the Extraction Group tab of the Resource Explorer, double-click the extraction group that contains the extraction definition.

2. Select an Extract Definition from the group contents and double-click the Registration Name.

− Resource Explorer. Displays a list of extraction groups and the extraction definitions within each group. To select and open an extraction definition, double-click on the extraction name. All the extraction details are displayed in the Extract Definition details panels.

− Extract Definition. Displays a list of extract definitions. It is possible to remove columns from the Extract Definition by clearing the associated box. By default, the automatically generated columns are not

Parameter Value Parameters Description

DTLAPP Name Allows the you to specify the capture extract application ID without having to define a new data source when accessing PowerExchange (CAPXRT) data through ODBC.

DTLDSN DSN Allows the SQL override of the physical file name coded in the data map. Allows you to use ODBC to read NRDB and NRDB2 data from various files through one ODBC data source. For ODBC this is coded anywhere in the SQL as {DTLDSN=xxx}.

DTLXTYPE/Type SL None Extract all new captured data since the last extraction.

RS None Default DTLXTYPE. Run the last-specified extraction process. This is either the last extraction that was run or the one that is specified with the Reset Start Point Wizard in the Application dialog box. For more information about Application Groups, see “Application Group” on page 24.

Extraction Maps 23

displayed. Click the mouse right button to display these columns. They can be selected as part of the extract definition. Click File > Save to save any changes.

− Extraction Details. Displays the Name, Version and referenced Table of the extraction definition.

− Table Details. Displays the column information of the table referenced by the extraction definition. This includes:

♦ Name

♦ Type

♦ Precision

♦ Scale

♦ Length

You can right-click the Table Details area of the Extract Definition dialog box and select Show Auto Generated Columns from the menu. The following columns appear:

In addition, null indicator hidden columns are shown adjacent to nullable columns and a length indicator when these are of variable length.

− Resource Inspector. The Extraction Group tab displays the connection properties of the extraction definition. If you make any changes, use the Apply button to confirm your changes.

Application GroupYou can display the information for a given extraction application and reset the status of the extraction process to allow for another run.

Column Description Data Type Length

DTL__CAPXRESTART1 Coded position of the extracted data in the extraction log.

VARBIN 255

DTL__CAPXRESTART2 Coded start point for the oldest unit-of-work. VARBIN 255

DTL__CAPXUOW Coded position of the start of the commitment control for the unit-of-work for this update.

VARBIN 255

DTL__CAPXUSER Value is always null. Microsoft SQL Server does not provide this information.

VARCHAR 255

DTL__CAPXTIMESTAMP The time the change is captured by PowerExchange. The time displays in the following format:YYYYMMDDHHMMSSnnnnnn (where nnnnnn=microseconds)

CHAR 20

DTL__CAPXACTION The change capture action: I for insert, D for delete, or U for update.

CHAR 1

DTL__BI The Before Image of the changed field is retrieved in addition to the normal retrieval of the After Image. For more information about DTL__BI, see the PowerExchange Navigator User Guide.

Has the data type and length of the source column.

DTL__CI You can add a change indicator (CI) to each column in which you are interested. For Updates this indicates if the column has changed: Y for yes, N for no. Inserts and Deletes always returns a null value.For more information about DTL__CI, see the PowerExchange Navigator User Guide.

CHAR 1


You can view an audit trail of all the extractions that have run, their status and an indication of the captured data that has been moved.

The Application Group is based on the same principle as the extraction and registration groups. The group level identifies the location of the PowerExchange Listener task. There is no need for a database instance type or name. The Resource Inspector displays the following:

♦ Name

♦ Location

♦ UserID

♦ Password

The Application Name list is based on the application name on the extraction run.

Click on an application name to display a list of all successful and unsuccessful extractions. Extractions appear in order of execution, starting with the most recent. Click on the timestamp to display details for the extraction run, including registration name, table name, and the CND files that were read.

Right-click an application name to displays a list of possible actions for the application name. Select a timestamp for a given application and then right-click to display a list of options.

Reset to a New Start PointThe Application tracks the extractions against a registration and stores the start point for the next run. There are occasions when one or more extractions need to be run again. You can do this by resetting the start point for the application.

This feature is used to reset the start point of the next extraction run for the selected application. It resets the start point to the start point of the open timestamp. You can then perform one of the following processes:

♦ Restart the application at the start of the selected run.

♦ Reset the application again.

Note: This is a powerful feature and can not be undone. If in doubt, use DTLUAPPL. Tokens can be printed for copy/paste using DTLUCDEP.

The following examples use an application MSSQLRT, which has had three extractions run against it. Because this is for demonstration purposes only, the extract times are very close together.

To restart to a new start point:

1. To open the required application, double-click the Application Group, and then select an application.

Note the order of runs shown in the right hand pane. The first and last runs show the time of the first and last runs and the Current Run is blank.

2. Highlight the run to which the start point is to be reset. In this case the earliest of the two runs is selected.

3. To select the Application Group name, right-click and select Reset To New Start Point. Note how the icon to the left of the application group turns red. The restart point is reset to the start point of the chosen run.

4. Save the change. The Current Run information is replaced by the run information from the second extraction.

XTYPE Description

SL Extract all new captured data since the last extraction.

RS Run the last-specified extraction process again. This is either the last extraction that was run or the one that is specified in the Application dialog box.

Application Group 25

Migrating to Condense File Extraction ModesYou can migrate real-time extractions to batch or continuous extraction modes. Consider the following:

♦ PowerExchange Condense uses the capture registration to parse the change data it writes to the condense files. During the capture registration creation process, PowerExchange records version information in the capture registration.

PowerExchange Condense writes the changes to the condense files in a different format if the capture registration is created using PowerExchange 8.5.x. This format change allows Condense to support multi-byte character sets. To prevent any future extraction incompatibilities, recreate any registrations enabled for Condense after installing PowerExchange 8.5.x.

♦ Restart tokens differ between real-time, batch, and continuous extraction mode.

In order to maintain positioning within the change stream, PowerExchange constructs restart tokens. These restart tokens are used to re-establish connection to the change stream to the point of interruption when extractions are restarted.

PowerExchange creates a one format of restart tokens for real-time and batch extraction mode and a different format for continuous extraction mode. These different formats of restart tokens are not compatible. Switching an extraction from one mode to another requires a migration process.

Use the following procedures to migrate between extraction modes.

To migrate from real-time to batch extraction mode:

1. Configure PowerExchange Condense. For more information, see “Configuring the PowerExchange Condense Process” on page 30.

2. Quiesce write activity to all tables switching from real-time to batch extraction mode.

3. Extract all captured changes and then stop the extraction.

4. Run DTLUAPPL with RSTTKN GENERATE to get restart point.

Configure DTLUAPPL to use the MSQL CAPI_CONNECTION statement. If the PowerExchange configuration file is configured with this CAPI statement as the default, no overrides are necessary when using DTLUAPPL. Otherwise, use the CONN_OVR parameter and point to the MSQL CAPI_CONNECTION statement.

5. Reinstate write access to all quiesced tables.

6. Delete and recreate the capture registration specifying Condense Part.

7. Update the RESTART_TOKEN and SEQUENCE_TOKEN statements in the PowerExchange Condense parameter file, dtlca.cfg, with restart information generated by DTLUAPPL. For more information, see “Configuring PowerExchange Condense Parameters” on page 32.

8. Cold start PowerExchange Condense.

9. Change the extraction from real-time to batch extraction mode.

If using PWXPC, change the session to use a PWX CDC Change connection instead of a PWX CDC Real Time connection. For more information, see PowerExchange Interfaces for PowerCenter.

If using ODBC, update the ODBC data source and change the DBTYPE from CAPXRT to CAPX. For more information, see the PowerExchange Reference Manual.

10. Cold start the PowerCenter extraction session.

To migrate from real-time to continuous extraction mode:

1. Configure the Condense task. For more information, see “Configuring PowerExchange Condense” on page 36.

2. Quiesce write activity to all tables switching from real-time to continuous extraction mode.

3. Extract all captured changes and then stop the extraction.


4. Run DTLUAPPL with RSTTKN GENERATE to get restart points.


5. Reinstate write access to all tables that were quiesced.

6. Delete and recreate the capture registration specifying Condense Part.

7. Update the RESTART_TOKEN and SEQUENCE_TOKEN statements in the Condense parameter file, dtlca.cfg, with restart information generated by DTLUAPPL for the Condense task. For more information, see “Configuring PowerExchange Condense Parameters” on page 32.

8. Update the PowerExchange configuration file, dbmover.cfg, with the CAPX CAPI_CONNECTION. For more information, see “CAPX CAPI_CONNECTION Parameters” on page 14.


10. Change the extraction from real-time to continuous extraction mode.

If using PWXPC, update the PWX CDC Real Time connection in the session to specify the name of the CAPX CAPI_CONNECTION statement in the CAPI Connection Name Override attribute. For more information, see PowerExchange Interfaces for PowerCenter.

If using ODBC, override the ODBC data source with the name of the CAPX CAPI_CONNECTION statement using the SQL Escape Sequence DTLCONNOVR. For more information, see the PowerExchange Reference Manual.


To migrate from batch to continuous extraction mode:

1. Stop the Condense task.

2. Extract all captured changes and then stop all extractions.

3. Run DTLUAPPL with RSTTKN GENERATE to get restart point.


4. Update the RESTART_TOKEN and SEQUENCE_TOKEN statements in the Condense parameter file, dtlca.cfg, with restart information generated by DTLUAPPL for the Condense task. For more information, see “Configuring PowerExchange Condense Parameters” on page 32.

5. Update the PowerExchange configuration file, dbmover.cfg, with the CAPX CAPI_CONNECTION. For more information, see “CAPX CAPI_CONNECTION Parameters” on page 14.


Note: PowerExchange Condense deletes all existing condense files when cold started. Extract all condensed data for all tables before using cold start.

7. Change the extraction from batch to continuous extraction mode.

If using PWXPC, change the session to use a PWX CDC Real Time connection instead of a PWX CDC Change connection. Specify the name of the CAPX CAPI_CONNECTION statement in the CAPI Connection Name Override attribute. For more information, see PowerExchange Interfaces for PowerCenter.

If using ODBC, update the ODBC data source changing the DBTYPE from CAPX to CAPXRT. Override the ODBC data source with the name of the CAPX CAPI_CONNECTION statement using the SQL Escape Sequence DTLCONNOVR. For more information, see the PowerExchange Reference Manual.


Migrating to Condense File Extraction Modes 27


C H A P T E R 5

PowerExchange Condense


♦ PowerExchange Condense Overview, 29

♦ Configuring the PowerExchange Condense Process, 30

♦ Configuring PowerExchange Condense Parameters, 32

♦ Starting and Stopping PowerExchange Condense, 35

PowerExchange Condense OverviewPowerExchange Condense captures changes from transactional databases, condenses the changes, and then writes the changes to condense files. PowerExchange Condense parameters control the frequency with which the data is condensed.

Changes are partially condensed into chronological order without eliminating any of the individual changes made to a record. This method is called partial condense.

PowerExchange extracts changes from condense files using one of the following modes:

♦ Batch extraction mode for condense files using the CAPX access method

♦ Continuous extraction mode for condense files using the CAPXRT access method

Making Capture Registrations Eligible for Condense ProcessingPowerExchange Condense only condenses data for capture registrations that request condense processing. Request condense processing for a capture registration by selecting Part for the condense option in the capture registration. For more information about capture registrations, see “Managing Capture Registrations” on page 17.

PowerExchange Condense requires active capture registrations with condense specified. If no capture registrations meet these criteria, PowerExchange Condense issue a message indicating that there are no active registrations and ends. For more information about modifying existing capture registrations to take advantage of the condense option, see “Extraction Maps” on page 20.

29

Configuring the PowerExchange Condense ProcessPowerExchange Condense is composed of the following subtasks:

♦ Controller. This subtask starts and stops the other subtasks and controls PowerExchange Condense processing.

♦ Condense. This subtask read changes from the change stream, condenses the change data, and writes the changes to condense files.

♦ Command handler. This subtask provides the command interface to the condense subtask.

♦ Dump. This subtask provides dump services to the condense subtask.

The PowerExchange log contains messages indicating when the various tasks start and end and also from which task a message is being issued.

PowerExchange Condense Operational ModesPowerExchange Condense has the following modes of operation:

♦ Batch

♦ Continuous

Set the mode of operation using the COLL_END_LOG parameter. For more information, see “Configuring PowerExchange Condense Parameters” on page 32.

PowerExchange Condense Batch ModeIn batch mode, a single condense cycle is run and then PowerExchange Condense shuts down. Running PowerExchange Condense in this fashion is well suited to batch extraction applications.

Specify batch mode for PowerExchange Condense by coding COLL_END_LOG=1 in the Condense configuration file.

PowerExchange Condense Continuous ModeIn continuous mode, PowerExchange Condense runs the condense subtask continuously for an extended period of time. The condense subtask pauses after each condense cycle. The next condense cycle is triggered by one of the following:

♦ The NO_DATA_WAIT period expires.

♦ A CONDENSE command is manually entered.

♦ A FILESWITCH command is manually entered.

♦ A SHUTCOND command is manually entered.

In continuous mode, PowerExchange Condense does not shut down. You must shut it down using the SHUTDOWN or SHUTCOND command.

Specify continuous mode for PowerExchange Condense by coding COLL_END_LOG=0 in the Condense configuration file.

File Switch ProcessingRunning in continuous mode, PowerExchange Condense periodically closes condense files and then starts new ones. This process is called file switch processing. File switch processing closes the open condense files if they contains data and opens a new set of condense files for future changes. PowerExchange Condense automatically does a file switch when the criteria defined by the FILE_SWITCH_CRIT, FILE_SWITCH_VAL, and the FILE_SWITCH_MIN configuration parameters are met. File switch also occurs if you manually issue a FILESWITCH command.

30 Chapter 5: PowerExchange Condense

Note: PowerExchange Condense does not file switch if no data is present in the current condense file. The file switch is skipped until the next interval. The file switch continues to be skipped until an interval expires and there is data in the condense file.

PowerExchange Condense Output FilesPowerExchange Condense writes output files to the directory specified by the CAPT_PATH parameter in the PowerExchange dbmover.cfg configuration file. If no CAPT_PATH parameter is specified, PowerExchange Condense writes the output files to the directory from where it is invoked.

CDCT FileThe CDCT file is updated by PowerExchange Condense. The PowerExchange Listener reads the CDCT when extracting data.

After each file switch, PowerExchange Condense writes records to the CDCT containing information about each condense file. These keyed records contain information about the data that has been condensed such as the condense file name, the start and end times, whether before images are included, the number of records in the file, and other control information.

For example, if a partial condensed file contains records for two registration tags, then PowerExchange Condense writes three records to the CDCT. All records show the name of the file. Two of the records show the record count for each registration tag. The other record has no registration tag and counts the number of “commit” records in the file, one for each unit of work.

CDCT records match DCT records on the Checkpoint file. PowerExchange Condense verifies the records match during warm start processing. If they do not match, PowerExchange Condense adjusts the CDCT by either inserting or deleting records as appropriate.

Condense FilesPowerExchange Condense creates condense files during the condense cycle if there are captured changes to write. The condense files contain the change data for the active registrations PowerExchange Condense found during its initialization. PowerExchange Condense customizes the names of the condense files using the value specified in EXT_CAPT_MASK parameter in the Condense configuration file, dtlca.cfg.

PowerExchange Condense creates the partial condense files using the following naming convention:

hlq.CND.CPyymmdd.Thhmmssn

where:

hlq = EXT_CAPT_MASK valueyymmdd = datehhmmss = timen = sequence number starting at 1 (for uniqueness)

You can use PowerCenter to extract the changes from the condense files. Use one of the following modes to read condense files:

♦ Batch extraction mode using PowerCenter PWX Change application connections

♦ Continuous extraction mode using PowerCenter PWX Real Time application connections

For more information about using PowerCenter to extract changes, see PowerExchange Interfaces for PowerCenter.

For more information accessing condensed data, see “Accessing CDC Data” on page 45.

For information about configuring PowerExchange, see “Configuring PowerExchange for Change Data Capture” on page 10.

Configuring the PowerExchange Condense Process 31

Checkpoint FilesThere must be at least two checkpoint files used by the system with allocation parameters defined in the Condense configuration file, dtlca.cfg.

Each time the system starts up, a fresh checkpoint records the results of merging the current registrations from the CCT file with the information from the latest checkpoint. If it is a Cold Start, then no Checkpoint files are brought forward.

After each file switch or shut down, a fresh checkpoint is taken.

The following table describes the information stored in Checkpoint files:

Configuring PowerExchange Condense ParametersTable 5-1 describes the required parameters for the PowerExchange Condense configuration file, dtlca.cfg:

Checkpoint Record Type Description

ERT records Registration tags and restart tokens that indicate the point to start receiving records from the change stream.

DCT records Information that is also held in the CDCT file, describing completed Condensed files. This information is purged based on the COND_CDCT_RET_P parameter in the Condense configuration file, dtlca.cfg.This record type enables you to restore the CDCT file to a point consistent with a restart.

SRT record A single record defining system-wide information.

Table 5-1. Required Parameters

Parameter Description Valid Values

CAPT_IMAGE Specifies whether before or after images are recorded. AI = After images.BA = Both before and after images.

CHKPT_NUM Specifies the number of checkpoint files.Warning: Decreasing CHKPT_NUM on a warm start can result in restart from an incorrect location. Only decrease CHKPT_NUM if doing a cold start.Default is 3.

1-999999

CHKPT_BASENAME Base name for checkpoint files.To form the full checkpoint file name, Vn is appended, where n is 0 to CHKPT_NUM-1.If a fully-qualified path is not specified, the checkpoint files are created in the PowerExchange directory.

For example, INFA.D.CHKPTMaximum length is 256.

COLL_END_LOG - 0. Continuous mode. After each condense operation, PowerExchange Condense waits the number of minutes defined in the NO_DATA_WAIT parameter, then performs another condense operation.

- 1. Batch mode. PowerExchange Condense shuts down after a single condense operation.

0 = Continuous1 = Batch

CONDENSE_SHUTDOWN_TIMEOUT

Specifies the maximum time period that PowerExchange Condense waits after receiving the SHUTDOWN command before stopping. PowerExchange Condense updates the CDCT for each capture registration during shutdown. With large numbers of capture registrations, increasing the timeout period may be necessary.

0 to 2147483647 secondsDefault is 600 seconds.


COND_CDCT_RET_P CDCT and Condensed files retention period in days. Files older than this period and their corresponding CDCT records are deleted during start-up, fileswitch, or shut down processing. Be sure to allow enough days for the files to be processed before they are deleted.

Any number > 0

CONN_OVR Specifies the CAPI_CONNECTION name to use when running PowerExchange Condense. Otherwise, PowerExchange Condense uses the default connection.

Valid source CAPI connection name.

DBID Specifies the instance name. In conjunction with DB_TYPE, DBID defines selection criteria for capture registrations in the CCT file.It must match the database name displayed in the PowerExchange Navigator for the Registration Group containing the capture registrations.

Database name used for capture registrations

DB_TYPE Database type. MSS

EXT_CAPT_MASK Unique mask for files created by the Condense. This should be specified in upper case.EXT_CAPT_MASK gives the condense process complete control over any condense files that match the specified EXT_CAPT_MASK. Warning: Do not share EXT_CAPT_MASK between multiple condense processes. Because each process considers all files its own, it might delete files associated with other condense processes.Do not re-use an EXT_CAPT_MASK until all files that match the mask have been finished with.Any file names that match the mask, even if unrelated to condense, are considered condense files by PowerExchange. Make sure your EXT_CAPT_MASK does not match any other files on the system.

For example: INFA.D Maximum length is 256 characters

FILE_FLUSH_VAL Specifies the file flush interval in seconds.The file flush interval is the number of seconds that elapse before a flush is performed on the current partial condense file. When the condense subtask flushes, the data is written to the disk condense files allowing it to be read by continuous extraction mode extractions.Valid values are:-1 causes the condense subtask to not flush data to the current file. Use this setting, or allow it to default, if using batch extraction mode for condense files. Change this setting if using continuous extraction mode for condense files. Otherwise, latency increases for continuous extraction mode extractions. This value is the default value.0 causes Condense to flush after every record. Warning: Specifying 0 impacts the condense subtask and file system performance.1 to 86400 sets the file flush interval in seconds.

-1 to 86400 seconds

FILE_SWITCH_CRIT Defines the criteria to use when deciding when to do an automatic file switch. Use in conjunction with FILE_SWITCH_VAL and FILE_SWITCH_MIN.Default is M.

M = MinutesR = Records



Configuring PowerExchange Condense Parameters 33

FILE_SWITCH_MIN Specifies file switch criteria for Condense when changes for new sources are encountered. Use this to reduce latency for continuous extraction mode. Format is:(min_val,min_val_ign)Where:min_val specifies the minimum number of FILE_SWITCH_CRIT units that must pass before a file switch is done when encountering a change for a source with no entry in the CDCT. Valid values:-1 causes FILE_SWITCH_MIN to be ignored. File switch processing is then controlled by FILE_SWITCH_VAL only.0 causes the condense subtask file to switch for every new source.1 to 2147483647 causes the condense subtask file to switch every n units.min_val_ign specifies the number of FILE_SWITCH_CRIT units which must occur during cold start processing before Condense uses the min_val value. Until min_val_ign is exceeded, FILE_SWITCH_CRIT alone controls file switch activity. Valid values:0 causes the condense subtask to use the minimum file switch value specified in min_val immediately on cold start. 1 to 2147483647 causes the condense subtask file to switch every n FILE_SWITCH_CRIT units.The min_val_ign is not used when Condense is warm started.Default is (-1,0).Warning: Specifying (0,0) results in large numbers of file switches during cold start, when many sources are registered in the CDCT.

min_val:-1 to 2147483647min_val_ign:0 to 2147483647

FILE_SWITCH_VAL Defines the number of FILE_SWITCH_CRIT units at which to perform a file switch automatically. Default is 30.For example, if this value is 30, the condense subtask file switches every 30 records if FILE_SWITCH_CRIT=R or every 30 minutes if FILE_SWITCH_CRIT=M. If the condense file is empty of data at the FILE_SWITCH_VAL interval, then no file switch takes place.

Any number > 0

NO_DATA_WAIT When Condense runs in continuous mode, it defines the number of minutes to wait on commands manually entered through the Command Handler before starting the next Condense. Typically, the system might wait for 60 minutes before starting the next Condense.If file switching is performed based on time and FILE_SWITCH_VAL is smaller than NO_DATA_WAIT, then the wait period is reduced to the smaller of the two values.When the system runs in Batch mode (COLL_END_LOG = 1), this parameter is not used.Default is 60 minutes.

Any number > 0




Starting and Stopping PowerExchange Condense

Starting PowerExchange CondenseUse the following command to start PowerExchange Condense:

dtlcacon [config=config_file] [cs=cond_cfg_file]

NO_DATA_WAIT2 Defines the number of seconds before the condense stops.The completion of a Condense run occurs when this period expires without data being provided.The optimum value for setting the parameter varies according to the loading of the system.- If the parameter is set too low, the Condense might report that

no data when data exists. A delay might occur if a large unit of work is started, such as one containing several thousand rows.

- If the parameter is set too high, then an excessive period of apparent inactivity elapses before control returns to the Command Handler, allowing you to enter commands such as FILESWITCH or SHUTDOWN.

- 60 seconds is recommended.- Default is 600 seconds.

Any number > 0

PWD Password used to identify the user (see UID) and check their authorizations.

String

RESTART_TOKEN and SEQUENCE_TOKEN

Every registration in the Change Data Extraction Repository file contains a restart token. The token has two parts: a sequence and a restart token. - RESTART_TOKEN - SEQUENCE_TOKEN The restart tokens can be provided at application level, registration level, or both. For more information, see “Creating Restart Points with DTLUAPPL” on page 81.

SIGNALLING - SIGNALLING = Y. The system takes automatic action in the event of certain errors, such as memory corruption. The system attempts to close down in an orderly manner.

- SIGNALLING = N. No automatic trapping of system errors is performed. The operating system uses default error handling, which is usually to report the offending program line and dump memory.

- Default is N.

Y = Attempt to close down after error.N = Abort on error.

UID User ID to identify the user and check their authorizations.This user ID must be a valid database user ID that has authority to access database resources.

String

VERBOSE Specifies whether PowerExchange Condense issues verbose or terse messages for frequent condense activity like cleanup, checkpoints, condense cycles, and file switch processing. Valid values:- Y for verbose messaging- N for terse messaging- Default is Y.

Y or N



Starting and Stopping PowerExchange Condense 35

[license=license_file]

For more information about controlling PowerExchange Condense, see PowerExchange Command Reference.

The following rules and guidelines apply:

♦ To cold start, remove all checkpoint files defined by the PowerExchange Condense configuration file parameter CHKPT_BASENAME.

♦ Cold start behaves differently depending upon the setting of the RESTART_TOKEN and SEQUENCE_TOKEN in the dtlca.cfg file:

− If the RESTART_TOKEN and SEQUENCE_TOKEN are not present in the dtlca.cfg parameters, then PowerExchange Condense starts from the current position.

− If the RESTART_TOKEN and SEQUENCE_TOKEN are set to zero in the dtlca.cfg parameters, then PowerExchange Condense starts from the oldest data available in the Publication database.

− If the RESTART_TOKEN and SEQUENCE token are to be set to a specific value, then PowerExchange Condense starts from the value specified.

♦ Create capture registrations using the PowerExchange Navigator for those sources for which change data capture is required.

♦ Edit dtlca.cfg to change the appropriate parameters. For more information, see “Configuring PowerExchange Condense Parameters” on page 32.

♦ Start PowerExchange Condense.

Stopping PowerExchange CondenseWhen running in foreground, use one of the following commands to stop PowerExchange Condense:

♦ SHUTDOWN. PowerExchange Condense performs a file switch, updates the CDCT, takes a checkpoint, and ends.

♦ SHUTCOND. PowerExchange Condense starts a final condense cycle. When the condense cycle ends, PowerExchange Condense performs a file switch, updates the CDCT, takes a checkpoint, and ends.

When running in background on Linux or UNIX, PowerExchange Condense processes the KILL command as SHUTDOWN.

For more information about controlling PowerExchange Condense, see PowerExchange Command Reference.


config_file Optionally specifies the DBMOVER configuration file name, including the full path.Default is <PWX_install_dir)/dbmover.cfg.

cond_cfg_file Optionally specifies the PowerExchange Condense configuration file name, including the full path.Default is <PWX_install_dir)/dtlca.cfg.

license_file Optionally specifies the license key file name, including the full path.Default is <PWX_install_dir)/license.key


C H A P T E R 6

Managing Change Data Capture Definitions


♦ Overview, 37

♦ Changing the Underlying Data Source, 37

♦ Changing Extraction Maps, 41

♦ Changing Properties of Capture Registrations, 43

OverviewAfter a source table has been registered for change data capture, changes may be required to accommodate business and application changes and schema changes to the source table. The following types of changes can affect change propagations:

♦ Schema changes to the source table

♦ Changes to the capture registration

♦ Changes to the extraction map

Changing the Underlying Data SourceSchema changes for tables registered for change data capture may require changes to the capture registration and extraction map. You can change, add, and remove unregistered columns in tables registered for capture without stopping any extraction precesses or making changes to the capture registration or extraction map. Changes to registered columns or additions to registered columns require PowerExchange changes.

Warning: PowerExchange cannot process schema changes logged by Microsoft SQL Server when stored procedures sp_repladdcolumn and sp_repldropcolumn are used to make schema changes. Use DDL to change the schema for SQL Server tables.

Adding Columns to Capture RegistrationsUse the following procedure to add a new column to an existing capture registration.

37

To add a column to a capture registration:

1. Stop any jobs or applications that update the SQL Server table.

2. Run all PowerCenter workflows that extract changes for the table. Shut down the workflows when all changes have been extracted.

3. Use DDL to perform the schema change to the SQL Server table.

4. In PowerExchange Navigator, click the registrations groups folder.

5. In the Resource Explorer, double-click to open the registration group containing the capture registration for the SQL Server table.

6. In the Resource Explorer, double-click to open the current capture registration for the table.

7. Right-click the capture registration, and then click Amend Columns.

8. In the Amend Columns dialog box, select the new column from the list of available columns and click Add.

The new column now appears in the Current Columns list.

9. Click OK. The new column displays in the middle window at the bottom of list of columns for the capture registration.

10. On the menu bar, click File > Save.

11. PowerExchange displays a dialog box containing message PWX-01742 to indicate that a new inactive version of the capture registration will be created. Click Yes.

Resource Explorer displays the new version of the capture registration, which is inactive. The current capture registration remains in active status. For information about version indicators, see “Version Indicator in Extraction Definitions” on page 42.

12. If necessary, re-populate the source SQL Server table.

13. In Resource Inspector, click Status > Active and then click Apply to activate the new capture registration.


15. PowerExchange displays a dialog box containing message PWX-01743. Click Yes.

PowerExchange changes the status of the new capture registration to active and the status of the existing capture registration to history.

16. Change the extraction map to use the new version of the capture registration. For details on changing the extraction map, see “Changing Extraction Maps” on page 41.

17. If necessary, change the schema of the target tables to which the changes from this source table will be applied.

18. Use the Designer to import the altered source and target tables into PowerCenter. Make any necessary changes to the mappings that contain the altered source and target tables.

19. If necessary, rematerialize the target tables. Run PowerExchange utility DTLUAPPL to generate new restart tokens once the materialization of the target tables complete. For further information about using DTLUAPPL, see the PowerExchange Utilities Guide. For information about updating the restart token files with the restart tokens generated by DTLUAPPL, see PowerExchange Interfaces for PowerCenter.

20. Restart any jobs or applications that update the source table.

21. Restart the PowerCenter workflows.

Removing Columns from Capture RegistrationsIf a column is no longer required for capture but is still required in the source table, you can remove it from the extraction map without changing the capture registration. Change data is still be captured for the column but, once the column is removed from the extraction map, no changes for the column data will be extracted. For more information about changing extraction maps, see “Changing Extraction Maps” on page 41.

38 Chapter 6: Managing Change Data Capture Definitions

If a column is no longer required for capture or in the source table, use the following procedure to remove the column from the table and an existing capture registration.

To remove a column from a capture registration:





5. In the Resource Explorer, double-click to open the current capture registration for the table.


7. In the Amend Columns dialog box, select the column from the list of current columns and click Remove.

The column now appears in the Available Columns list.

8. Click OK.



Resource Explorer displays a new version of the capture registration, which is inactive. The current capture registration remains in active status. For information about version indicators, see “Version Indicator in Extraction Definitions” on page 42.

11. In the Resource Explorer, double-click to open the current active capture registration.

12. In Resource Inspector, click Status > History and then click Apply to change inactivate the current capture registration.

13. On the menu bar, click File > Save to save the changes to current capture registration.



16. In the Resource Explorer, double-click to open the new capture registration.



PowerExchange now collects change data for the new capture registration with the column removed.

19. Change the extraction map to use the new version of the capture registration. For more information about changing the extraction map, see “Changing Extraction Maps” on page 41.



22. If necessary, rematerialize the target tables. Run PowerExchange utility DTLUAPPL to generate new restart tokens once the materialization of the target tables complete. For more information about using DTLUAPPL, see the PowerExchange Utilities Guide. For more information about updating the restart token files with the restart tokens generated by DTLUAPPL, see PowerExchange Interfaces for PowerCenter.



Changing the Underlying Data Source 39

Modifying Columns in Capture RegistrationsUse the following procedure to modify a column in a table and an existing capture registration.

To modify a column in a capture registration:





5. In the Resource Explorer, double-click to open the capture registration for the table.


7. In the Amend Columns dialog box, select the column from the list of current columns and click Remove.

The column now appears in the Available Columns list.

8. Click OK.



Resource Explorer displays a new version of the capture registration, which is inactive. The current capture registration remains in active status. For information about version indicators, see “Version Indicator in Extraction Definitions” on page 42.

11. In the Resource Explorer, double-click to open the current active capture registration.

12. In Resource Inspector, click Status > History and then click Apply to inactivate the current capture registration.

13. On the menu bar, click File > Save to save the changes to current capture registration.



16. In the Resource Explorer, double-click to open the new capture registration for the table.


18. In the Amend Columns dialog box, select the column that changed from the list of available columns and click Add.

The column now appears in the Current Columns list.

19. Click OK.




PowerExchange now collects change data for the new capture registration with the changed column.

23. Change the extraction map to use the new version of the capture registration. For more information about changing the extraction map, see “Changing Extraction Maps” on page 41.




26. If necessary, rematerialize the target tables. Run PowerExchange utility DTLUAPPL to generate new restart tokens once the materialization of the target tables complete. For more information about using DTLUAPPL, see the PowerExchange Utilities Guide. For information about updating the restart token files with the restart tokens generated by DTLUAPPL, see PowerExchange Interfaces for PowerCenter.



Changing Extraction MapsYou can alter extraction maps to select a new capture registration, select a new version of a capture registration, or to remove columns. For more information about capture registration versions, see “Version Indicator in Extraction Definitions” on page 42.

Associating Capture Registrations with Extraction MapsUse the following procedure to associate a new capture registration or new version of a capture registration with an extraction map.

To change an extraction map to associate a different capture registration:

1. In PowerExchange Navigator, click the extraction groups folder.

2. In the Resource Explorer, double-click to open the extraction group containing the extraction map to be modified.

3. In the Resource Explorer, double-click to open the extraction map.

4. Right-click the extraction map, and then click Amend Capture Registrations.

5. In the Amend Capture Registrations dialog box, select the capture registration to associate with the extraction map and click Add. Only a single capture registration can be selected.

The new capture registration now appears in the Current Capture Registrations list and the previous current capture registration is placed in the Available Capture Registrations list.

The following information is shown for the Available Capture Registrations:

The information is shown for the Current Capture Registration:

Item Description

Name Displays the name of the capture registrations that are available for selection.



Status Displays the status of the capture registration. This can have one of three valuesI - (Inactive) the registration is to be defined but is not to be used by the Collection agentA - (Active) the registration is to be defined and used by the Collection Agent

Item Description

Name Displays the name of the capture registration that has been selected.


Changing Extraction Maps 41

6. Click OK.


Adding or Removing Columns from Extraction MapsYou can add columns to and remove columns from an extraction map. While changes to the columns can still be captured, PowerExchange only extracts the columns contained in the extraction map. You do not need to change the capture registration or the source table when adding or removing columns from an extraction map.

To change an extraction map to remove a column:

1. In PowerExchange Navigator, click the extraction groups folder.

2. In the Resource Explorer, double-click to open the extraction group containing the extraction map to be modified.

3. In the Resource Explorer, double-click to open the extraction map.

4. On the Map tab of the Resources pane, select columns to add or clear columns to be removed from the extraction map.


Version Indicator in Extraction Definitions The Version indicator is used to identify the current version of the Extract Definition. The Extract Definition is represented as a Data Map and the extraction process uses this data map to read and format the data. The qualifier used in the SQL request used by the application to retrieve the changes specifies a data map name, such as Extraction Definition name.

The Version indicator is a mechanism included to handle situations where a registration changes, when a column is added or dropped. Changes to a registration will result in the layout of the extracted data changing. If the application retrieves the changes using the old data map, the returned data will not match with the new data map and the results will be unpredictable. When a column is added to a registration it might just work but this must not be relied upon.

Creating a new version of a registration is covered in “Changing Extraction Maps” on page 41. This section covers creating new versions of a registration and making those versions active. An application processing this data, be it PowerCenter or a third party product, cannot specify the version of the capture registration to use. Use of the required version is handled by making the required registration active and including the relevant columns in the Extract Definition.

What other changes are required if the version identifier changes? By making the changes in the extraction process only a part of the changes are completed. If the application is an ETL product, the source definition has to be adjusted so that the new columns are recognized or old ones are being dropped. The target definitions have to be made aware of the changes in the data layout.

It is important when changing a version that all the required data is captured and applied to the target before making a new version active.

When defining a new capture registration the PowerExchange Navigator application will automatically define a corresponding extraction definition with a version identifier of 1. Activating a new Version is a manual process from that point onwards to enable new definitions to begin capturing new formats in a controlled manner.


Status Displays the status of the capture registration.

Add Adds the selected capture registration to the Current Capture Registrations list.

Remove Removes the selected capture registration from the Current Capture Registrations list.

Item Description


Changing Properties of Capture RegistrationsIt may be necessary to change some of the registration properties of the Registration. The following properties can be amended:

♦ Status

♦ Comments

For example you may have created a registration with a status of Inactive and now wish to use it in your environment. You will need to change it to Active.

Amending a Capture RegistrationThe information available to change in the capture registration is limited, but to do so select the relevant Registration group and open the capture registration.

Changes cannot be made to the columns selected in this screen, although columns can be removed from the extraction definition. Changes can be made here to the Status of the registration.

Setting to ActiveWhen capture registrations are created they are usually created as Inactive. It is usual to create the registration in this way to prevent data being captured until everything is in place for the data to be useful to apply to the target. The registration will normally remain inactive until the target has been materialized, i.e. an initial data load has taken place to bring the target database to the required point in time. When ready, the registration will be set to Active.

Set to HistoryIf a registration is no longer required, the preferred method of disabling the registration is to set it to history. Open the Registration Group and then double-click or open the Registration definition. On the screen below, click on the field that says “Active” and select “History” from the drop-down selection list.

Changing Properties of Capture Registrations 43


C H A P T E R 7

Accessing CDC Data


♦ Accessing Captures Changes, 45

♦ Creating Restart Points with DTLUAPPL, 45

♦ Real-time Extraction Mode, 48

♦ SQL Server CDC Condense File Extraction, 49

Accessing Captures ChangesUse one of the following methods to access captured changes for Microsoft SQL Server:

♦ Real-time extraction mode

♦ Batch extraction mode from condense files

♦ Continuous extraction mode from condense files

Prior to extracting captured changes, establish an extraction start point.

Creating Restart Points with DTLUAPPLThe DTLUAPPL utility creates a baseline point to which data capture can be aligned. Each subsequent extraction registers new sequence and restart tokens. There are two ways to create a restart point for a capture extraction:

♦ Using the utility DTLUAPPL. For more information about DTLUAPPL, see the PowerExchange Utilities Guide.

♦ Running an extraction without specifying a restart point so the default restart point is used. For information about the default restart points, see PowerExchange Interfaces for PowerCenter.

If using PWXPC connections in PowerCenter, see the PowerExchange Interfaces for PowerCenter for more information about managing restart tokens.

If using ODBC, the restart tokens can be viewed and manipulated through the PowerExchange Navigator. If a data capture process needs to be run again, you can reset the restart tokens to an earlier point in time. When you do this, make sure all relevant captured changes are still available.

45

The DTLUAPPL utility is designed to set or reset start points for the extraction process and uses the following files:

♦ DTLTKNP.TXT which is supplied with example configurations

♦ A log of your chosen name to which DTLUAPPL will append information

DTLUAPPL can both register and modify applications, and then report on the status of those applications with its print utility.

DTLUAPPL - The PurposeThe purpose of this utility is to initially create a baseline point to which data capture can be aligned. Each data extract after that will register new sequence and restart tokens which can be viewed through the PowerExchange Navigator. If a single, or multiple, data capture process needs to be rerun the current restart tokens can then be reset to an earlier point in time. Care must be taken when doing this to ensure all the relevant captured changes are still available.

Running DTLUAPPLAmend DTLTKNP.TXT to create or modify the applications as required and run DTLUAPPL from a command prompt. DTLTKNP.TXT is supplied with sample statements commented out. The utility will take a few seconds to run. There are three ways of running DTLUAPPL:

1. Issue command DTLUAPPL on the command line. This will assume the tokens are in DTLTKNP.TXT and display the output in the command window.

2. Issue the following:

DTLUAPPL > logname.txt

DTLUAPPL will assume the tokens are in DTLTKNP.TXT but will write the output to logname.txt.

3. Issue the following:

DTLUAPPL myfile.txt > logname.txt

DTLUAPPL is pointed to tokens in myfile.txt and the output will be written to logname.txt.

Adding an ApplicationTo add a new application, add the following parameters in the DTLTKNP.TXT file:

UID user EPWD epwd CONN_OVR capi_connection ADD APPL appname instance RSTTKN GENERATE CAPTMETH=acc_meth CONDTYPE=P ADD RSTTKN regname END APPL appname

Where::‘


appname* Specifies the application name to be added within the Application Group.

regname* Specifies name of the capture registration defined to the PowerExchange Navigator.

instance* Specifies the generated instance name created when the Registration group was created. The instance can be seen when opening the registration group in PowerExchange Navigator.

CAPTMETH Specifies the capture access method:- CAPXRT for real-time or continuous extraction mode- CAPX for batch extraction mode

46 Chapter 7: Accessing CDC Data

The registration is made of two parts, a 20 byte sequence number and a 16 byte restart token. The restart tokens can be supplied at an application and/or registration level. If no token is specified at the registration level the one at application level will be propagated to that registration.

The restart tokens can be provided by the user or generated by use of the GENERATE keyword shown in the previous example.

If the PRINT statement is included, covered in “Printing an Application” on page 48, the tokens generated in this add application step will be seen.

Modifying an ApplicationThe syntax for modification is the same as for the ADD except the MOD keyword is used instead of ADD. If the application to be modified does not exist a warning will be issued and the application added.

The SEQUENCE and RESTART tokens can be set explicitly in a modification by using the following syntax:

UID user EPWD xxxxMOD APPL appname instanceMOD RSTTKN regnameSequence C4062A1643000000000000000000000000000000000500000000C4062A1643000000000000000000000000000000000500000000Restart C4062A16430000000000000000000000000000000005C4062A1643000000000000000000000000000000000500064F454D524550END APPL appname

Note: In DTLTKNP.TXT each of these tokens will fit in a single line.

Where:

♦ instance is the instance that is shown on the screen that is displayed in the topic “Viewing an Extraction Definition” on page 23.

♦ appname is the name of the application to be added within the Application Group.

♦ regname is the name of the capture registration defined to the PowerExchange Navigator.

At registration level there is an extra option, MODALLSAMETAG. If this option is not coded and DTLUAPPL is run to attempt to modify a registration for which other registrations are recorded (but are not included in the modify) then a warning will be issued and the modify will fail. If MODALLSAMETAG is included then all the registrations for that application will be updated with the same value. The format of this statement is shown below:

UID user EPWD xxxxMOD APPL appnameMOD RSTTKN regname MODALLSAMETAGEND APPL appname

CONDTYPE Specifies the Condense Type:- P for Partial condense - F for Full condense

CONN_OVR* Specifies the CAPI_CONNECTION name to use when running the utility. DTLUAPPL uses the default connection if this is not specified.

UID Specifies the SQL Server user ID.

EPWD Specifies encrypted password for the user ID generated using the PowerExchange Navigator.

Note: * Parameter values are case sensitive.


Creating Restart Points with DTLUAPPL 47

Printing an ApplicationThe DTLUAPPL utility can also print the applications, giving the status of existing sequence and restart information. When running on a Windows environment it is best to run this utility from a command line where the output will be displayed. The syntax of the print command is:

PRINT APPL appname/ALL

This will generate details for either the application or all applications as directed.

Real-time Extraction ModeThis manual has, so far, described how data is made available from the SQL Server replication databases by PowerExchange software. Captured data can be accessed through PowerExchange Real-time (CAPXRT). The syntax of the commands are documented completely in the PowerExchange Reference Manual, however, this manual gives an example that will help to describe the process.

PowerExchange Real-time or CAPXRT access is near real-time with access to the data written through the capture process to the PowerExchange Logger. When registering a table through PowerExchange Navigator for CAPXRT access the CONDENSE option should be set to NONE if access will be CAPXRT exclusive, otherwise select PART to allow CAPXRT and CAPX access.

At this point Microsoft replication should be configured, the tables registered for capture through PowerExchange Navigator and a restart token registered using the DTLUAPPL utility as described in “Creating Restart Points with DTLUAPPL” on page 81 (if DTLUAPPL is to be used to control the restart points). The data is now available for access from the PowerExchange Logger.

Testing with the NavigatorYou can use the PowerExchange Navigator, Row Test functionality to prove that the changes are being captured from your registered tables.

To perform a row test:

1. Select the extraction group that holds definition that you require to test.

2. Select the Extract Definition that you require to test.

3. Select the File, Database Row Test menu option or click on the Row Test button.

4. Ensure that the correct information is entered into the Row Test dialog box.

5. Click Go. The results of the data retrieval are displayed. These represent the changes that have occurred on the table that you registered for data capture.

Field Description

Location The location for the target node as set in the dbmover.cfg file.

UserID/Password Your UserID and Password used to access the data in SQL Server.

Application Name For a test this can be any character string. If extracting data in a production environment this would be the application name registered using DTLUAPPL.

SQL Statement The SQL statement is automatically generated. The table is identified in the format:D10<Target Subsystem id>.<CaptureRegistrationName>_<Tablename>where:D10 - always used for SQL Server.<Target Subsystem id> - From your capture registration.<CaptureRegistrationName> - From your capture registration.<Tablename> - From your capture registration.


6. If a significant amount of data is to be returned it may be possible the timeout parameter will need to be adjusted. You can do this by clicking on the Advanced button. The Advanced Parameters dialog box is displayed.

For more information about the Advanced Parameters dialog box, see the PowerExchange Navigator User Guide.

Using Real-time Extraction Mode in PowerCenterUsing PowerExchange Client for PowerCenter (PWXPC), a workflow can extract change data in real-time extraction mode. Use PWX MSSQL CDC Real Time application connections. For more information about PWXPC, see PowerExchange Interfaces for PowerCenter.

SQL Server CDC Condense File ExtractionPowerExchange SQL Server CDC can access changes from condense files created by PowerExchange Condense. PowerExchange Condense creates the condense files by reading the change data from SQL Server distribution database. Specify the frequency that the data is condensed using parameters in the PowerExchange Condense configuration file. For more information, see “Configuring PowerExchange Condense Parameters” on page 38.

To enable condense for capture registrations, set the condense option to Part. You can access change data from condense files using the following methods:

♦ Batch extraction mode using PowerCenter PWX CDC Change application connections or using the CAPX access method in PowerExchange Navigator.

♦ Continuous extraction mode using PowerCenter PWX CDC Real Time application connections or the CAPXRT access method in PowerExchange Navigator. When using PowerCenter to extract changes in continuous extraction mode, specify the name of the CAPX CAPI_CONNECTION in the CAPI Connection Name Override attribute.

For more information about PWXPC application connections, see PowerExchange Interfaces for PowerCenter.

SQL Server CDC Condense File Extraction 49


C H A P T E R 8

Stopping Change Capture

This chapter includes the following topic:

♦ Options, 51

OptionsCapture of SQL Server data is through the Microsoft Publication technology. PowerExchange software has the capability of then extracting that information and either applying it directly to a target database or making it available to partner software to manipulate and load. Stopping collection of data can be carried by either switching all capture registrations to history from active, or disabling replication at the SQL Server level.

Note: Great care should be taken when doing this because data changes will be missed while this change is in effect. If the SQL Server replication is to be reinstated, then rematerializing the target database may be the only sensible choice. If this is not possible, then this should be discussed with Informatica Global Customer Support before stopping Microsoft SQL Server Replication.

51

52 Chapter 8: Stopping Change Capture

I N D E X

AAccess Methods

CAPX 1CAPXRT 1

accessingMicrosoft SQL Server 5

Accessing Captures DataRow Test 48

Accessing SQL Server DataChecklist 5

addingextraction definitions to extraction groups 21extraction group 21

Adding a New ApplicationDTLUAPPL 46

Amending a Capture Registration 43application groups

overview 24Application Name

Row Test 48applications

resetting start points 25

BBatch mode

for condensing changes 30

CCAPI

CAPX parameters 14SQL parameters 12statements 12

CAPI_CONN_NAMECAPI statement 12

CAPI_CONNECTIONCAPI statement 12

capture registrationsadding to existing groups 19adding to extraction definitions 22repository file 19status 19types 18viewing 19, 20

capturing dataregistering 32

CAPXstarting 35

CAPX Access Method 1CAPX parameters

CAPI 14CAPXRT

PowerExchange Real-time 48CAPXRT Access Method 1CAPXRT Advanced Parameters 49CDCT file

condense output file 31change capture

components 3phases 2with PowerExchange Change and PowerExchange Real-time 1

Checkpoint Datasets 32Checkpoint files

condense output files 32record types 32

CHKPT_BASENAMEparameter 36

cold starteffect on Checkpoint files 32

Condensecapture registration type 18

condenseconfiguration file parameters 32output files 31shutting down 36

Condense filescondense output files 31

condense output filesCheckpoint files 32Condense files 31

condensing changesoperational modes 30

configuration filecondense parameters 32

Continuous modefor condensing changes 30

creatingregistration groups 17

Ddata capture

specifying tables 17data delivery

format 22

53

dbmover.cfg configuration fileviewing personal metadata 7

DCT recordsin Checkpoint files 32matching CDCT records 31

deletingextraction definitions 23extraction groups 23registration entries 19registration groups 19

DISTDB 12distribution databases

access requirement 9DISTSRV 12Dropping a Column 38DTL__BI

column 24DTL__CAPXACTION

column 24DTL__CAPXRESTART1

column 24DTL__CAPXRESTART2

column 24DTL__CAPXTIMESTAMP

column 24DTL__CAPXUOW

column 24DTL__CAPXUSER

column 24DTL__CI

column 24DTLAPP

extraction criteria parameter 23DTLDSN

extraction criteria parameter 23DTLTKNP.TXT

DTLUAPPL configuration file 46DTLUAPPL

Modifying an Application 47Printing an Application 48Running 46

DTLXTYPEextraction criteria parameter 23

DWFLAGS 13

EEOF 13ERT records

in Checkpoint files 32extraction criteria

defined with ODBC interface 23specifying 23

extraction definitionsadding capture registrations 22adding to extraction groups 21deleting 23viewing details 23

extraction groupsadding 21adding extraction definitions 21deleting 23

extraction processdefining 21overview 20

FFILE_SWITCH_CRIT

file switch tag 30FILE_SWITCH_VAL

file switch tag 30FILESWITCH 30filtering

tables 8formats

data delivery 22

HHistory 43

LLocation

Row Test 48Long Column Restriction 10

MMEMCACHE 13metadata profiles

creating 7Microsoft SQL Server

accessing 5publications 9requirement 3

Microsoft SQL Server distributionaccess requirement 9

modesoperational for condensing changes 30

Modifying an ApplicationDTLUAPPL 47

NNAME 13NO_DATA_WAIT 30

OODBC

configuring extraction criteria 23output files

CDCT file 31condense 31

Pparameters

condense configuration 32

54 Index

Personal Metadata 6personal metadata

adding 7creating metadata profiles 7filtering 8row testing 8viewing 7

POLWAIT 13PowerExchange Architecture 2PowerExchange Batch

overview 1PowerExchange Change

change capture 1PowerExchange Listener

testing remote 5PowerExchange Real-time

CAPXRT 48change capture 1

Primary Key Requirement 10Printing an Application

DTLUAPPL 48

Rrecord types

in Checkpoint files 32registering

captured data 32registration entries

deleting 19registration groups

creating 17deleting 19viewing 20

Registration Tokens 47registrations

active 19viewing 19

Removing a Column from a Capture Registration 38, 39, 40required authority

Microsoft SQL Server 9requirements

Microsoft SQL Server 3software 3

Restart TokenDTLUAPPL 47

RESTART_TOKENparameter 36

RestrictionsSQL Server restrictions 10

row testingpersonal metadata 8

RSextraction type 23

RSTRADV 13RSTTKN

DTLUAPPL 46RSTTKN GENERATE

DTLUAPPL 46

Running DTLUAPPL 46

Ssecurity

dbmover parameter 7Sequence Token

DTLUAPPL 47SEQUENCE_TOKEN

parameter 36shutting down

condense 36SL

extraction type 23Sourcing SQL Server Data 5sp_adddistpublisher

stored procedures 10sp_adddistributiondb

stored procedure 10sp_adddistributor

stored procedure 10specifying

extraction criteria 23SQL Server Replication Environment 6SQL Server restrictions 10SQL Statement

RowTest 48sql_variant

Unsupported data type 10SRT record

in Checkpoint files 32start point

resetting application 25starting

CAPX 35Status

capture registration type 18Stopping 51Stopping Change Capture 51stored procedures

sp_adddistpublisher 10sp_adddistributiondb 10sp_adddistributor 10

System Table Restriction 10

Ttables

specifying for data capture 17Testing with the Navigator 48Timestamp Restriction 10TYPE 13

UUnsupported data types

sql_variant 10User ID

restriction 10UserID/Password

RowTest 48

Index 55

Vviewing

capture registrations 20extraction definition details 23personal metadata 7registration groups 20

56 Index

NOTICES

This Informatica product (the “Software”) includes certain drivers (the “DataDirect Drivers”) from DataDirect Technologies, an operating company of Progress Software Corporation (“DataDirect”) which are subject to the following terms and conditions:

1. THE DATADIRECT DRIVERS ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.

2. IN NO EVENT WILL DATADIRECT OR ITS THIRD PARTY SUPPLIERS BE LIABLE TO THE END-USER CUSTOMER FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR OTHER DAMAGES ARISING OUT OF THE USE OF THE ODBC DRIVERS, WHETHER OR NOT INFORMED OF THE POSSIBILITIES OF DAMAGES IN ADVANCE. THESE LIMITATIONS APPLY TO ALL CAUSES OF ACTION, INCLUDING, WITHOUT LIMITATION, BREACH OF CONTRACT, BREACH OF WARRANTY, NEGLIGENCE, STRICT LIABILITY, MISREPRESENTATION AND OTHER TORTS.

Documents

PowerExchange SQL Server Adapter Guide...1 C HAPTER 1 Overview This chapter includes the following topics: ♦ PowerExchange Batch, 1 ♦ Change Capture with PowerExchange Change and