Embed Size (px)
Citation preview
Informatica PowerExchange for Netezza(Version 9.6.0)
User Guide for PowerCenter
Informatica PowerExchange for Netezza User Guide for PowerCenter
Version 9.6.0January 2014
Copyright (c) 2005-2014 Informatica Corporation. All rights reserved.
This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. This Software may be protected by U.S. and/or international Patents and other Patents Pending.
Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as provided in DFARS 227.7202-1(a) and 227.7702-3(a) (1995), DFARS 252.227-7013©(1)(ii) (OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR 52.227-14 (ALT III), as applicable.
The information in this product or documentation is subject to change without notice. If you find any problems in this product or documentation, please report them to us in writing.
Informatica, Informatica Platform, Informatica Data Services, PowerCenter, PowerCenterRT, PowerCenter Connect, PowerCenter Data Analyzer, PowerExchange, PowerMart, Metadata Manager, Informatica Data Quality, Informatica Data Explorer, Informatica B2B Data Transformation, Informatica B2B Data Exchange Informatica On Demand, Informatica Identity Resolution, Informatica Application Information Lifecycle Management, Informatica Complex Event Processing, Ultra Messaging and Informatica Master Data Management are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
Portions of this software and/or documentation are subject to copyright held by third parties, including without limitation: Copyright DataDirect Technologies. All rights reserved. Copyright © Sun Microsystems. All rights reserved. Copyright © RSA Security Inc. All Rights Reserved. Copyright © Ordinal Technology Corp. All rights reserved.Copyright © Aandacht c.v. All rights reserved. Copyright Genivia, Inc. All rights reserved. Copyright Isomorphic Software. All rights reserved. Copyright © Meta Integration Technology, Inc. All rights reserved. Copyright © Intalio. All rights reserved. Copyright © Oracle. All rights reserved. Copyright © Adobe Systems Incorporated. All rights reserved. Copyright © DataArt, Inc. All rights reserved. Copyright © ComponentSource. All rights reserved. Copyright © Microsoft Corporation. All rights reserved. Copyright © Rogue Wave Software, Inc. All rights reserved. Copyright © Teradata Corporation. All rights reserved. Copyright © Yahoo! Inc. All rights reserved. Copyright © Glyph & Cog, LLC. All rights reserved. Copyright © Thinkmap, Inc. All rights reserved. Copyright © Clearpace Software Limited. All rights reserved. Copyright © Information Builders, Inc. All rights reserved. Copyright © OSS Nokalva, Inc. All rights reserved. Copyright Edifecs, Inc. All rights reserved. Copyright Cleo Communications, Inc. All rights reserved. Copyright © International Organization for Standardization 1986. All rights reserved. Copyright © ej-technologies GmbH. All rights reserved. Copyright © Jaspersoft Corporation. All rights reserved. Copyright © is International Business Machines Corporation. All rights reserved. Copyright © yWorks GmbH. All rights reserved. Copyright © Lucent Technologies. All rights reserved. Copyright (c) University of Toronto. All rights reserved. Copyright © Daniel Veillard. All rights reserved. Copyright © Unicode, Inc. Copyright IBM Corp. All rights reserved. Copyright © MicroQuill Software Publishing, Inc. All rights reserved. Copyright © PassMark Software Pty Ltd. All rights reserved. Copyright © LogiXML, Inc. All rights reserved. Copyright © 2003-2010 Lorenzi Davide, All rights reserved. Copyright © Red Hat, Inc. All rights reserved. Copyright © The Board of Trustees of the Leland Stanford Junior University. All rights reserved. Copyright © EMC Corporation. All rights reserved. Copyright © Flexera Software. All rights reserved. Copyright © Jinfonet Software. All rights reserved. Copyright © Apple Inc. All rights reserved. Copyright © Telerik Inc. All rights reserved. Copyright © BEA Systems. All rights reserved. Copyright © PDFlib GmbH. All rights reserved. Copyright ©
Orientation in Objects GmbH. All rights reserved. Copyright © Tanuki Software, Ltd. All rights reserved. Copyright © Ricebridge. All rights reserved. Copyright © Sencha, Inc. All rights reserved.
This product includes software developed by the Apache Software Foundation (http://www.apache.org/), and/or other software which is licensed under various versions of the Apache License (the "License"). You may obtain a copy of these Licenses at http://www.apache.org/licenses/. Unless required by applicable law or agreed to in writing, software distributed under these Licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the Licenses for the specific language governing permissions and limitations under the Licenses.
This product includes software which was developed by Mozilla (http://www.mozilla.org/), software copyright The JBoss Group, LLC, all rights reserved; software copyright © 1999-2006 by Bruno Lowagie and Paulo Soares and other software which is licensed under various versions of the GNU Lesser General Public License Agreement, which may be found at http:// www.gnu.org/licenses/lgpl.html. The materials are provided free of charge by Informatica, "as-is", without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose.
The product includes ACE(TM) and TAO(TM) software copyrighted by Douglas C. Schmidt and his research group at Washington University, University of California, Irvine, and Vanderbilt University, Copyright (©) 1993-2006, all rights reserved.
This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (copyright The OpenSSL Project. All Rights Reserved) and redistribution of this software is subject to terms available at http://www.openssl.org and http://www.openssl.org/source/license.html.
This product includes Curl software which is Copyright 1996-2013, Daniel Stenberg, <[email protected]>. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://curl.haxx.se/docs/copyright.html. Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
The product includes software copyright 2001-2005 (©) MetaStuff, Ltd. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://www.dom4j.org/ license.html.
The product includes software copyright © 2004-2007, The Dojo Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://dojotoolkit.org/license.
This product includes ICU software which is copyright International Business Machines Corporation and others. All rights reserved. Permissions and limitations regarding this software are subject to terms available at http://source.icu-project.org/repos/icu/icu/trunk/license.html.
This product includes software copyright © 1996-2006 Per Bothner. All rights reserved. Your right to use such materials is set forth in the license which may be found at http:// www.gnu.org/software/ kawa/Software-License.html.
This product includes OSSP UUID software which is Copyright © 2002 Ralf S. Engelschall, Copyright © 2002 The OSSP Project Copyright © 2002 Cable & Wireless Deutschland. Permissions and limitations regarding this software are subject to terms available at http://www.opensource.org/licenses/mit-license.php.
This product includes software developed by Boost (http://www.boost.org/) or under the Boost software license. Permissions and limitations regarding this software are subject to terms available at http:/ /www.boost.org/LICENSE_1_0.txt.
This product includes software copyright © 1997-2007 University of Cambridge. Permissions and limitations regarding this software are subject to terms available at http:// www.pcre.org/license.txt.
This product includes software copyright © 2007 The Eclipse Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http:// www.eclipse.org/org/documents/epl-v10.php and at http://www.eclipse.org/org/documents/edl-v10.php.
This product includes software licensed under the terms at http://www.tcl.tk/software/tcltk/license.html, http://www.bosrup.com/web/overlib/?License, http://www.stlport.org/doc/ license.html, http:// asm.ow2.org/license.html, http://www.cryptix.org/LICENSE.TXT, http://hsqldb.org/web/hsqlLicense.html, http://httpunit.sourceforge.net/doc/ license.html, http://jung.sourceforge.net/license.txt , http://www.gzip.org/zlib/zlib_license.html, http://www.openldap.org/software/release/
license.html, http://www.libssh2.org, http://slf4j.org/license.html, http://www.sente.ch/software/OpenSourceLicense.html, http://fusesource.com/downloads/license-agreements/fuse-message-broker-v-5-3- license-agreement; http://antlr.org/license.html; http://aopalliance.sourceforge.net/; http://www.bouncycastle.org/licence.html; http://www.jgraph.com/jgraphdownload.html; http://www.jcraft.com/jsch/LICENSE.txt; http://jotm.objectweb.org/bsd_license.html; . http://www.w3.org/Consortium/Legal/2002/copyright-software-20021231; http://www.slf4j.org/license.html; http://nanoxml.sourceforge.net/orig/copyright.html; http://www.json.org/license.html; http://forge.ow2.org/projects/javaservice/, http://www.postgresql.org/about/licence.html, http://www.sqlite.org/copyright.html, http://www.tcl.tk/software/tcltk/license.html, http://www.jaxen.org/faq.html, http://www.jdom.org/docs/faq.html, http://www.slf4j.org/license.html; http://www.iodbc.org/dataspace/iodbc/wiki/iODBC/License; http://www.keplerproject.org/md5/license.html; http://www.toedter.com/en/jcalendar/license.html; http://www.edankert.com/bounce/index.html; http://www.net-snmp.org/about/license.html; http://www.openmdx.org/#FAQ; http://www.php.net/license/3_01.txt; http://srp.stanford.edu/license.txt; http://www.schneier.com/blowfish.html; http://www.jmock.org/license.html; http://xsom.java.net; http://benalman.com/about/license/; https://github.com/CreateJS/EaselJS/blob/master/src/easeljs/display/Bitmap.js; http://www.h2database.com/html/license.html#summary; http://jsoncpp.sourceforge.net/LICENSE; http://jdbc.postgresql.org/license.html; http://protobuf.googlecode.com/svn/trunk/src/google/protobuf/descriptor.proto; https://github.com/rantav/hector/blob/master/LICENSE; http://web.mit.edu/Kerberos/krb5-current/doc/mitK5license.html. and http://jibx.sourceforge.net/jibx-license.html.
This product includes software licensed under the Academic Free License (http://www.opensource.org/licenses/afl-3.0.php), the Common Development and Distribution License (http://www.opensource.org/licenses/cddl1.php) the Common Public License (http://www.opensource.org/licenses/cpl1.0.php), the Sun Binary Code License Agreement Supplemental License Terms, the BSD License (http:// www.opensource.org/licenses/bsd-license.php), the new BSD License (http://opensource.org/licenses/BSD-3-Clause), the MIT License (http://www.opensource.org/licenses/mit-license.php), the Artistic License (http://www.opensource.org/licenses/artistic-license-1.0) and the Initial Developer’s Public License Version 1.0 (http://www.firebirdsql.org/en/initial-developer-s-public-license-version-1-0/).
This product includes software copyright © 2003-2006 Joe WaInes, 2006-2007 XStream Committers. All rights reserved. Permissions and limitations regarding this software are subject to terms available at http://xstream.codehaus.org/license.html. This product includes software developed by the Indiana University Extreme! Lab. For further information please visit http://www.extreme.indiana.edu/.
This product includes software Copyright (c) 2013 Frank Balluffi and Markus Moeller. All rights reserved. Permissions and limitations regarding this software are subject to terms of the MIT license.
This Software is protected by U.S. Patent Numbers 5,794,246; 6,014,670; 6,016,501; 6,029,178; 6,032,158; 6,035,307; 6,044,374; 6,092,086; 6,208,990; 6,339,775; 6,640,226; 6,789,096; 6,823,373; 6,850,947; 6,895,471; 7,117,215; 7,162,643; 7,243,110; 7,254,590; 7,281,001; 7,421,458; 7,496,588; 7,523,121; 7,584,422; 7,676,516; 7,720,842; 7,721,270; 7,774,791; 8,065,266; 8,150,803; 8,166,048; 8,166,071; 8,200,622; 8,224,873; 8,271,477; 8,327,419; 8,386,435; 8,392,460; 8,453,159; 8,458,230; and RE44,478, International Patents and other Patents Pending.
DISCLAIMER: Informatica Corporation provides this documentation "as is" without warranty of any kind, either express or implied, including, but not limited to, the implied warranties of noninfringement, merchantability, or use for a particular purpose. Informatica Corporation does not warrant that this software or documentation is error free. The information provided in this software or documentation may include technical inaccuracies or typographical errors. The information in this software and documentation is subject to change at any time without notice.
NOTICES
This Informatica product (the "Software") includes certain drivers (the "DataDirect Drivers") from DataDirect Technologies, an operating company of Progress Software Corporation ("DataDirect") which are subject to the following terms and conditions:
1.THE DATADIRECT DRIVERS ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.
2. IN NO EVENT WILL DATADIRECT OR ITS THIRD PARTY SUPPLIERS BE LIABLE TO THE END-USER CUSTOMER FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR OTHER DAMAGES ARISING OUT OF THE USE OF THE ODBC DRIVERS, WHETHER OR NOT INFORMED OF THE POSSIBILITIES OF DAMAGES IN ADVANCE. THESE LIMITATIONS APPLY TO ALL CAUSES OF ACTION, INCLUDING, WITHOUT LIMITATION, BREACH OF CONTRACT, BREACH OF WARRANTY, NEGLIGENCE, STRICT LIABILITY, MISREPRESENTATION AND OTHER TORTS.
Part Number: PWX-NZU-96000-0001
Table of Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiInformatica Resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Informatica My Support Portal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Informatica Documentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Informatica Web Site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Informatica How-To Library. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Informatica Knowledge Base. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Informatica Support YouTube Channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Informatica Marketplace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Informatica Velocity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Informatica Global Customer Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Chapter 1: Introduction to PowerExchange for Netezza. . . . . . . . . . . . . . . . . . . . . . . . . 1PowerExchange for Netezza Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Code Pages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Chapter 2: PowerExchange for Netezza Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . 2PowerExchange for Netezza Configuration Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Prerequisites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Registering the Plug-in. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Connecting to a Netezza Database from Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Configuring ODBC Connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Connecting to a Netezza Database from UNIX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Configuring ODBC Connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
PowerExchange for Netezza Upgrade. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Upgrading PowerExchange for Netezza from Versions Earlier than 9.0. . . . . . . . . . . . . . . . . 6
Upgrading PowerExchange for Netezza from Version 9.0 or Later. . . . . . . . . . . . . . . . . . . . 6
Chapter 3: Netezza Sources and Targets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Netezza Sources and Targets Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Source Qualifier Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Importing Netezza Source Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Importing Netezza Target Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Chapter 4: Netezza Sessions and Workflows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Data Transfer Modes in Netezza. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Normal Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Bulk Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Normal and Bulk Mode Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
PowerExchange for Netezza Connections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Table of Contents i
Netezza Connection Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Creating a Netezza Connection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Session Configuration with a Netezza Source. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Session Configuration with a Netezza Target. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Target Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Null Values and Empty Strings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Unprojected Columns. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Pipeline Partitioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Target Connection Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Multiple Targets Configuration for the Same Target Table. . . . . . . . . . . . . . . . . . . . . . . . . 17
Netezza Target Data Update. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Update As Insert. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Update Else Insert. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Netezza Session Configuration for Optimal Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Netezza Distribution Key. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Troubleshooting Netezza Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Appendix A: Datatype Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Netezza and Transformation Datatypes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
ii Table of Contents
PrefaceThe Informatica PowerExchange for Netezza User Guide for PowerCenter provides information about extracting data from a Netezza source and loading data into a Netezza target. It is written for database administrators and developers who are responsible for extracting data from Netezza and loading data to Netezza. This book assumes you have knowledge of Netezza and PowerCenter.
Informatica Resources
Informatica My Support PortalAs an Informatica customer, you can access the Informatica My Support Portal at http://mysupport.informatica.com.
The site contains product information, user group information, newsletters, access to the Informatica customer support case management system (ATLAS), the Informatica How-To Library, the Informatica Knowledge Base, Informatica Product Documentation, and access to the Informatica user community.
Informatica DocumentationThe Informatica Documentation team takes every effort to create accurate, usable documentation. If you have questions, comments, or ideas about this documentation, contact the Informatica Documentation team through email at [email protected]. We will use your feedback to improve our documentation. Let us know if we can contact you regarding your comments.
The Documentation team updates documentation as needed. To get the latest documentation for your product, navigate to Product Documentation from http://mysupport.informatica.com.
Informatica Web SiteYou can access the Informatica corporate web site at http://www.informatica.com. The site contains information about Informatica, its background, upcoming events, and sales offices. You will also find product and partner information. The services area of the site includes important information about technical support, training and education, and implementation services.
Informatica How-To LibraryAs an Informatica customer, you can access the Informatica How-To Library at http://mysupport.informatica.com. The How-To Library is a collection of resources to help you learn more about Informatica products and features. It includes articles and interactive demonstrations that provide
iii
solutions to common problems, compare features and behaviors, and guide you through performing specific real-world tasks.
Informatica Knowledge BaseAs an Informatica customer, you can access the Informatica Knowledge Base at http://mysupport.informatica.com. Use the Knowledge Base to search for documented solutions to known technical issues about Informatica products. You can also find answers to frequently asked questions, technical white papers, and technical tips. If you have questions, comments, or ideas about the Knowledge Base, contact the Informatica Knowledge Base team through email at [email protected].
Informatica Support YouTube ChannelYou can access the Informatica Support YouTube channel at http://www.youtube.com/user/INFASupport. The Informatica Support YouTube channel includes videos about solutions that guide you through performing specific tasks. If you have questions, comments, or ideas about the Informatica Support YouTube channel, contact the Support YouTube team through email at [email protected] or send a tweet to @INFASupport.
Informatica MarketplaceThe Informatica Marketplace is a forum where developers and partners can share solutions that augment, extend, or enhance data integration implementations. By leveraging any of the hundreds of solutions available on the Marketplace, you can improve your productivity and speed up time to implementation on your projects. You can access Informatica Marketplace at http://www.informaticamarketplace.com.
Informatica VelocityYou can access Informatica Velocity at http://mysupport.informatica.com. Developed from the real-world experience of hundreds of data management projects, Informatica Velocity represents the collective knowledge of our consultants who have worked with organizations from around the world to plan, develop, deploy, and maintain successful data management solutions. If you have questions, comments, or ideas about Informatica Velocity, contact Informatica Professional Services at [email protected].
Informatica Global Customer SupportYou can contact a Customer Support Center by telephone or through the Online Support.
Online Support requires a user name and password. You can request a user name and password at http://mysupport.informatica.com.
The telephone numbers for Informatica Global Customer Support are available from the Informatica web site at http://www.informatica.com/us/services-and-training/support-services/global-support-centers/.
iv Preface
C H A P T E R 1
Introduction to PowerExchange for Netezza
This chapter includes the following topics:
• PowerExchange for Netezza Overview, 1
• Code Pages, 1
PowerExchange for Netezza OverviewPowerExchange for Netezza provides bidirectional connectivity between PowerCenter and IBM Netezza Platform Software to read and write data.
The Designer uses a relational connector to connect to the Netezza database. You can import Netezza tables as sources and target definitions. You can connect to the Netezza Performance Server to read data from Netezza tables and load data to Netezza tables. The Netezza Performance Server integrates database, server, and storage in a single system.
Configure a Netezza database connection to read data from and write to Netezza.
Code PagesWhen the PowerCenter Integration Service runs in Unicode mode, it encodes Netezza data of the Nchar(m) and NVarchar(m) datatypes in UTF-8. It encodes Netezza data of the Varchar and Char datatypes in Latin-9.
If the data contains extended ASCII characters or UTF-8 characters, run the PowerCenter Integration Service in Unicode mode.
1
C H A P T E R 2
PowerExchange for Netezza Configuration
This chapter includes the following topics:
• PowerExchange for Netezza Configuration Overview, 2
• Prerequisites, 2
• Registering the Plug-in, 3
• Connecting to a Netezza Database from Windows, 3
• Connecting to a Netezza Database from UNIX, 4
• PowerExchange for Netezza Upgrade, 6
PowerExchange for Netezza Configuration OverviewPowerExchange for Netezza installs with PowerCenter.
To configure PowerExchange for Netezza, complete the following steps:
1. Complete the prerequisites.
2. Register the PowerExchange for Netezza plug-in if you want to read or write data in bulk mode. To read or write data in normal mode, you do not need to perform configuration steps.
PrerequisitesBefore you configure PowerExchange for Netezza, complete the following tasks:
• Install or upgrade PowerCenter.
• Install the client and server components of the Netezza Performance Server.
• Verify that the Netezza database user has the following privileges on the database:
- CREATE TABLE
- CREATE EXTERNAL TABLE
- DELETE
2
- DROP
- INSERT
- LIST
- SELECT
- TRUNCATE
- UPDATE
Registering the Plug-inTo read or write Netezza data in bulk mode, you need to register the plug-in with the repository. You do not need to register the plug-in with the repository to read or write Netezza data in normal mode.
To register the plug-in, the repository must be running in exclusive mode. Use the Informatica Administrator or the pmrep RegisterPlugin command to register the plug-in.
The plug-in file for PowerExchange for Netezza is pmnetezza.xml. When you install the Service component, the installer copies pmnetezza.xml to the following directory:
<PowerCenter Installation Directory>\server\bin\Plugin
Note: If you do not have the correct privileges to register the plug-in, contact the user who manages the PowerCenter Repository Service.
Connecting to a Netezza Database from WindowsInstall and configure ODBC on the machines where the PowerCenter Integration Service process runs and where you install PowerCenter Client. You must configure connectivity to the following Informatica components on Windows:
• PowerCenter Integration Service. Install the Netezza ODBC driver on the machine where the PowerCenter Integration Service process runs. Use the Microsoft ODBC Data Source Administrator to configure ODBC connectivity.
• PowerCenter Client. Install the Netezza ODBC driver on each PowerCenter Client machine that accesses the Netezza database. Use the Microsoft ODBC Data Source Administrator to configure ODBC connectivity. Use the Workflow Manager to create a database connection object for the Netezza database.
Registering the Plug-in 3
Configuring ODBC ConnectivityYou can configure ODBC connectivity to a Netezza database.
The following steps provide a guideline for configuring ODBC connectivity. For specific instructions, see the database documentation.
1. Create an ODBC data source for each Netezza database that you want to access.
To create the ODBC data source, use the driver provided by Netezza.
Create a System DSN if you start the Informatica service with a Local System account logon. Create a User DSN if you select the This account log in option to start the Informatica service.
After you create the data source, configure the properties of the data source.
2. Enter a name for the new ODBC data source.
3. Enter the IP address/host name and port number for the Netezza server.
4. Enter the name of the Netezza schema where you plan to create database objects.
5. Configure the path and file name for the ODBC log file.
6. Verify that you can connect to the Netezza database.
You can use the Microsoft ODBC Data Source Administrator to test the connection to the database. To test the connection, select the Netezza data source and click Configure. On the Testing tab, click Test Connection and enter the connection information for the Netezza schema.
Connecting to a Netezza Database from UNIXInstall and configure Netezza ODBC driver on the machine where the PowerCenter Integration Service process runs. Use the DataDirect Driver Manager in the DataDirect driver package shipped with the Informatica product to configure the Netezza data source details in the odbc.ini file.
Configuring ODBC ConnectivityYou can configure ODBC connectivity to a Netezza database.
The following steps provide a guideline for configuring ODBC connectivity. For specific instructions, see the database documentation.
1. To configure connectivity for the integration service process, log in to the machine as a user who can start a service process.
2. Set the ODBCHOME, NZ_ODBC_INI_PATH, and PATH environment variables.
ODBCHOME. Set the variable to the ODBC installation directory. For example:
Using a Bourne shell:$ ODBCHOME=<Informatica server home>/ODBC7.1; export ODBCHOME
Using a C shell:$ setenv ODBCHOME =<Informatica server home>/ODBC7.1
PATH. Set the variable to the ODBCHOME/bin directory. For example:
Using a Bourne shell:PATH="${PATH}:$ODBCHOME/bin"
4 Chapter 2: PowerExchange for Netezza Configuration
Using a C shell:$ setenv PATH ${PATH}:$ODBCHOME/bin
NZ_ODBC_INI_PATH. Set the variable to point to the directory that contains the odbc.ini file. For example, if the odbc.ini file is in the $ODBCHOME directory:
Using a Bourne shell:NZ_ODBC_INI_PATH=$ODBCHOME; export NZ_ODBC_INI_PATH
Using a C shell:$ setenv NZ_ODBC_INI_PATH $ODBCHOME
3. Set the shared library environment variable.
The shared library path must contain the ODBC libraries. It must also include the Informatica services installation directory (server_dir).
Set the shared library environment variable based on the operating system. Set the Netezza library folder to <NetezzaInstallationDir>/lib64.
The following table describes the shared library variables for each operating system:
Operating System Variable
Solaris LD_LIBRARY_PATH
Linux LD_LIBRARY_PATH
AIX LIBPATH
For example, use the following syntax for Solaris:
• Using a Bourne shell:$ LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:$HOME/server_dir:$ODBCHOME/lib:<NetezzaInstallationDir>/lib64”export LD_LIBRARY_PATH
• Using a C shell:$ setenv LD_LIBRARY_PATH "${LD_LIBRARY_PATH}:$HOME/server_dir:$ODBCHOME/lib:<NetezzaInstallationDir>/lib64"
For AIX
• Using a Bourne shell:$ LIBPATH=${LIBPATH}:$HOME/server_dir:$ODBCHOME/lib:<NetezzaInstallationDir>/lib64; export LIBPATH
• Using a C shell:$ setenv LIBPATH ${LIBPATH}:$HOME/server_dir:$ODBCHOME/lib:<NetezzaInstallationDir>/lib64
4. Edit the existing odbc.ini file or copy the odbc.ini file to the home directory and edit it.
This file exists in $ODBCHOME directory.$ cp $ODBCHOME/odbc.ini $HOME/.odbc.ini
Add an entry for the Netezza data source under the section [ODBC Data Sources] and configure the data source.
For example:[NZSQL]Driver = /export/home/appsqa/thirdparty/netezza/lib64/libnzodbc.soDescription = NetezzaSQL ODBC
Connecting to a Netezza Database from UNIX 5
Servername = netezza1.informatica.comPort = 5480Database = infaUsername = adminPassword = passwordDebuglogging = trueStripCRLF = falsePreFetch = 256Protocol = 7.0ReadOnly = falseShowSystemTables = falseSocket = 16384DateFormat = 1TranslationDLL =TranslationName =TranslationOption =NumericAsChar = false
For more information about Netezza connectivity, see the Netezza ODBC driver documentation.
5. Verify that the last entry in the odbc.ini file is InstallDir and set it to the ODBC installation directory.
For example:InstallDir=/usr/odbc
6. Edit the .cshrc or .profile file to include the complete set of shell commands.
7. Restart the Informatica services.
PowerExchange for Netezza UpgradeYou can upgrade from PowerExchange for Netezza versions 8.1.1.0.3 and later. The upgrade steps depend on the version that you upgrade from.
Upgrading PowerExchange for Netezza from Versions Earlier than 9.0
1. Upgrade PowerCenter.
2. Configure the PowerCenter Repository Service to run in exclusive mode.
To change the PowerCenter Repository Service operating mode, you can use the Administrator tool or the infacmd UpdateRepositoryService command.
3. Use the pmrep UpgradeNetezzaToRelational command to upgrade PowerExchange for Netezza.
Enter the following command: pmrep upgradeNetezzaToRelational4. Configure the PowerCenter Repository Service to run in normal mode.
Upgrading PowerExchange for Netezza from Version 9.0 or LaterTo upgrade PowerExchange for Netezza from version 9.0 or later, complete the prerequisite tasks. Update the PowerExchange for Netezza plug-in registration if you want to read or write data in bulk mode.
6 Chapter 2: PowerExchange for Netezza Configuration
C H A P T E R 3
Netezza Sources and TargetsThis chapter includes the following topics:
• Netezza Sources and Targets Overview, 7
• Source Qualifier Properties, 7
• Importing Netezza Source Definitions, 8
• Importing Netezza Target Definitions, 8
Netezza Sources and Targets OverviewNetezza source and target definitions represent metadata for Netezza tables. When you import Netezza definitions, you can choose to preview data in the tables.
You can edit definitions to configure the properties that you did not import from Netezza. If you want to enforce key constraints, define them in the Designer. When you run a session, the PowerCenter Integration Service establishes relationships within the pipeline based on source and target definitions. Netezza does not enforce key constraints.
Source Qualifier PropertiesYou can configure source qualifier properties to sort the number of input ports and to retrieve distinct data from a Netezza source. You can override the values in the session properties.
The following table describes the source qualifier properties:
Source Options Description
Select Distinct Selects unique values. Netezza ignores trailing spaces. Therefore, the PowerCenter Integration Service might extract fewer rows than expected.
Source Filter Reduces the number of rows the PowerCenter Integration Service queries.Use the following syntax:
<table name>.”<field name>” <operator> <value>The filter condition is case sensitive.
7
Source Options Description
Number of Sorted Ports
Number of columns used when sorting rows queried from the source. The PowerCenter Integration Service adds an ORDER BY clause to the default query when it reads source rows. The ORDER BY clause includes the number of ports specified, starting from the top of the transformation. When you specify the number of sorted ports, the database sort order must match the session sort order.Default is 0.
SQL Query Overrides the default query. Enclose column names in double quotes. The SQL query is case sensitive.
Importing Netezza Source DefinitionsTo create a Netezza source definition, use the Source Analyzer to import source metadata with the Netezza relational data source.
1. In the Source Analyzer, click Sources > Import from Database.
2. Select the Netezza data source used to connect to the source database.
If you need to create or modify a Netezza data source, click the Browse button to open the ODBC Administrator. Create the Netezza data source and click OK. Select the new Netezza data source.
3. Enter a database user name and password to connect to the database.
Note: The user must have the appropriate database permissions to view the object.
You may need to specify the owner name for database objects you want to use as sources.
4. Optionally, use the search field to limit the number of tables that appear.
5. Click Connect.
If no table names appear, or if the table you want to import does not appear, click All.
6. Scroll down through the list of sources to find the source you want to import. Select the relational object or objects you want to import.
You can hold down the Shift key to select a block of sources within one folder or hold down the Ctrl key to make non-consecutive selections within a folder. You can also select all tables within a folder by selecting the folder and clicking Select All. Use the Select None button to clear all highlighted selections.
7. Click OK.
The source definition appears in the Source Analyzer. In the Navigator, the source definition appears in the Sources node of the active repository folder under the source database name.
Importing Netezza Target DefinitionsTo create a Netezza target definition, use the Target Designer to import source metadata with the Netezza relational data source.
1. In the Target Designer, click Targets > Import from Database.
8 Chapter 3: Netezza Sources and Targets
2. Select the Netezza data source used to connect to the target database.
If you need to create or modify a Netezza data source, click the Browse button to open the ODBC Administrator. Create the Netezza data source and click OK. Select the new Netezza data source.
3. Enter the user name and password to connect to the database, and click Connect.
If you are not the owner of the table you want to use as a target, specify the owner name.
4. Drill down through the list of database objects to view the available tables as targets.
5. Select the relational table or tables to import the definitions into the repository.
You can hold down the Shift key to select a block of tables, or hold down the Ctrl key to make non-consecutive selections. You can also use the Select All and Select None buttons to select or clear all available targets.
6. Click OK.
The selected target definitions appear in the Navigator under the Targets node.
Importing Netezza Target Definitions 9
C H A P T E R 4
Netezza Sessions and WorkflowsThis chapter includes the following topics:
• Data Transfer Modes in Netezza, 10
• PowerExchange for Netezza Connections, 12
• Session Configuration with a Netezza Source, 13
• Session Configuration with a Netezza Target, 14
• Netezza Target Data Update, 17
• Netezza Session Configuration for Optimal Performance, 19
• Troubleshooting Netezza Sessions, 19
Data Transfer Modes in NetezzaYou can transfer data in Netezza by using normal and bulk mode.
Normal ModeIn normal mode, the PowerCenter Integration Service extracts and loads data row by row.
Bulk ModeYou can transfer data in Netezza by using bulk mode. Use bulk mode to increase session performance.
In bulk mode, the PowerCenter Integration Service reads and writes Netezza data through an external table. An external table definition is stored within the Netezza database but the data is saved externally in a location that is accessible to the Netezza host or the client system. Create external tables to structure your loading operation and manipulate data by using Netezza SQL.
When the PowerCenter Integration Service extracts from Netezza, it creates an external table in the pipe directory path specified for extraction. When the PowerCenter Integration Service loads to Netezza, it creates an external table in the pipe directory path specified for loading. You can load data from the external table to the target. If duplicate row handling is configured, data is loaded from the external table to a temporary table and then finally to the target.
To transfer data in Netezza in bulk mode, complete the following steps:
1. Verify that the Netezza database user has the LIST and CREATE EXTERNAL TABLE privileges on the database.
10
2. Register the PowerExchange for Netezza plug-in with the repository.
3. Configure the session to use a Netezza bulk reader and Netezza bulk writer.
4. Configure the session properties as described in the following sections.
Normal and Bulk Mode FeaturesThe following table lists the features supported in normal and bulk modes:
Feature Normal Mode Bulk Mode
Source and target table name prefix
Supported Unsupported
Recovery Supported Unsupported
Real-time sessions Supported Unsupported
Session on grid Supported Limited support. For more information about running sessions on a grid in bulk mode, see Informatica Knowledge Base article 136654.
Commit interval Supported Unsupported
Implicit join based on primary key and foreign key
Supported Unsupported
Pushdown optimization Supported Unsupported
Duplicate row handling Unsupported Supported
Pre-SQL Supported Supported
Post-SQL Supported Supported
Target table name override Supported Supported
Truncate target table Supported. Run the delete command. If you specify an SQL statement in the Pre-SQL property, the PowerCenter Integration Service runs the SQL statement after the data in the table is deleted.
Supported. Run the truncate table command. If you specify an SQL statement in the Pre-SQL property, the PowerCenter Integration Service runs the SQL statement before the table is truncated.
Target connection group Supported Limited support. Follow the criteria, and rules and guidelines applicable for using target connection groups in bulk mode.
Partitioning Database, hash, key range, pass-through, and round-robin partitioning are supported.
Pass-through partitioning is supported.
Data Transfer Modes in Netezza 11
PowerExchange for Netezza ConnectionsUse a relational connection object for each Netezza source or target that you want to access.
The relational database connection defines how the PowerCenter Integration Service accesses the underlying database for Netezza Performance Server. When you configure a Netezza connection, you specify the connection attributes that the PowerCenter Integration Service uses to connect to Netezza.
Netezza Connection PropertiesThe following table describes the Netezza connection properties that you must configure:
Property Description
User Name Database user name with the appropriate read and write database permissions to access Netezza Performance Server.
Use Parameter in Password
Indicates the password for the database user name is a session parameter, $ParamName. Define the password in the workflow or session parameter file, and encrypt it by using the pmpasswd CRYPT_DATA option. Default is disabled.
Password Password for the database user name.
Connect String ODBC data source to connect to Netezza Performance Server.
Code Page Code page associated with Netezza Performance Server.
Connection Environment SQL
Runs an SQL command with each database connection. Default is disabled.
Transaction Environment SQL
Runs an SQL command before the initiation of each transaction. Default is disabled.
Connection Retry Period
Number of seconds the PowerCenter Integration Service attempts to reconnect to the database if the connection fails. If the PowerCenter Integration Service cannot connect to the database in the retry period, the session fails. Default value is 0.
Creating a Netezza ConnectionCreate a Netezza connection before you run a Netezza session.
1. In the Workflow Manager, connect to a repository.
2. Click Connections > Relational.
The Relational Connection Browser dialog box appears.
3. Click New.
The Select Subtype dialog box appears.
4. Select Netezza from the Select Subtype list.
5. Click OK.
The Connection Object Definition dialog box appears.
6. Enter the connection properties.
12 Chapter 4: Netezza Sessions and Workflows
7. Click OK.
The Netezza connection appears in the Connection Browser list.
Session Configuration with a Netezza SourceYou can configure the session properties for a Netezza source on the Mapping tab. Define the properties for each source instance in the session.
To extract data in normal mode, configure the session to use a relational reader. To extract data in bulk mode, configure the session to use a Netezza bulk reader. The session properties for normal mode are the same as that of any other relational source.
The following table describes the session properties that you must configure to extract Netezza source data in bulk mode:
Attribute Name Description
Socket Buffer Size Set the socket buffer size to 25 to 50 % of the DTM buffer size to increase session performance. You might need to test different settings for optimal performance. Enter a value between 4096 and 2147483648 bytes.Default is 8388608 bytes.
Pipe Directory Path
Path for the PowerCenter Integration Service to create the pipe for the external table. If you do not specify the path, the PowerCenter Integration Service uses the following directory to create the pipe for the external table:<PowerCenter Installation Directory>/server/binRequired if the machine hosting the PowerCenter Integration Service is on HP-UX and the following directory is on an NFS-mounted directory:<PowerCenter Installation Directory>/server/bin Enter a path that does not use an NFS mount.
Delimiter Delimiter separates successive input fields. You can enter any value supported by the Netezza Performance Server. The value can be a part of the data for the Netezza source. Default is |.
NullValue NullValue parameter of an external table. The PowerCenter Integration Service uses the NullValue internally. Maximum value is one character. Default is blank.
EscapeCharacter Escape character of an external table. If the data contains NULL, CR, and LF characters in the Char or Varchar field, you need to add an escape character in the source data before extracting. Enter an escape character before the data. The supported escape character is backslash (\).
Note: You can view load statistics in the session log. The load summary in the Workflow Monitor does not display load statistics.
Session Configuration with a Netezza Source 13
Session Configuration with a Netezza TargetYou can configure target properties for a session that writes data to Netezza targets:
• Target database connection
• Target properties
• Update strategy
• Multiple targets referring to the same table
• Pipeline partitioning
Target PropertiesYou can configure the session properties for Netezza targets in the Transformations view on the Mapping tab. Define the properties for each target instance in the session.
To load data in normal mode, configure the session to use a relational writer. To load data in bulk mode, configure the session to use a Netezza bulk writer. The session properties for normal mode are the same as that of any other relational target.
The following table describes the session properties that you must configure to load Netezza target data in bulk mode:
Target Property Description
Socket Buffer Size Set the socket buffer size to 25 to 50 % of the DTM buffer size to increase session performance. You might need to test different settings for optimal performance. Enter a value between 4096 and 2147483648 bytes.Default is 8388608 bytes.
Pipe Directory Path Path for the PowerCenter Integration Service to create the pipe for the external table. If you do not specify the path, the PowerCenter Integration Service uses the following directory to create the pipe for the external table:<PowerCenter Installation Directory>/server/binRequired if the machine hosting the PowerCenter Integration Service is on HP-UX and the following directory is on an NFS-mounted directory:<PowerCenter Installation Directory>/server/binEnter a path that does not use an NFS mount.
Error Log Directory Name
Error log directory can reside on the machine where the PowerCenter Integration Service runs. For example, you can use the following directory:$PMBadFileDirBy default, the PowerCenter Integration Service creates the error log in the following directory on the machine hosting the Netezza Performance Server:/tmpThe PowerCenter Integration Service creates a bad file in the error log directory if the data is not valid.
14 Chapter 4: Netezza Sessions and Workflows
Target Property Description
Truncate Target Table Option
The PowerCenter Integration Service truncates the target before loading. Run the truncate table command. Default is disabled. If you specify an SQL statement in the Pre-SQL property, the PowerCenter Integration Service runs the SQL statement before the table is truncated.Note: In normal mode, run the delete command. If you specify an SQL statement in the Pre-SQL property, the PowerCenter Integration Service runs the SQL statement after the data in the table is deleted.
Target Table Name You can override the default target table name.
Delimiter Set the delimiter to any value supported by the Netezza Performance Server. The delimiter separates successive input fields. The value must not be a part of the input data. Default is |.
Control Character CTRLCHARS parameter of the external table to transfer data containing control characters. You can enter control characters for Char and Varchar fields. If you enter a control character, you must add an escape character for the NULL, CR, and LF fields. Default is TRUE.
CRINSTRING CRINSTRING parameter to transfer data containing carriage returns (CR). You can enter a non escape CR in Char or Varchar fields. To load the control characters present in the Char and Varchar fields, set the CTRLCHARS and CRINSTRING parameters to TRUE in the session properties for the Netezza source. Default is TRUE.
NullValue NullValue parameter of the external table. The PowerCenter Integration Service uses the NullValue internally. Maximum value is one character. Default is blank.
EscapeCharacter Escape character of the external table. If the data contains NULL, CR, and LF characters in the Char or Varchar field, you need to add an escape character for these fields before loading. Enter a backslash (\) as the escape character.
Quoted Value QUOTEDVALUE parameter of the external table. Select SINGLE or DOUBLE to enclose the field in single or double quotes. Select NO to omit quotes. Default is NO. The quoted value is not a part of the data.
Ignore Key Constraints
Ignores constraints on primary key fields. When you select this option, the PowerCenter Integration Service can write duplicate rows with the same primary key to the target. Default is disabled. The PowerCenter Integration Service ignores this value when the target operation is “update as update” or “update else insert.”
Duplicate Row Handling Mechanism
Determines how the PowerCenter Integration Service handles duplicate rows. Select one of the following values:- First Row. The PowerCenter Integration Service passes the first row to the target and rejects
the rows that follow with the same primary key.- Last Row. The PowerCenter Integration Service passes the last duplicate row to the target
and discards the rest of the rows.Default is First Row.
Add Escape Character
Adds an escape character to all the special characters in the data. The special characters include \n, \r, \0, delimiter, escape character, and null value character. Ensure that the value of the escape character is entered in the EscapeCharacter attribute.
Null Values and Empty StringsThe target may contain null values even if you configure a column in the source definition to be not null. The PowerCenter Integration Service loads empty strings as null values to the target.
Session Configuration with a Netezza Target 15
Unprojected ColumnsWhen the PowerCenter Integration Service generates SQL to load to a Netezza target, it ignores target columns that are not connected in the mapping. If a default value is defined in Netezza for an unconnected column, Netezza updates or populates the column with the default value.
Pipeline PartitioningYou can increase the number of partitions in a pipeline to improve session performance. When you increase the number of partitions, the PowerCenter Integration Service can create multiple connections to sources and targets and process partitions of sources and target data concurrently.
The Netezza Performance Server divides data into data slices. In a partitioned session that reads data from Netezza, each partition reads a different data slice to prevent data duplication except in the following cases:
• You enter an SQL override query for a partition.
• You enter different values for the source filter across partitions.
• You enter different values for the user-defined join across partitions.
Rules and Guidelines for Pipeline PartitioningUse the following rules and guidelines when you configure multiple partitions in a Netezza session:
• If you load in bulk mode, use pass-through partitioning. If you load in normal mode, you can use database, hash, key range, pass-through, or round-robin partitioning.
• Verify that the session properties for delete and update on the Mapping tab are not enabled for more than one partition. You cannot perform multiple updates, multiple deletes, or update and delete simultaneously on a Netezza target.
• To avoid unpredictable session results, configure the session properties for insert, delete, update, and duplicate row handling to have the same value for each partition.
• If you run a partitioned session that joins multiple sources, link the first column in the Source Qualifier transformation to a source column that represents data for the Netezza table with the best distribution in Netezza. This means that the Netezza table is more uniformly distributed across Snippet Processing Units (SPU) than other tables.
• If you run a partitioned session with key constraints, only one partition shows load statistics.
Target Connection GroupsA target connection group is a group of targets that the PowerCenter Integration Service uses to determine commits and loading. When the PowerCenter Integration Service writes to Netezza, it commits data in the same transaction for all targets in a target connection group. When the PowerCenter Integration Service needs to perform a rollback, the PowerCenter Integration Service rolls back all targets in the target connection group.
Criteria for using Target Connection Groups in Bulk ModeWhen you load data in bulk mode, all Netezza targets in the same target connection group must meet the following criteria:
• Belong to the same pipeline.
• Belong to the same partition.
• Have the same database connection name, user name, and password.
16 Chapter 4: Netezza Sessions and Workflows
Rules and Guidelines for using Target Connection Groups in Bulk ModeThe following table describes the rules and guidelines that you can use when you configure multiple targets in a target connection group to write to the same Netezza target table in bulk mode:
Target Load Type
Target Options Rules and Guidelines
Insert InsertUpdate as Insert
Select the Ignore Key Constraints target property for insert targets.
Update Update as UpdateUpdate else Insert
Use a maximum of one update table for any target.Do not use with delete tables.
Delete Delete Use a maximum of one delete table for any target.Do not use with update tables.
Multiple Targets Configuration for the Same Target TableYou can configure multiple targets to write to the same Netezza table, even if they are not in the same target connection group. When you configure targets in different partitions or pipelines to write to the same target table, use the same rules and guidelines as for target connection groups.
Netezza Target Data UpdateIn bulk mode, the PowerCenter Integration Service updates target rows based on the update options and the duplicate row handling.
Update As InsertWhen you configure the session to update as insert rows, the PowerCenter Integration Service uses the following process to update target rows:
• If the source key value matches a target key value, the PowerCenter Integration Service does not insert the source row.
• If the source primary key value does not exist in the target, the PowerCenter Integration Service inserts the source row.
The following table describes how the PowerCenter Integration Service updates the target:
Source Data Target Data Updated Target Data
Comment
1,a,1a1 - - The source primary key is found in the target. The row is not inserted.
1,b,1b1 - 1,b,1b1 Inserts 1,b,1b1.
Netezza Target Data Update 17
Source Data Target Data Updated Target Data
Comment
1,a,1a2 - - The source primary key is found in the target. The row is not inserted.
1,c,1c1 1,c,1c1 1,c,1c1 The source primary key is found in the target. The existing row 1,c,1c1 is retained. No insert is required.
1,d,1d1 - 1,d,1d1 Inserts 1,d,1d1.
1,a,1a3 1,a,1a3 1,a,1a3 The source primary key is found in the target. The existing row 1,a,1a3 is retained. No insert is required.
Note: In the pair of values, the first two values are the primary key, for example 1 (primary key), a (primary key), 1a1. The session is configured to consider key constraints.
Update Else InsertWhen you configure the session to update else insert rows, the PowerCenter Integration Service uses the following process to update target rows:
• If the source key value matches a target key value, the PowerCenter Integration Service updates each target row. It updates with the first or last source row matched, based on how you configure duplicate row handling.
• If the source primary key value does not exist in the target, the PowerCenter Integration Service inserts the source row.
The following table describes how the PowerCenter Integration Service updates the target:
Source Data
Target Data
Updated Target Data
Comment
1,2 1,6 1,2 Updates 1,6 with 1,2.The source primary key is found in the target. The target row is updated based on duplicate row handling to use first row.
1,3 1,8 1,2 Updates 1,8 with 1,2.Duplicate row handling is configured to update with first source row. Subsequent target rows with primary key “1” are updated with first source row.
2,4 - 2,4 Inserts 2,4.The source primary key is not found in the target. The row is inserted.
2,5 - - Drops 2,5.The source primary key is found in the target, and first duplicate row has been updated in the target.
- 3,7 3,7 Retains 3,7.No update required.
Note: In the pair of values, the first value is the primary key, for example 1 (primary key), 2.
18 Chapter 4: Netezza Sessions and Workflows
Netezza Session Configuration for Optimal Performance
You can increase the performance of PowerExchange for Netezza by setting the properties in the session. Set the following parameters to increase the session performance:
• Default buffer block size. You can increase or decrease the number of available memory blocks that are used to hold the source and target data in the session.
• Line sequential buffer length. You can improve the session performance by setting the number of bytes the PowerCenter Integration Service reads per line.
• Commit interval. You can increase or decrease the value of commit interval to determine the point at which the PowerCenter Integration Service commits data to the target.
• DTM buffer size. You can increase or decrease the value of DTM Buffer Size to specify the amount of memory the PowerCenter Integration Service uses as DTM buffer memory.
• Socket buffer size. You can configure the socket buffer size to specify the size of the buffers used to extract data from and load data to Netezza.
• Escape characters. You can improve session performance by avoiding the use of escape characters in a session.
• Ignore key constraints. You can improve session performance by ignoring key constraints when writing to Netezza targets. Since Netezza does not enforce key constraints, the PowerCenter Integration Service performs additional processing when a session that writes to Netezza requires key constraints.
For example, to obtain optimal performance for 3 million rows and 32 KB row size, set the following parameters:
• Default buffer block size: 1,280,000
• Line sequential buffer length: 202,400
• Commit interval: 200,000
• DTM buffer size: 28,000,000
• Socket buffer size: 8388608 bytes
• Escape characters: None
• Ignore key constraints: Selected
Netezza Distribution KeyUse a Netezza distribution key to increase session performance with parallel processing. Netezza uses a distribution key to distribute data for processing. By default, the distribution key is the first column of a table.
You can configure the distribution key to include up to four columns in a database table. When you configure a distribution key to evenly distribute data across available data slices, you can greatly increase session performance. For more information, see the Netezza documentation.
Troubleshooting Netezza Sessions
A Netezza session stops responding with no definite error messages in the logs
Netezza Session Configuration for Optimal Performance 19
A Netezza session can stop responding because of the following reasons:
• The source data contains special characters like delimiter.A Netezza Reader session can stop responding if the source contains special characters like delimiter. Add escape characters in the session to eliminate the delimiters. This is a Netezza issue and the reference number is SWS-40577.
• Netezza drivers 3.1.2 or 3.1.4 are used.Multi-pipe and multi-partition sessions can stop responding randomly with Netezza drivers 3.1.2 or 3.1.4 on HP-UX. Use the Netezza driver 4.04 P2 to avoid this issue.
• The pipe directory path on HP-UX is on an NFS-mounted drive.If the pipe directory path on HP-UX is on an NFS-mounted drive, Netezza writer sessions can stop responding or terminate unexpectedly. Specify a non NFS-mounted drive (for example, /tmp) in the pipe directory path of the Netezza Writer session property.
• The environment variables are set incorrectly.Check whether the environment variables PATH, LIBPATH, ODBCINI, and NZ_ODBC_INI_PATH are set correctly.
• The permissions for the file paths in the session properties are set incorrectly.Ensure that all the file paths in the session properties set for Netezza reader and writer sessions are correct and have proper permission. Check out for the ones which need directory path specification.
If the issue persists, you can try killing the blocking Netezza sessions or disable Netezza ODBC tracing and ODBC tracing.
How can I kill blocking Netezza sessions?
Use the nzsession utility, which comes with client tools, to kill blocking Netezza sessions.
Run the following command to view the active Netezza sessions:
nzsession show -host <hostname> -u <user> -pw <password> -maxColW <column width> |grep -i "active
Run the following command to kill active sessions:
-host <hostname> -u <user> -pw <password> -id <session id> [-force]
How can I enable or disable Netezza ODBC tracing?
In the odbcinst.ini file, set the parameter debugLogging as true to enable Netezza ODBC tracing and as false to disable Netezza ODBC tracing.
How can I enable or disable ODBC tracing?
In the odbc.ini file, set the parameter Trace as 1 to enable Netezza ODBC tracing and as 0 to disable Netezza ODBC tracing.
20 Chapter 4: Netezza Sessions and Workflows
A P P E N D I X A
Datatype ReferenceThis appendix includes the following topic:
• Netezza and Transformation Datatypes, 21
Netezza and Transformation DatatypesPowerCenter uses the following datatypes in Netezza mappings:
• Netezza native datatypes. Netezza datatypes appear in Netezza definitions in a mapping.
• Transformation datatypes. Set of datatypes that appear in the transformations. They are internal datatypes based on ANSI SQL-92 generic datatypes, which the PowerCenter Integration Service uses to move data across platforms. They appear in all transformations in a mapping.
When the PowerCenter Integration Service reads source data, it converts the native datatypes to the comparable transformation datatypes before transforming the data. When the PowerCenter Integration Service writes to a target, it converts the transformation datatypes to the comparable native datatypes.
The following table lists the Netezza datatypes that PowerCenter supports and the corresponding transformation datatypes:
Netezza Datatype
Range Transformation Datatype
Range
BigInt Precision 19, scale 0 Bigint From -9,223,372,036,854,775,808 through 9,223,372,036,854,775,807Precision of 19, scale of 0 Integer value
Bool True or false, on or off, 0 or 1, yes or no
String Precision 1
ByteInt Precision 3, scale 0 Small Integer Precision 5, scale 0
Char Single character String From 1 through 104,857,600 characters
Date ANSI SQL date Date/Time Jan. 1, 0001 A.D. to Dec. 31, 9999 A.D.(precision to the nanosecond)
Float8 Precision 15 Double Precision 15
Float4 Precision 6, scale 0 Double Precision 15
21
Netezza Datatype
Range Transformation Datatype
Range
Integer Precision 10, scale 0 Integer Precision 10, scale 0
NChar(m) Single characterUsed for storing UTF-8 data
String From 1 through 104,857,600 characters
NVarchar(m) BVarchar (length)Non-blank-padded string, variable storage lengthUsed for storing UTF-8 data
String From 1 through 104,857,600 characters
Numeric Numeric (precision, decimal), arbitrary precision numberPrecision must be between 1 and 38
Decimal Precision from 1 through 28 digits, scale from 0 through 28
Real Precision 6, scale 0 Real Precision of 7, scale of 0Double-precision floating-point numeric value
SmallInt Precision 5, scale 0 Small Integer Precision 5, scale 0
Time hh:mm:ss. ANSI SQL time Date/Time Jan. 1, 0001 A.D. to Dec. 31, 9999 A.D.(precision to the nanosecond)
Timestamp Precision 26, scale 6 Date/Time Jan. 1, 0001 A.D. to Dec. 31, 9999 A.D.(precision to the nanosecond)
Varchar Varchar (length)Non-blank-padded string, variable storage length
String From 1 through 104,857,600 characters
22 Appendix A: Datatype Reference
I N D E X
Ddatabases
connecting to Netezza (UNIX) 4connecting to Netezza (Windows) 3
datatypes PowerExchange for Netezza 21
default values Netezza targets 16
Eempty strings
in Netezza 15
HHP-UX
pipe directory path, setting 13
Iinstallation
Netezza prerequisites 2
Kkey constraints
example 17
Mmultiple targets
for the same Netezza table 17
NNetezza
connecting from an integration service (Windows) 3connecting from Informatica clients(Windows) 3connecting to an Informatica client (UNIX) 4connecting to an integration service (UNIX) 4
Netezza target connection groups using multiple targets for the same table 16
null values in Netezza 15
Ppartitioning
Netezza sessions 16pipe directory path
setting 13setting for HP-UX 13
plug-ins registering for Netezza 3
PowerExchange for Netezza connections creating 12overview 12properties 12
prerequisites Netezza installation 2
Ssocket buffer size
target property 13, 14Source Qualifier
Netezza, overview 7
Ttarget connection groups
using with Netezza 16target property
socket buffer size 13, 14targets
Netezza default values 16unprojected columns in Netezza 16using multiple for the same Netezza table 17
Uupdate as insert
description for Netezza 17update else insert
description for Netezza 18update strategy
example 17update as insert for Netezza 17update else insert for Netezza 18
23
