20
© 2013 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. All other company and product names may be trade names or trademarks of their respective owners and/or copyrighted materials of such owners. Big Machines User Guide Beta Version

Title of the H2L Article - Informatica ETL Tool · example,...\...\BigMachines\config\DataTables.tbl. WSDL Folder Path Enter the required WSDL file path. Endpoint URL Enter the required

  • Upload
    dodien

  • View
    218

  • Download
    0

Embed Size (px)

Citation preview

© 2013 Informatica Corporation. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. All other company and product names may be trade names or trademarks of their respective owners and/or copyrighted materials of such owners.

Big Machines User Guide

Beta Version

2

Abstract

Big Machines user guide provides a brief introduction on cloud connectors and its features.

The guide provides detailed information on setting up the connector and running data synchronization tasks (DSS). A brief overview of supported features and task operations that can be performed using Big Machines connector is mentioned.

Table of Contents

Overview .......................................................................................................................................... 3

Big Machines Connector ................................................................................................................. 3

Supported Modules and Task Operations ....................................................................................... 4

Transaction Module ..................................................................................................................... 4

Configuration Module ................................................................................................................... 4

Enabling Big Machines Connector .................................................................................................. 4

Creating a Big Machines Connection .............................................................................................. 5

Creating a Big Machines Data Synchronization Task ..................................................................... 6

Data Filters .................................................................................................................................... 11

Modules and Operators ............................................................................................................. 12

Generating Client-Stub for Big Machines WSDL, using WSimport ............................................... 13

To create login JAR / WSDL ...................................................................................................... 13

Commands to generate Quote.WSDL ....................................................................................... 14

Commands to generate Data tables.WSDL ............................................................................... 15

Commands to generate Parts.WSDL ........................................................................................ 15

Commands to generate Config WSDL ...................................................................................... 15

WSDL File Path Screenshot .......................................................................................................... 16

Adding Datatables ......................................................................................................................... 16

Sample Datatables.tbl file .......................................................................................................... 17

Creating Schema File .................................................................................................................... 17

Sample Schema file ................................................................................................................... 18

Frequently Asked Questions and Answers ................................................................................... 19

3

Overview

Informatica cloud connector developed using SDK framework are off-cycle, off release “add-ins” that facilitate data integration to SaaS and on-premise applications, which are not supported natively by Informatica cloud. The cloud connectors are specifically designed to address most common use cases such as moving data into cloud and retrieving data from cloud for individual application.

Figure 1. Informatica Cloud Architecture

Once the Big Machines cloud connector is enabled for your ICS ORG Id, you need to create a connection in Informatica cloud to access the connector.

Big Machines Connector

The Big Machines cloud connector offers customer to integrate parts, transactions like quotes with on-premise or cloud applications like SalesForce.

The connector also supports extracting/loading custom objects (data tables) between big machines and on premise or cloud applications.

4

Supported Modules and Task Operations

The following table provides the list of supported modules and operations by Big Machines connector.

Modules

Task Operations Data

Preview Export File Attachment

Import File Attachment

Fetch Insert Upsert

Update Delete

Transactions

NA NA NA NA

Parts NA NA NA

Data Tables NA NA NA NA

Configuration NA NA

NA NA NA NA

NA

Note: The objects of data table module will be displayed with prefix 'DT_'. For example, 'DT_abcd'

Transaction Module

The transaction module contains following objects as a part of fetch operation

Export File Attachment

Import File Attachment

Configuration Module

The configuration module supports following objects as a part of fetch operation:

Configure

Get_Price

Get_BoM

Get_Spare

Enabling Big Machines Connector

To enable Big Machines connector, contact Informatica support or Informatica representative. It usually takes 15 minutes for the connector to download to secure agent, after it is enabled.

Note: To install secure agent, see Installing Informatica Secure Agent.

: Supported

: Not Applicable

5

Creating a Big Machines Connection

To use Big Machines connector in data synchronization task, you must create a connection in Informatica Cloud.

Note: When you create a connection for data tables’ module, it is mandatory to have configured schema file and mention its file path while creating the connection

The following steps help you to create Big Machines connection in Informatica Cloud.

1. In Informatica Cloud home page, click Configure.

2. The drop-down menu appears, select Connections.

3. The Connections page appears.

4. Click New to create a connection.

5. The New Connection page appears.

Figure 2. Creating a new connection

6

6. Specify the following details.

Connection Property Description

Connection Name Enter a unique name for the connection.

Description Provide a relevant description for the connection.

Type Select Big Machines from the list.

Secure Agent Select the appropriate secure agent from the list.

Username Enter relevant username.

Password Enter relevant password.

Data Tables Schema Path Enter the required schema file path. For example,...\...\BigMachines\config\DataTables.tbl.

WSDL Folder Path Enter the required WSDL file path.

Endpoint URL Enter the required endpoint URL.

Attribute Control File Path

Enable Logging Tick the checkbox to enable logging.

Note: When WSDL URL is not mentioned in connection properties, the default WSDL file path packaged along with connector will be selected as WSDL URL.

The default WSDL URL file path is :\Program Files\Informatica Cloud Secure Agent\main\tomcat\plugins\ <pluginid> \ <module name> \. See WSDL File Path Screenshot.

Note: The Data Table schema file name must contain .tbl extension. The table header names mentioned in the schema file must have a separate .sch file in the same folder path. The .sch files define field details of the table names present in the schema file. See Adding Datatables

7. Click Ok to save the connection.

Note: It is recommended to test the connection before saving it. Click Test to evaluate the connection.

Note: When you test the connection, a config file will be created in the connector directory.

Creating a Big Machines Data Synchronization Task

Note: You need to create a connection before getting started with data synchronization task.

The following steps help you to setup a data synchronization task in Informatica Cloud.

Note: A config file generated while creating a connection contains all the field display names and Xpath of the fields/columns.

7

The location of the config file is C:\Program Files\Informatica Cloud Secure Agent\main\tomcat\plugins\<pluginid>\<module name>\.

For example, C:\Program Files\Informatica Cloud Secure Agent\main\tomcat\plugins\517400\Transaction

Let us consider the task operation Insert to perform the Data synchronization task.

1. In Informatica Cloud home page, click Apps.

2. The drop-down menu appears, select Data Synchronization.

3. The Data Synchronization page appears.

4. Click New to create a data synchronization task.

5. The Definition tab appears.

Figure 3. Definition Tab

6. Specify the Task Name, provide a Description and select the Task Operation Insert.

7. Click Next.

8. The Source tab appears.

8

Figure 4. Source Tab

9. Select the source Connection, Source Type and Source Object to be used for the task.

10. Click Next.

11. The Target tab appears. Select the target Connection and Target Object required for the

task.

Note: When you want to setup update/delete task operation for transaction module then,

- In target tab, select target object "Transaction_Update" for update task operation.

- In target tab, select target object “Transaction_Delete" for delete task operation.

Note: when you setup an ‘insert’ task operation for transaction module, the "TransactionID" field in target tab must be populated with an existing transaction ID. Otherwise, a new transaction will be created.

Note: when you setup a ‘delete’ task operation for transaction module, you must populate "TransactionId" and "Transaction_LineItem" in target tab. The delete task operation is based on Transaction_LineItemsNo and TransactionId.

9

Figure 5. Target Tab

12. Click Next.

13. In Data Filters tab by default, Process all rows is chosen. To assign filters to select specific data, see Data Filters.

14. Click Next.

15. In Field Mapping tab, map source fields to target fields accordingly.

Note: You can edit the config file to show/hide the fields in the field mapping. You can show/hide the fields in field mapping, by changing the status of display field from 'true' to 'false' or vice versa. See Configfile.

10

Figure 6. Field Mapping Tab

16. Click Next.

17. The Schedule tab appears.

18. In Schedule tab, you can schedule the task as per the requirement and save.

19. If you do not want schedule the task, click Save and Run the task.

Figure 7. Save and Run the Task

After you Save and Run the task, you will be redirected to monitor log page. In monitor log page, you can monitor the status of data synchronization tasks.

11

Data Filters

Data filters help you to fetch specific data of a particular object. The DSS task will process the data based on the filter field assigned to the object.

Note: When you setup the data synchronization task for Transaction object, it is mandatory to apply the filter field "_FLT_TransactionFileName". The Transaction file must contain a column with header name 'transaction_id'.

Note: All the fields present in data tables object can be used as data filters. You can create custom filter fields for data tables object.

Note: Big Machine connector supports basic filters only. Advanced data filters are not supported by Big Machines connector.

The following steps help you to use data filters.

1. In Data synchronization task, select Data Filters tab.

2. The Data Filters tab appears.

3. Click New as shown in the figure below.

Figure 8. Data Filters

4. The Data Filter dialog box appears.

12

Figure 9. Data Filters-2

5. Specify the following details.

Field Type Description

Object Select Object for which you want to assign filter fields.

Filter By Select the Filter Field.

Operator Select the corresponding operator.

Filter Value Enter the filter value.

6. Click Ok.

Note: The data filters support only Equals operator for the objects of transaction module.

Modules and Operators

The following table provides information on the various operators supported by modules

Modules Supported Operators

Transaction Equals

Parts Equals, notEquals, greaterThan, lessThan, greaterOrEquals, lessOrEquals, contains, startsWith, endsWith

Data Tables Equals, notEquals, greaterThan, lessThan, greaterOrEquals, lessOrEquals, contains, startsWith, endsWith

Note: The supported operators are valid only for Query or fetch operation.

13

Generating Client-Stub for Big Machines WSDL, using WSimport

The following steps help you to create a Stub (i.e. Bigmachines-<ModuleName>.jar) for the BigMachines WSDL.

Follow the given filename format to create a WSDL file for the different services:

Service Filename

SecurityService Login.wsdl

PartService Parts.wsdl

CommerceService Quote.wsdl

DataTablesService Datatables.wsdl

ConfigurationService Configure.wsdl

To create login JAR / WSDL

1. Go to jdk1.7.0_25 download page.

2. Download the jdk that is compatible with your system.

Figure 1: JDK Download

14

3. After you download and install the jdk successfully, make use of the commands given below to create the Big Machines stub.

Note: You can run the command one after another in the command prompt or copy and paste the entire command list to a win batch file and then run it as a batch file.

set JAVA_HOME= C:\Program Files\Java\jdk1.7.0_25\bin

set classpath=%JAVA_HOME%

set PATH=%JAVA_HOME%;%PATH%

mkdir C:\Applications\BigMachines\MetaData_Jars\BM

mkdir C:\Applications\BigMachines\MetaData_Jars\classes

wsimport C:\Applications\BigMachines\MetaData_Jars\Login.wsdl -B-

XautoNameResolution -XadditionalHeaders -Xnocompile -d

C:\Applications\BigMachines\MetaData_Jars\BM -keep -p

com.bigmachines.login.api

javac -cp .;"%classpath%"

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\login\api\*.jav

a

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\login\api\runti

me\*.java -d C:\Applications\BigMachines\MetaData_Jars\classes

jar -cvf C:\Applications\BigMachines\MetaData_Jars\BigMachines-login.jar -C

C:\Applications\BigMachines\MetaData_Jars\classes com

4. After successfully running the commands, a new Big Machines-login.jar is created.

5. Stop the Informatica Cloud secure agent.

6. Copy/replace the BigMachines-login jar to the path

- c:\Progra~1\Informatica Cloud Secure Agent\main\bin\rdtm\javalib\<plugin-id number>

- c:\Progra~1\Informatica Cloud Secure Agent\main\tomcat\plugins\<plugin-id number>

7. Restart the Informatica Cloud secure agent.

Note: Repeat the above mentioned procedure, creating login JAR / WSDL to generate other JAR/ WSDL files and replace only the set of commands given below to generate different WSDL files.

Note: Delete the content of C:\temp\BM and C:\temp\classes before creating the jar for next WSDL.

Commands to generate Quote.WSDL

wsimport C:\Applications\BigMachines\MetaData_Jars\Quote.wsdl -B-

XautoNameResolution -XadditionalHeaders -Xnocompile -d

C:\Applications\BigMachines\MetaData_Jars\BM -keep -p

com.bigmachines.quote.api -b

C:\Applications\BigMachines\MetaData_Jars\bm.jxb

javac -cp .;"%classpath%"

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\quote\api\*.jav

a

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\quote\api\runti

me\*.java -d C:\Applications\BigMachines\MetaData_Jars\classes

jar -cvf C:\Applications\BigMachines\MetaData_Jars\BigMachines-quote.jar -C

C:\Applications\BigMachines\MetaData_Jars\classes com

15

Commands to generate Data tables.WSDL

wsimport C:\Applications\BigMachines\MetaData_Jars\Datatables.wsdl -B-

XautoNameResolution -XadditionalHeaders -Xnocompile -d

C:\Applications\BigMachines\MetaData_Jars\BM -keep -p

com.bigmachines.datatable.api

javac -cp .;"%classpath%"

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\datatable\api\*

.java

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\datatable\api\r

untime\*.java -d C:\Applications\BigMachines\MetaData_Jars\classes

jar -cvf C:\Applications\BigMachines\MetaData_Jars\BigMachines-datatable.jar

-C C:\Applications\BigMachines\MetaData_Jars\classes com

Commands to generate Parts.WSDL

wsimport C:\Applications\BigMachines\MetaData_Jars\Parts.wsdl -B-

XautoNameResolution -XadditionalHeaders -Xnocompile -d

C:\Applications\BigMachines\MetaData_Jars\BM -keep -p

com.bigmachines.parts.api

javac -cp .;"%classpath%"

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\parts\api\*.jav

a

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\parts\api\runti

me\*.java -d C:\Applications\BigMachines\MetaData_Jars\classes

jar -cvf C:\Applications\BigMachines\MetaData_Jars\BigMachines-parts.jar -C

C:\Applications\BigMachines\MetaData_Jars\classes com

Commands to generate Config WSDL

wsimport C:\Applications\BigMachines\MetaData_Jars\Configure.wsdl -B-

XautoNameResolution -XadditionalHeaders -Xnocompile -d

C:\Applications\BigMachines\MetaData_Jars\BM -keep -p

com.bigmachines.configure.api

javac -cp .;"%classpath%"

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\configure\api\*

.java

C:\Applications\BigMachines\MetaData_Jars\BM\com\bigmachines\configure\api\r

untime\*.java -d C:\Applications\BigMachines\MetaData_Jars\classes

jar -cvf C:\Applications\BigMachines\MetaData_Jars\BigMachines-config.jar -C

C:\Applications\BigMachines\MetaData_Jars\classes com

16

WSDL File Path Screenshot

Figure 11. WSDL File Path

Adding Datatables

Follow the give steps to add Datatables in source or target for Bigmachines DSS task,

1. At first, enter “Datatables.tbl” file in ‘Datatables Schema Path’ field while creating a Bigmachines Connection.

2. The default .tbl file is located in the following folder path

For example, “C:\a_IOD\Files\BigMachines\Datatables.tbl”

The Datatables.tbl file needs to be created in specific format as shown in the figure below.

17

You can create Datatables.tbl file using the following table definitions:

Header Name Allowable Value Remarks

ISCUSTOMOBJ Y Only Y is supported in the current release.

OPR_READ Y, Any Character When you enter Y, the table is displayed in source Tab. When you enter any other character, the table will not be displayed in source tab.

OPR_CREATE N Insert (create) task operation is not supported for Datatable.

OPR_UPDATE N Update task operation is not supported for Datatable.

OPR_DELETE Y, Any Character Enter Y to support delete task operation.

OPR_UPSERT Y, Any Character Enter Y to support upsert task operation.

TABLE_NAME Any Name Object name as shown in Informatica cloud UI.

OBJECT_TYPE CUSTOMOBJECT Enter CUSTOMOBJECT as static value, same as table_name

CUSTOMOBJ_NAME Any Name Custom Object, mostly same as Table Name.

Sample Datatables.tbl file

You can create a sample Datatables.tbl file using the data given below:

1. Copy and paste the sample data given below in any blank text file

"ISCUSTOMOBJ","OPR_READ","OPR_CREATE","OPR_UPDATE","OPR_DELETE","OPR_UPSERT","T

ABLE_NAME","BUSINESS_TABLE_NAME","CUSTOMOBJ_NAME"

"Y","Y","N","N","Y","Y","Supplemental","Supplemental","Supplemental"

"Y","Y","N","N","Y","Y","Retail_Visit","Retail_Visit","Retail_Visit"

2. Name the file as Datatables.tbl.

3. Click Save

Creating Schema File

You must create schema file to define each Datatable added in Datatables.tbl file.

Schema file helps you to configure each Datatable parameters. The schema file name must be similar to the value mentioned TABLE_NAME Field of Datatables.tbl file. You must create a schema file in a specific format as shown in the figure below.

18

Figure 11. Schema File

Header Name Allowable Value Remarks

ISKEY Y, Any Character When you enter Y the Key Field is active. Define Key field, when you want to add Upsert operation for Datatable.

ISNULLABLE Y, Any Character When you enter ‘Y’, the field is defined as mandatory or non-nullable in Informatica Cloud DSS Task.

ISFILTERABLE Y, Any Character When you enter ‘Y’ the field is displayed in the data filters tab for the selected Datatable.

DATATYPE Integer, string, date, Boolean,

long, float

Any data type mentioned in the ‘Allowable Value’ is supported in the schema file.

PRECISION Valid Number Set the precision required using this field. Note: Precision for string data type is variable, while for other data types it is constant.

SCALE Valid Number The scale value should be equal to precision value.

FLD_DISPLAY_NAME Characters A to Z or a to z.

Special character Underscore.

The field name that is visible in Informatica Cloud UI. Alphabets (both Caps and small) are allowed. Only underscore ’_’ special character is allowed.

FLD_NAME Any MS Latin1 Characters This name is used for RPC call to the Webservice Endpoint.

FLD_DESCRIPTION - Field description

Y-Yes N-No

Sample Schema file

You can create a sample schema file using the data given below.

1. You can copy and paste the sample data given below in any blank text file, to create a file.

19

ISKEY","ISNULLABLE","ISFILTERABLE","DATATYPE","PRECISION","SCALE","FLD_DISPLAY_

NAME","FLD_NAME","FLD_DESCRIPTION"

"Y","N","Y","integer",10,10,"IndIdNbr","IndIdNbr","Lead Id"

"N","Y","Y","string",256,256,"LetterType","LetterType","Typoe of Letter"

"Y","Y","Y","date",256,256,"LatestLtrDate","LatestLtrDate","Latest Letter Date"

"N","Y","Y","string",256,256,"ModelMatch","ModelMatch","Model Type"

"N","Y","Y","string",256,256,"CampaignName","CampaignName","Name of the

Campaign"

"N","N","Y","date",256,256,"PromoDate","PromoDate","Date of Promotion"

"N","Y","Y","long", 15, 15,"TmOperatorId","TmOperatorId","Operator Id"

2. Name the file and mention the file extension as .SCH

3. Click Save

Frequently Asked Questions and Answers

1. For which objects is data filter mandatory?

Answer:

Parts, Datatables, Transactions

2. What are the different objects supported by Transaction module of Big Machine?

Answer:

Given below are the objects supported in Transaction operation:

Insert : This object creates new transactionid and also adds transaction line item to transactionid

Update: This object updates the existing transaction and adds lineItem to existing transaction

Delete: This object helps you to delete line item from a given transaction.

3. What are the different types of Attachment?

Answer:

Given below are the two types of attachments:

exportAttachment - file available in base64 format.

importAttachment - you can provide file in a port/field in base64 format.

4. How does the delete work in Datatables or Parts?

Answer:

delete in datatable is used to delete rows or records

delete in parts is allowed to delete a part

delete in transaction is allowed to remove line from the transaction

20

5. Which modules support batch insert?

Answer:

Parts, Datatable, Update