Atrium Integration Engine: What Is It? How Do You Use It? · 2020-05-19 · Atrium Integration Engine: What Is It? ... – Ideal for bulk, incremental, data synchronization transfers

4/9/20101

Atrium Integration Engine: What Is It? How Do You Use It?

› Atrium Integration Engine (AIE) Overview– What is it?– Overview of latest features

› Tell me how to do it– Under the hood details of creating CMDB Integration using AIE

© Copyright 4/9/2010 BMC Software, Inc.2

BMC Atrium Integration Engine

› What is it?– A mechanism to manage data transfers between external data sources and

CMDB (also AR Forms)

– Bi-directional and multi-threaded data transfer engine

– An Enterprise Application Integration tool with data transformation capability

– It supports event-based or scheduled information flow

– Ideal for bulk, incremental, data synchronization transfers

– Provides interfaces to external data source and the CMDB

– Provides an intuitive graphical interface for field mapping configuration


BMC Atrium Integration Engine

› How is it used?– Transform data to the CMDB data model– Efficiently load data in bulk– Can create relationships between instances as they are inserted in the CMDB.

› Advantages– Designed for complex integrations– Easily configurable– Can be used against a variety of applications– Extensible

› When Is AIE a Fit?– Complex transfers requiring data transformation – Handling large amount of data transfers– Multiple integration points


BMC Atrium Integration Engine 7.1

› New Name and Repackaging for EIE› Released in late August 2007› Broad Enhancement Areas

– Performance and Scalability• Improved performance of Relationship exchanges• Better memory management for all update operations

– Usability Improvements• Lot of usability improvements (please see the release notes for details on new features)

› Packaging– Ships with Atrium CMDB 2.1 to integrate with Atrium CMDB– One product, multiple adapters

• Oracle • SQL • DB2 New!• XML New!

– XML adapter only supports class based relationships (not vendor)• Flat file Improved!


How to Create AIE Based Integrations to CMDB: The Ingredients To Success

› Data Structure Knowledge› External Data from Discovery/Inventory› Atrium Integration Engine› Atrium CMDB› Reconciliation Rules› Workflow Filters – if Needed


AIE Terminology

› Data Exchange Application (DEA)– AR System based application at the heart of AIE

› AIE Service– DEA passes the service instructions on how to interact with the adaptor

› Data Mapping– CMDB Class/AR Form mapped to External Table– CMDB Attributes (or fields of AR Form) mapped to Columns of External Table– Values mapped to External Table Columns

› Exchange– Collection of “similar” Mappings– Definition of when to run– “Pipe that manages the flow”

› Adapter– The Plug at the end of the wire

DataExchangeApplication

Data Mappings

Data ExchangesAdapter

AIEService


Discovery Environment Remedy Environment

Configuration SQL Example

SQL Database

DiscoveryAR System

(can be on either server)

DataExchangeApplication

Database for AR System

CMDBDiscovery

Dataset

Data Cleansing Workflow

CMDBBMC.ASSET

Dataset

RemedyAsset

Management

CMDBReconciliation

Engine

CMDBReconciliation

Rules

Data Mappings

Data ExchangesAdapter

AIEService


External Data Source

› Database data› XML data› Flat file data› Source Relationships can be stored as

– foreign key– relationship tables


Atrium CMDB

› Configuration Management Database– Store multiple datasets– Reconcile datasets– Store Relationships– Base data for OOTB Applications

› Data Sets– Data value in a field... but so much more– Data Sets you should use

• BMC.ASSET – The “Gold Standard”• BMC.SANDBOX – Temp location for GUI changes to Assets• VENDOR.PURPOSE.PRODUCT – custom datasets

– Examples: BMC.IMPORT.MARIMBA, MS.IMPORT.SMS, ACME.IMPORT.LEGACY


AIE Main Console


Configuration > Database Field Menus

› Populates the “vendor” table and column menus› Run before you start any mapping work

1. Load

2. Add Desired Tables

3. Create Menus


How To – Data Mapping

› Mapping Name = name for new mapping› Direction = External Data into CMDB› Namespace = BMC.CORE in most cases› Class = Destination for data, pick from the list› External Data Store = SqlServer, Flatfile or Oracle› Table Name = Source of data (pick from list)

– if not in the list, run rulehelper.exe to create the menu items.› Row Element = (XML only) the element that represents a row› Click “Create” to save

Data Exchange

Data Mapping Relationship Exchange

Relationship Mapping



If the Table Name does not show in the drop down list, it is because you have not configured the Database Field Menus from the main console

Data Exchange





› Used to map source and destination side primary/foreign keys› Keys must be unique on both sides

– If multiple keys, stored as AND to define Unique

Data Exchange





› Map appropriate CDMB Class.Attribute to a Data Store Attribute› Make sure to specify CMDB mandatory fields such as

– Fields use in Key mappings– Data Set ID (automatic if “Use Dataset” is checked)– Short Description – Submitter – Name = anything that identifies the entry– AssetLifeCycleStatus = 3 (if Asset Management is installed)

Data Exchange





› Rules – Works like “Function” key in Remedy User and Administrator Tools– Select by highlighting the row and click “Add”– Rule is added– Refer to the Rules box for hints on how to complete the rule

› Available Rules



› Only works if you can write to the Vendor / Source Database› Can be any value› Usually is completion status

– 0 = transfer with no error– N = error # - corresponds to ARERROR

Data Exchange





› If you “Update Row to Indicate Delete” i.e. Soft Delete– Set MarkAsDeleted to 1 (Yes)– Other attributes may be set as desired

› If you choose to “Permanently Delete” – Hard Delete– No field to set. AIE will do trigger the deletion

Data Exchange





- Use to limit amount of data in a given transfer. Important if you are experiencing performance or memory issues

- If you specified “Restrict to Dataset” on the Main Tab, you don’t need to include it here.

Data Exchange





1. If set to all data that can be transferred will be. This could result in partial data records.2. Transfer as NULL cleans up any data that might be mistaken for null in a field3. If trailing blanks are possible in your key values, choose to consider4. Stores change history in a diary field you designate. Attribute must exist in CMDB.5. If Yes, updates are only made when a change is detected in the source data

› You must select a field that can store a negative integer to store the checksum6. Allows you to retrieve new or change data since last transfer

› You must select a date/time field to store last transfer timeData Exchange



123456


How To - Data Exchange

› Exchange Name – Name you create to identify the exchange› Instance Name – Specify the AIE thread to use› Data Exchange Type – Specify the type of mapping contained in the exchange› External Data Store – Must match the mapping(s)› Execute Exchange As

– Schedule Only• Use Schedule tab

– Event Driven Only• run eiexfer.exe from command line, filter or active link

Data Exchange





› Select the mapping(s) to include in the Exchange container.› Multiple Mappings

– Cuts down on queries for source or target› Main Mapping

– Need to designate if multiple mappings included– Key values only come from Main Mapping

Data Exchange





› Interval– Works like the interval function in escalation workflow

› On Completion Section– When this exchange is done, you can configure

another exchange to run

Data Exchange





› Time– Works like the time function in escalation workflow– Set day and time to run

Data Exchange





› To Be Triggered– Use this option when you want to exchange to run on

completion of some other exchange– Must list the exchange in the On Completion section

on the other exchange

Data Exchange





You can either set the external connection information on each exchange or on the Connections link from the Data Exchange Console

Alternate Method

Use if the AR Server involved in the exchange is NOT the server that DEA is installed on set to “No” and fill in the info.

Advantage – 1 license/DEA can service Dev, Test & Production



› While Testing– Limit records so you have less to undo

› In Production – Set Enable Debug to No

Data Exchange




How To – Relationship Mapping

Data Exchange



Data Mapping

Data Mapping

Vendor Relationship Mapping

•Map using External Data Store Mapping.•Trigger on Complete of Data Exchange.

V_Computer

V_DiskDrive

Vendor Relationship Table

BMC_ComputerSystem

BMC_DiskDrive

BMC_HostedSystemComponent


How To – Relationship Mapping

Data Mapping

Class Relationship Mapping

•Map using CI Class Relationship Mapping•Put in an Event Driven Only Exchange*

*In this case, the relationship dataexchange only runs after the data transfer for the two configuration item classes has completed.

V_Computer

V_DiskDrive

BMC_ComputerSystem

BMC_DiskDrive

BMC_HostedSystemComponent

Data Exchange



Data Mapping


How To - Relationship Mapping

› This is where you define which 2 classes to relate and which relationship› Must Select Relationship Source on Create› External Data into CMDB is the only valid Direction› If you choose Relationship Source of “External Data Store”

– must specify DB2, FlatFile, Oracle, SqlServer– Table Name

› It is best to Restrict to DatasetID and specify

Data Exchange





› Define mapping criteria that defines the relationship› Works just like creating Relationship in CMDB

– Source on Left, Target on RightData Exchange





Data Exchange



› Used to define values for a relationship instance› Mandatory attributes shown above must be mapped› To get a list of Attributes of each CI Class choose class from drop down.

Attributes list will then appear



› Used to define values for a relationship instance› Mandatory attributes shown above must be mapped› To get a list of Attributes of each CI Class choose class from drop down.

Attributes list will then appear

Data Exchange





› If you “Update Row to Indicate Delete” i.e. Soft Delete– Set MarkAsDeleted to 1 (Yes)– Other attributes may be set as desired

› If you choose to “Permanently Delete” – Hard Delete– No field to set. AIE will do trigger the deletion



› Use if you are doing class based mapping› Should match query used on class mapping› If you specified “Restrict to Dataset” on the Main Tab, you

don’t need to include it here.

Data Exchange







Data Exchange







Data Exchange





› Used only if a External Data Store mapping› This is used to limit the external records

returnedData Exchange




How To - Relationship Exchange

› Works the same as a normal Data Exchange› Data Exchange Type = Relationship Class

Data Exchange




General AIE Hints

› When testing an exchange –– Test one exchange at a time (when done chain them to be triggered)– Use Run Now button to trigger an exchange immediately– Set Limit = 1 (so that you only run once)– Be sure you know your debug path

› When writing a Vendor Query –– Test it with Direct SQL – If it doesn’t work there, it won’t work here– Use Indexed fields if possible

This won’t work – MacAddress != NULLThis will – MacAddress IS NOT NULL


General AIE Hints

› When setting an AR System field in a mapping –– Doesn’t work like a setfield or pushfield operation– Must Use Rules to convert functions, keywords, etc.

Example of ConcatenationThis won’t work – $Manufacturer0$+”-”+$MediaType0$This will – function|concat($Manufacturer0$,”-”,$MediaType0$)

Example of an Integer FieldThis won’t work – 0This will – Constant|0


Atrium CMDB

› Reconciliation Rules– Identify

• Look for potential duplicates– Compare– Merge– Cleanup / Purge

• Can delete entire data set from all tables• Can delete selected items

– Configuration Item Set field MarkedForDelete = Yes– Create Reconciliation Rule to Purge if field is set

» Define how to handle related items– Run as a Reconciliation Job

CMDB 3rd PartyDataset

CMDB Reconciliation

Engine

CMDB Reconciliation

Rules

CMDB MasterDataset

Documents

Atrium Integration Engine: What Is It? How Do You Use It? · 2020-05-19 · Atrium Integration Engine: What Is It? ... – Ideal for bulk, incremental, data synchronization transfers