Basics DataMigration[1]

8/10/2019 Basics DataMigration[1]

http://slidepdf.com/reader/full/basics-datamigration1 1/14

Basics of Data Migration

Using IS Migration Workbench

-Gagan Agrawal



Basics of Data Migration Page: 1/13

1 INTRODUCTION

1.1 Purpose o f the Document

The purpose of this document is to explain

Basics of Data Migration

What Is Data Migration Steps Required For Data Migration Data Migration Process Flow IS-U/CCS Data Migration Scenario

What Tools/Technologies Is Being Used And Why

IS Migration Workbench Concepts

The IS-U Migration Workbench Principle Migration Object Definition

Automation Data Definition Field Rule Definition Fixed Value Definition Conversion Object Definition Other Important Terms Special Technologies List of Standard Migration objects Hierarchy display of Migration Objects

2 BASICS OF DATA MIGRATION

2.1 What is Data MigrationWhen implementing a SAP system in any organization the data stored in existing ITapplications has to be transferred to the SAP database. Normally it will require manual entryof data through keyboard using user transactions. If the amount of data is too large thisprocess becomes difficult, costly, and time consuming.

Data Migration is the transfer of data from Non-SAP systems (Legacy applications) to SAPsystem through the use of data transfer technologies and tools provided by SAP.

Relevance of Data Migration : Accounts for 15% - 20% of the total SAP implementation costs Smaller implementation projects: up to 40%

Why is data migration so expensive?In many cases

you need to develop conversion programs (which can not be used later on) you need to deal with lots of technical details and different technologies data volume depends on the project volume the effort for programming does not depend on the data volume




2.2 Steps Required For Data Migrat ion

To successfully carry out any data migration project following steps should be followed:

Determine which business objects are to be transferred. Unload data from legacy system into files. Determining required Fields of source and target systems.

Mapping source and target fields. Analyzing and cleaning legacy data. Determining what data transfer method/tools should be used. Configuring Data Transfer Tools – Creating Projects, Setting field rules and

generating data transfer programs. Testing – Testing data transfer with test data. Convert extracted and cleaned data into the structures of the transfer file (file in SAP

format). Final Data Migration – Load converted data into SAP system. Checking the results with migration statistics

Determin ing Business Objects To Be TransferredThe objects to be transferred depend on the business application that you are using.Determine which business objects needs data to be transferred and which don’t.

Unloading Data from Legacy systemData stored in the database of legacy system needs to be unloaded into sequential file or spreadsheet. If the legacy system is R/2 then it can be done by SAP tools, an otherwise,external program needs to be written to perform this task.

Determin ing Required Fields of Source and Target systemsYou need to determine what data is needed by finally configured SAP systems i.e. prepare alist of fields that must be filled by legacy data. Also, make a list of data fields provided bylegacy system.

Mapping Source and Target FieldsThe one to one mapping between source field and target field needs to be done i.e. fromwhere data is coming and where it is going. Also, it should be determined that whatmodification or conversion needs to be done while processing that field. Determine how thedata is structured (length and sequence). Data type of source field and target field should begiven a special care.

Analyzing and Cleaning Legacy DataData in the legacy system must be cleaned to get rid of incorrect or incomplete data.Determine which data can be transferred unmodified, which must be converted and whichcannot be transferred at all. Perform all necessary corrections on incorrect data.

Determining What Data Transfer Method/Tools Shou ld Be Used.SAP provides a large number of tools and techniques for migrating data from legacy systemto SAP system. Each of these has its own benefits and limitations. You must determinewhich tool / technique best suits your migration needs.




You must take following facts into consideration: How much time you have. Time taken for developing data transfer programs. Time taken to transfer data (Speed of Data Transfer). Level of Data Integrity Offered. Error handling/ Analysis tools offered.

Periodic or One time Data Transfer. Internal or External Data Transfer. Foreground or Background Data Migration

SAP provides following techniques for Data Migration:

BDC (Batch Input Session and Call Transaction) SAP supplied Batch Input and Direct Input programs BAPI/RFC ALE/IDoc

SAP provides following tools for Data Migration:

Data Transfer Workbench LSMW (Legacy System Migration Workbench) CATT (Computer Aided Test Tool) IS-U Migration Workbench

Configuring Data Transfer ToolsIf using any Data Transfer Tool, you must configure it to carry out data migration according toyour requirement. It involves creating data transfer projects, setting various attributes, settingfield rules (rules that determine how data is processed during migration), and generatingdata transfer programs.

Testing – Testing Data Transfer With Test Data

After configuration of data transfer tools and generation of data transfer program all settingsand developments must be tested with test data. For this, a small test file should be createdwith test data and data transfer testing should be done using this test file. Error logs if anyshould be analyzed and necessary corrections should be done. Almost all of data transfer tools provides the option of creating test data file and error log generation.

Convert Data File to SAP FormatSAP requires data to be arranged according to its own format. Extracted and cleaned datamust be converted to the format required by SAP. Some Data Migration Tools provides thisfacility. If this facility is not provided by data migration tool you are using, customer programsneeds to be written to carry out this conversion.

Final Data Migration

After all configuration, development and testing has been done and input file has beenprepared according to format required by SAP, final Data Migration can be carried out usingthis file. After this data migration, all data from the input file will be transferred to SAPsystem’s database.

Checking the ResultsIf in any case error has occurred, analyzing error logs and migration statistics can check it.Necessary action must be taken to migrate the data correctly.




2.3 Data Migrat ion Process Flow

Data to be migrated is stored in database situated either on PC or on Legacy Application Server.

The first step is to extract this data from legacy application’s database and put it in

either a tab delimited text file or any spreadsheet. This data is then cleaned toremove any existing inconsistencies.

The data stored in these files are in legacy format. This data must be arrangedaccording to SAP format. So, next step will be to convert this data to SAP format.

After the data has been converted to SAP format, using any one of SAP providedtechnologies i.e. BDC, Direct Input or BAPI/IDoc does actual data migration. This canbe done by using any one of SAP provided data transfer tools or by developingcustom data migration programs, which use above mentioned technologies.

The data is now migrated to SAP system.

2.4 IS-U/CCS Data Migration Scenario

During the implementation of IS-U-CCS component of SAP, an unusually large data volumehas to be extracted from one or more legacy systems, and moved to the database of the R/3system. Since the production system cannot tolerate an extended downtime (alsounacceptable for customer care and contract accounting reasons), this data transfer issubject to high performance demands. It is therefore necessary to optimize all the involvedresources.

One or severalfiles

DATA TRANSFER TOOLSConvert data

Batch Inputprocessing (BDC)

Legacy data

on PCRead data

SAP format

Legacy

format

Legacy data onapplication

BAPI/IDoc

Direct Input

processing

How Data Migration works

Structurerelations

Field mapping

Conversion

rules

SAPSystem




In case of data migration for IS-U-CCS there are a large number of issues which must beaddressed for successful migration of data to SAP system. They are listed below:

Data to be transferred is usually very large in volume. So, special care should begiven to the speed of data transfer.

Various business objects of IS-U-CCS are interlinked and dependent to each other.So, a referential integrity check must be implemented while transferring data of thesebusiness objects.

Since IS-U-CCS is a billing and customer care system, it cannot tolerate extendeddowntime. It requires data to be migrated in the fastest possible way.

Migration of data for several business objects is time-dependent. For example for MOVE-IN process. So, there should be a mechanism which can handle this timedependency during data migration.

The data of IS-U-CCS system is of very high magnitude of complexity. So, amechanism is needed which can effectively process data on field by field basis.

In case of errors, a restart mechanism is needed so that data transfer can becontinued from where the error occurred.

Some business objects of IS-U-CCS needs to be transferred in a single run to cover interdependency. This facility must be provided by data transfer tool/technologybeing used.

2.5 What Tools/Technologies Is Being Used and Why

For data migration of HT and LT consumers we are using IS-U Migration Workbench whichtransfers data from legacy system to SAP system through use of Direct Input method.

Direct Input is the fastest available technique for transferring data. It doesn’t processscreens for transferring data but perform all necessary checks through function modules.

IS-U Migration Workbench is a special tool provided by SAP to handle complexityinvolved in migration of IS-U data.

SAP IS-U Migration Workbench – General Facts

SAP IS-U Migration Workbench A tool that complements the known migration techniques Used for migrating SAP IS-U/CCS and SAP IS-Waste business objects Comprehensive cover of all data transfer actions

Designed for data migration from R2-RIVA system with SAP unload programs Any source system with own unload program

High Performance Migration Import programs based on fast Direct Input technique Parallel import possible Interface structure width can be reduced




SAP IS-U Migration Workbench - Technical Facts

Open Interface Structures Independent data model from the source system Transfer in object-oriented form (business objects)

Service function modules save data Optimum consistency check (based on dialog check) High throughput using Direct input Technology

Possibility of adjusting migration structures Reducing data flow from source system

Migration control due to migration customizing Easy to create and change conversion rules for each field Offers high degree of flexibility for each single field mapping

Key and Status Management in SAP R/3 Easy to reference data objects using their legacy system key

Test run feature Import without database updates

Migration statistics continually updated Overview of processed file and current status of import runs Counter for successfully imported objects and errors Performance overview

Error Log Collects all reported error messages Identifies corrupt data records

Error Files Reusable Import directly once error is solved

SAP IS-U Migration Workbench – Overall Funct ional Scope

Migration of All master data

All billing relevant master and transaction data needed for the next billing execution Device Installations Meter Readings Billing Parameters

Consumption history Billing Period

All necessary transactional data Open items Dunning Items Budget Billing Plans Instalment plans Deposit




SAP IS-U Migration Workbench – How it solve our problems

It solves our migration speed problem by using Direct Input method – the fastest datatransfer method.

For our problem concerning interlinking of various business objects and referentialintegrity check it provides a mechanism called Key and Status Management whichinterlinks inter-dependent business objects with a key.

It perfectly handles date dependency of various transaction data by keeping proper context of historical data. For example it handles the MOVE-IN process as describedbelow

It provides a facility to set processing rules for individual fields which handles data atfield level.

It provides a restart mechanism which enables us to continue data migration from thepoint at which error occurred.

It provides a special migration object called Hyper-Object which can incorporate all inter-

dependent migration objects and enable us to perform data migration of all these objects in asingle go.




3 THE IS-U MIGRATION WORKBENCH CONCEPTS

3.1 The IS-U Migration Workbench Principle

IS-U migration is an open interface for transferring master data and transaction data to theR/3 application IS-U. The outflow is orientated around the target system R/3-IS-U. Thismeans that the data model or the function of the legacy system for migration is neither

required nor used.The migration procedure cannot run based on tables, because of the relational IS-U datamodel and the considerable complexities it contains. Instead it has to compile related data insuch a way as to guarantee consistence in the data after the transfer.The data is compiled into so-called migration objects for the migration into the R/3application. In these units the data is transferred into R/3 System. At the same time, specialservice function modules have been created in IS-U for the business objects of thisapplication. They make an object orientated transfer possible. These modules operate indirect input mode and can thus avoid the performance disadvantages of batch inputprocesses.The structure of a migration object is defined in controlling tables delivered by SAP. It canconsist of several R/3 structures. You can find all the structures connected with a migration

object in the dictionary. User-defined enhancements (fields) are taken into consideration atthe same time.Data can be extracted from the legacy system using user-defined extraction programs. Theexact record structure is determined per table settings in the IS-U migration workbench(transaction EMIGALL). Data sets that have been generated in this way, and transferred toyour target computer by file transfer, are read and processed in the R/3 System by the importprogram, which is generated by the customer system. The code conversion (EBCDID to

ASCII) is carried out at the same time. Following that, the data is formatted and transferredto the direct input module.To view the complete documentation of IS-U Migration Workbench use transaction EQ81

3.2 Migrat ion Object Def in it ion

A migration object is a logical business unit containing the data that is transferred to the R/3

System during IS-U migration

A migration object is described by one or more structures. As a whole they are called Autodata. The service function modules being used predetermine them; in the R/3 environmentthey are defined via the dictionary. For example, PARTNER object

3.3 Automat ion Data Def in it ion

Automation data specifies the quantity of all fields of a migration object, as well as itsstructural order.

The automation data is clearly predetermined by the interface of the function module beingused.

3.4 Field Rule Def in it ion

In IS-U Migration, f ield rule denotes the unique rule basis, with which an individualautomation data field is processed during import.Examples of field rules are:

Initial value (allocating the typical initial value) Supplying a constant value (Fixed value) Allocating a value transferred from the legacy system




Executing a series of ABAP statements (complex field rules) Mapping through a conversion table

3.5 Fixed Value Def ini tion

In IS-U migration, a fixed value is a definition of a generally used assignment of a constantvalue or table content.

Examples of fixed values are:

The constant for the field company code (for example '0001') The constant for the application of contract accounts receivable and payable The current date (field content sy-date) The logon language (field content sy-langu)

A fixed value can be used in several Field Rules and is suitable for the uniform supply of fields with the same meaning.

3.6 Conversion Object Def in it ion

In IS-U migration, a conversion object refers to the definition of a generally applicablemodelling rule for a set of discrete input values to output values. The data category and fieldlength of the input field can be set independently of the target field.Conversion objects can be used for:

Mapping the industry key in the legacy system to the industry in IS-U

Converting the company code from RIVA (e.g. from "01" to "0010")

Mapping the values from a check table in the legacy system to the values in anIS-U check table

A conversion should be used when no clear fixed value can be set, and when creating an

ABAP rule as a CASE distributor would take too long. A conversion object can be used inseveral field rules, and is therefore suitable for supplying fields with the same meaning.

3.7 Other Important Terms

CompanyMigration customizing is a client independent application. The Company represents theproject to which migration objects are allocated.

In order to work in various clients with their own migration configurations, differentcompanies have to be created. The required migration objects can then be copied for eachcompany.

The company SAP serves as a reference configuration. SAP delivers the individual migrationobjects; they are overwritten when an IS-U upgrade takes place.

Migration classThe migration class defines the business object. A migration object is based on the businessobject.In this way you can, for example, give the SAP migration object ACCOUNT its own namewhen you copy it. Now, when you transport settings, you can still recognize that theoperation involves the migration object ACCOUNT.






List of Standard Migration objects(as of SAP ERP 2005 with IS-Utlit ies 600)

Migration Object Description

ACCOUNT Contract Account

ACCOUNTCHA Contract Account Change

ACC_NOTE Adding notes to contract account

ADRCITY Create City

ADRCITYISU Add IS-U data to City

ADRPSTCODE Create Postal Code

ADRSTREET Create Street

ADRSTRTISU Add IS-U data to Stree

AREA Area of a cleaning object - Waste Management

BANK Bank Master

BBP_CB_TAB Collective document numbers for budget billing requests

BBP_EXT Transfer Payment Plans

BBP_INT Create budget billing plans according to IS-U schedul ling

BBP_MULT Create budget billing plans according to external schedulling

BBP_PAYSC Budget Billing Payment Scheme

BCONTACT Transfer data to Customer Contacts

BCONT_NOTE Adding notes to Customer Contacts

BILLDOC Transfer Billing documents with all billing line items

BILLDOCEBF Transfer EBF(easy bill correction framework) Billing documents

BILLTRIGCRCreate billing orders for billing which has already taken place in legacy system(in case SAP billing schemas are very close to schemas in legacy system

BILLTRIGG Execute billing for billing orders

CLEANOBJ Cleaning Object - Waste Management

CONNCTINST Installing Connection(equipment) in a Connection Object(Functional location)

CONNECTION Creating a Connection(Equipment)

CONNOBJ Create Connection ObjectCONSHIST Create partial billing documents to show consumption history

CONSUMPT Transfer data of Period Consumption in Installation structure

CONTAINER Create Container(technical object equipment) - Waste Management

CONTGRP Create Container Groups along with Containers

DEVGRP Create Device Groups for devices

DEVICE Create Device (equipment)

DEVICEMOD Modications of both Installed & Non-Installed devices

DEVICERATE Carry out Rate changes in the installation structure

DEVICEREL Carry out Device Allocations

DEVINFOREC Create Device Info Record

DEVLOC Create Device LocationDISC_CLOSE Closing Disconnection documents

DISC_DOC Create Disconnection document

DISC_ENTER Enter Disconnection entries

DISC_EVENT Create Disconnection workflows

DISC_ORDER Create Disconnection order

DISC_RCENT Create Reconnection entries

DISC_RCORD Creating Reconnection order




DISC_WF Creating Disconnection workflows

DOCUMENT Transfer open items - receivables/credit memo

DOC_DOUBT Transfer Doubtful entries - Write Off for open items already migrated

DOC_INT Transfer date of last interest calculation for open items already migrated

DUNNING Transfer dunning data & creditworthiness of business partners

FACTS Transfer facts for installationFUNCLOC Create Functional Location in PM

FUNCLOCCHA Change data for Functional Locations CONNOBJ, DEVLOC, FUNCLOC

GOODSMVT Create material document for goods movement

HYPER Template to create new hyper objects

INSTLN Create Installation

INSTLNCHA Change Installation data

INSTPLAN Create Installment Plans for migrated open items

INST_MGMT Device Instal lation/Removal/Replacement

INTCASHDEP Transfer Interest documents for payment of Cash Security deposits

LOADPROF Create load profiles and allocating to Installation

LOT Create Data object sample lot for devices

LOTFINAL Transfer Completion data for creating sample lots

METERREAD Transfer Meter Readings

MOVE_IN Create Move-In

MOVE_IN_H Move-In and Move-out of historical contracts of legacy

MOVE_OUT Create Move-Out

NOTE_CON Transfer Static note to field service in Connection object(For meter reader)

NOTE_DLC Transfer Static note to field service in device location (For meter reader)

OBJCLASS Creating classification of technical objects

OBJSTATUS Changing user status of technical objects and profiles

OFFICEDOC Create SAPOffice documents linked to objects

PARTNER Create Business Partner

PARTNERCHA Changes to Business Partner

PART_REL Create Business Partner relationships

PAYMENT Transfer payment documents for open items

POD Create Point of Delivery for Installation or Device

PODCHANGE Changes to Point of Delivery

PODSERVICE Creating object Point of Delivery Service

PREMISE Create Premise

PREMISEWA Extend IS-U-WA specif ic data for device

PROFASSIGN Allocate profiles to registers

PROFHEAD Create Profile header data

PROPERTY

Property allocation like connection object, premise, installation to Business

Partner REFVALUES Transferring information on reference values (assigned to installation)

REGPOLIT Transfer political regional structure data

REGRELSHIP Transfer register relationships

REQUEST Create requests in FICA

ROUTE Create route master data object

SALES_STAT Transfer data on Sales statistics

SECURITY Create security deposit request




SERVFREQ Create Service frequency

SM_NOTIF Create service notification

SM_NOTIFOK Change status of service notification to "done"

STRT_ROUTE Create street routes

TECHINST Create technical installation

TECHOBINSP Allocate last inspection date to device or technical installation

Hierarchy display of Migration Objects

The following diagram represents the individual migration objects with their dependencies.Read from above to below, the diagram displays the chronological order of migration.

Note: The migration objects to be used and their corresponding hierarchy differs from oneproject to another project depending upon the business scenarios involved.

| (1) ADRPSTCODE REGPOLIT | (2) BANK

| | | |----------- | | | ADRCITY-------- | | | | | ADRCITYISU | | | ADRSTREET------- | | | ADRSTRTISU | | -----------------------------------------------------------------------

DEVICE CONNOBJ ------- PARTNER -------- | | | | | | | NOTE_CON | PART_REL | | | | -------- ------ ---------------- -- | | | | | | (5)

| | | | | BCONTACT | PREMISE --| | ACCOUNT-- | | | | | | | | | BCONT_NOTE | | | | | | | DEVGRP | (3)--- DEVLOC -- | | ---- | | | | | | | | -------- ------ NOTE_DLC | ACC_NOTE | | | | | | | | | | INSTLN -------------- | -----------(7) | | | | | | | -------- --------- FACTS | | | PROPERTY | | REFVALUES | | | ---- INST_MGMT | | | | | (INSTALL) | LOADPROF | POD | | | | | METERREAD ----------- ---------------- | CONSUMPT (6) |

| STRT_ROUTE MOVE_IN PODSERVICE REGRELSHIP | | ------------------------------------------- (4) | | | | BBP* DOCUMENT BILLDOC SECURITY | | | - - - - - ------------- PAYMENT | | | PAYMENT INSTPLAN INTCASHDEP

Documents

Basics DataMigration[1]