Embed Size (px)
Citation preview
W H I T E P A P E R
Implementing or Upgrading SAP® Solutions?Don’t Forget the Data
Addressing the Challenges and Risks of Data Migration
This document contains Confi dential, Proprietary and Trade Secret Information (“Confi dential Information”) of Informatica Corporation and may not be copied, distributed, duplicated, or otherwise reproduced in any manner without the prior written consent of Informatica.
While every attempt has been made to ensure that the information in this document is accurate and complete, some typographical errors or technical inaccuracies may exist. Informatica does not accept responsibility for any kind of loss resulting from the use of information contained in this document. The information contained in this document is subject to change without notice.
The incorporation of the product attributes discussed in these materials into any release or upgrade of any Informatica software product—as well as the timing of any such release or upgrade—is at the sole discretion of Informatica.
Protected by one or more of the following U.S. Patents: 6,032,158; 5,794,246; 6,014,670; 6,339,775; 6,044,374; 6,208,990; 6,208,990; 6,850,947; 6,895,471; or by the following pending U.S. Patents: 09/644,280; 10/966,046; 10/727,700.
This edition published January 2006
1Implementing or Upgrading SAP
White Paper
Table of ContentsExecutive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Data Migration Poses Unique Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Mission-Critical Enterprise Applications Require a Mission-Critical Approach to Data Migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Data Migration Challenges and Your SAP Implementations . . . . . . . . . . . . . 4
Identifying and Analyzing Source Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Accessing Source Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Addressing Data Quality of Legacy Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Preparing and Loading Data into SAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Supporting the Data Migration Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Using a Single, Unifi ed Enterprise Data Integration Platform to Address SAP Data Migration Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Data Profi ling Capabilities for Identifying and Analyzing Source Data. . . . . . . . . . . . . . . 10
Universal Data Access Capabilities for Accessing Source Data . . . . . . . . . . . . . . . . . . . 12
Built-In Data Transformation and Correction Capabilities to Address Data Quality in Legacy Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Certifi ed Connectivity SAP to Prepare and Load Data into SAP . . . . . . . . . . . . . . . . . . . 15
Single, Unifi ed, Metadata-Based Data Integration Platform to Support the Data Migration Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Informatica and SAP: Working Together for Joint Customer Sucess . . . . . .18
“Powered by SAP NetWeaver” Certifi cation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Master Relationship Agreement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Track Record of Joint Customer Success. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Conclusion and Next Steps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
2
Executive SummaryForrester Research recently conducted a survey showing investment in enterprise resource applications (ERP) and enterprise applications in general remains the top IT spending priority for 2005. A major driver for many large companies is regulatory compliance initiatives, such as Sarbanes-Oxley for public companies, Basel II in the banking industry, and FDA Part 11 for biotechnology and pharmaceutical organizations.1
Other business drivers behind the decision to purchase and implement ERP applications like SAP include:
Retirement of legacy systems to support new regulatory requirements such as Sarbanes-Oxley, • Basel II, and the Patriot Act
Standardization of enterprise business applications to support and unify business processes • that are changing and evolving
Need for providing accountability through operational transparency•
Upgrade or consolidation of existing SAP systems due to new functionality or a company merger • or acquisition
Creation of a single view of the customer to cut order costs and increase customer satisfaction•
ERP applications like SAP have an impact on business processes; therefore, the decision to purchase and implement them is approached with considerable due diligence. While the vendor selection and implementation decision gets much of the spotlight, there are critical project phases upon which the success of an SAP implementation hinges. These project phases are not always considered with the same degree of detail as the purchase and implementation decision. Examples of such project phases include:
Business process reengineering.• Software that touches and drives processes across the enterprise cannot simply be installed and turned on. Current “as is” processes must be understood and methodically mapped to the new SAP system and its “to be” capabilities. Invariably, gaps are uncovered during business process reengineering which must be considered and planned for.
Change management and user adoption.• SAP implementations cannot rely on an “if we build it, they will come” approach. The success of a new business application is ultimately measured by its adoption by business users. Careful consideration must be given to executive sponsorship and to business as well as technical user training.
1Hamerman, Paul and R “Ray” Wang. “ERP Applications—The Technology And Industry Battle Heats Up.” Forrester Market Overview, June 9, 2005
White Paper
3
Data Migration Poses Unique ChallengesWhile business processes evolve and are enhanced during these aforementioned project phases, no data is migrated directly from one system to another during these phases. Data migration is different.
Data migration is the only phase during which data is actually moved from legacy applications to SAP. Effective data migration directly affects business user adoption rates. Data migration is therefore a critical component of an SAP implementation.
Mission-Critical Enterprise Applications Require a Mission-Critical Approach to Data MigrationWhile data migration is essential to the success of an SAP implementation, its role in the project often overlooked and underestimated. The common assumption is that tools exist to extract and move the data into SAP, or that data migration is something a consulting partner will handle. Often project teams tasked with data migration focus solely on the timely conversion and movement of data between systems. But data migration is not just about moving the data into SAP; it’s about making the data work once within SAP. This means that the data in the SAP application must be accurate and trustworthy for business users to readily transition from their legacy applications to adopt SAP applications.
Research has shown that software implementations are put at risk when data migration is not thoroughly considered and planned. According to recent research, more than 80 percent of software implementation projects fail or overrun their budgets and schedules. Of the projects that are overrun, half exceed timescales by 75 percent and two-thirds exceed the overall project budgets. A major reason why these failure rates are so high is because data migration is considered a minor, one-time event during the overall implementation.
The successful implementation of mission-critical SAP enterprise applications requires a mission-critical approach to data migration.
This white paper examines fi ve common data migration challenges associated with software implementations. It discusses the value of using a single, unifi ed enterprise data integration platform to address these challenges.
Implementing or Upgrading SAP
Data migration is about more than moving data into SAP; it’s about making data work once within SAP.
4
Data Migration Challenges and Your SAP ImplementationsThis section examines the challenges of data migration in the context of an SAP implementation. Five common challenges are:
1. Identifying and analyzing source data
2. Accessing source data
3. Addressing data quality in legacy applications
4. Preparing and loading data into SAP
5. Supporting the data migration lifecycle
Identifying and Analyzing Source DataMost teams tasked with migrating data into SAP to have experience with the legacy systems. But these teams are often new to SAP and the requirements corresponding to migrating data into SAP. Data migration project teams often may not give suffi cient attention to the identifi cation and analysis of the source data to be required by SAP.
Data migration project teams need to consider and answer fundamental questions about their enterprise data. These questions include:
Does the data exist?•
Where does the data exist? •
What kinds of business rules or context are associated with the data?•
Business logic can be embedded in the data itself. For example, consider the following order number from a legacy application: POUNE55289. Without proper context, it is impossible to tell whether this number is a sales order, purchase order, or purchase requisition. It is impossible to know which system generated the order, or if it is an active or historic order. Such insight can only be achieved by identifying and analyzing the context of the data. As shown in Figure 1, valuable details that are critical to data migration are exposed when the context of the data is analyzed.
Figure 1: True Context of Data Revealed Only with Proper Analysis of Data
White Paper
5
Common Misconceptions About Legacy Data
It is common for teams, especially those that are new to SAP and data migration projects in general, to harbor misconceptions about the source data in the legacy applications. These misconceptions include:
Acquiring legacy data is easy. What about data spread across mainframes, proprietary legacy • applications, other packaged applications? What about data residing with a business partner or remote data center across the fi rewall?
Existing data will fi t the new system. How will XML/hierarchical data structures or non-relational • database (NRDB) data formats be reconciled in SAP?
Existing data is of good quality. Without proper data analysis, how can the data migration team • be sure?
Existing data and business processes are understood. Does the SAP project team include the • business and technical experts who understand the legacy data and business rules? Are these experts even still with the company?
Good, or even adequate, documentation exists. Is the documentation complete? Is it reliable? • Does documentation even exist?
The reality is that source data to be migrated into SAP will typically be spread across numerous existing applications, databases, spreadsheets, or documents. The data may even be “orphaned” or not stored anywhere in the company although this data is still essential to business operations. Sometimes a critical piece of data exists not in any database, application, or system but in someone’s head or notebook. Sourcing legacy data to be migrated into SAP is a complex process that requires careful consideration.
Accessing Source DataAccording to a recent survey of more than 350 fi rms, the typical organization relies on more than 50 core business applications, and companies with more than $1 billion annual revenue have as many as 500 systems. Regardless of whether there are fi ve, 50, or 500 source systems to migrate, the question needs to be answered as to how this will be accomplished.
SAP project teams usually expect that another team will deliver data from required legacy systems. However, simple extraction and upload often proves to be unrealistic due to the volume of source systems and the availability of legacy application resources.
Volume of Source Systems
Moving data into SAP is rarely a simplistic 1:1 mapping between the legacy application and the new SAP solution. Legacy systems migration requires multiple business applications to be migrated into SAP.
Let’s look at an example. Company A runs its fi nancials across fi ve separate fi nancial applications—one per division. Company A is acquired by Company B. All of Company A’s fi nancial departments are to be consolidated into Company B’s standard fi nancial application, which is mySAP ERP Financials. Estimating conservatively, each of Company A’s fi ve divisions have 10 fi nancial data sources (e.g., tables, spreadsheets, fi les, etc.) must be handled to migrate Company A’s chart of accounts into Company B’s mySAP ERP Financials solution.
Implementing or Upgrading SAP
6
This means the migration team is faced with the reality of data extraction across 50 sources. Given the volume of source systems, the ‘Migrating Chart of Accounts’ line item in the project plan does not accurately refl ect the amount of data migration work required. It is not a single data migration unit of work. Moreover, the basic count of sourcing requirements does not take into consideration the joining and reconciling of data between at the source application.
The diversity and volume of source systems can complicate and jeopardize the overall success of the migration.
Dependency and Availability of Legacy Application Resources
The type of legacy applications and the corresponding skill sets and resources required to extract the data are often overlooked during high-level planning of the SAP implementation.
Legacy applications often run in disparate systems on multiple platforms such as:
Mainframe•
Mid-range (AS/400)•
Home grown client/server .Net applications•
Relational databases in conjunction with Excel spreadsheets•
Not only do the number of sources required for extraction need to be identifi ed, but how the data will be extracted and by whom also needs to be addressed. The data migration team must have resources to extract data from the legacy applications, and these resources need to be reliable and trustworthy in providing high quality and timely data extracts.
Tension between the data migration and legacy application teams is common because their goals are often at odds with one another. After all, once the SAP solution goes live, the applications that the legacy teams have built and have supported for years may be deemed obsolete and eventually shut off.
Aside from the politics associated with the access and ownership of legacy data, the legacy resources with the experience and skill sets essential to providing data extracts are likely busy and in-demand resources within their organizations. Between maintaining and enhancing the legacy applications, they may not have the bandwidth to dedicate the proper attention to the SAP data migration effort.
Addressing Data Quality of Legacy ApplicationsThe quality of the data across heterogeneous legacy applications is directly proportional to the effort required to convert, or transform, the data into the format required for entry into SAP.
Data quality can be compromised as a result of how the data has been:
Entered.• Fields are left blank or fi lled in incorrectly. Users may enter cross-sell response codes in a purchase date fi eld because there is no other place for this particular type of data. Customers may mistype their names or addresses when placing an order using a Web site.
Maintained.• Application maintenance, such as enhancements or a version upgrades, may have unpredictable consequences. For example, after an upgrade, a fi eld that was once optional now is mandatory, invalidating previously valid data.
White Paper
7
Processed.• When incorrect data enters the application, it may be propagated across multiple systems. For example, a system of record for material master information feeds data to downstream applications, such as supply chain management or purchasing applications. Data quality issues are replicated and may multiply as data is fed downstream to constituent applications.
Stored.• Storing data across multiple business applications often leads to redundant and inconsistent data. For example, various attributes of customer master entity information are frequently stored in multiple business applications, such as customer relationship management, sales force automation, and sales and marketing applications. Customer information, such as names, titles, addresses, and purchase history, may be stored in different formats or duplicated across different systems, preventing a single view of the customer.
Data migration teams need to understand and accept that there may be “dirty” data. To address data quality issues when migrating legacy applications into SAP, data migration teams should consider the data’s:
Existence. Does the required data for the SAP solution exist? Does it exist within the enterprise, • or possibly in a partner’s or outsourcing vendor’s environment? If it doesn’t exist, what is the business rule to populate the required information in SAP?
Validity. Do data values fall within an acceptable range or domain? For example, if the legacy • applications have 73 U.S. state codes instead of 50, is this valid?
Consistency. Is the same data stored in multiple applications in a common format? For • example, is “John Doe” from Company XYZ the same as “Mr. Jon Doe” from the same company?
Timeliness. Is the data that is required to support the SAP business processes available at the • optimal time?
Accuracy. Does the data correctly describe the properties of the object it is meant to model?•
Relevance. Does the data meet and support the SAP business processes?•
Data migration project teams commonly leverage custom code to support the data conversion process required to address data quality issues. Custom code can initially offer some degree of fl exibility. However, as the number and complexity of integration touch points increase, custom coding limitations in scale and maintenance are exposed.
Preparing and Loading Data into SAPBeyond analyzing, sourcing, and addressing data quality issues in the legacy data, actually understanding and selecting the appropriate techniques to prepare and load data into SAP can be challenging. One of the top reasons organizations choose to implement SAP is that it helps to centralize business processes and data within a consistent application. With the mature and growing list of mySAP business applications, SAP boasts more than 30,000 interrelated tables driving business across more than 25 industry verticals.
While the SAP application platform provides mature interfaces to upload data into SAP, these application program interfaces (APIs) typically require the data in a specifi c format to be properly validated and accepted by the SAP application layer. Data typically is not loaded directly into the database layer of any SAP system, but instead must pass through strict validation checks based on SAP business rules within the application layer.
Figure 2 presents the various interface and loading techniques available for SAP data migration. This table serves as a high-level glossary data migration terms and options.
Implementing or Upgrading SAP
8
SAP INTERFACETECHNIQUE
READ WRITE DESCRIPTION
Data Migration Interface (DMI)
X SAP certifi ed interface tailored for SAP data migration•
Includes SAP delivered programs for the most common • master and transactional data needed in any SAP data migration project
Programs require data in a valid and well-formed fl at • fi le
Programs support a combination of batch and direct • input of data into SAP
Batch Input Processing
X Common method of moving data into SAP•
Programmatically automates and “mimics” processing • data in the same fi elds and screens an online user would step through in a given SAP transaction
Direct Input Processing
X X SAP supported technique of writing data directly to the • database layer of an SAP system
Unlike Batch Input, does not walk through the entire • SAP transaction logic
Contains SAP application validation checks•
Should be considered if throughput from Batch Input • is not suffi cient
Intermediate Document (IDOC)
X X Standard SAP data structures for common business • entities, such as material master or purchase order
Supports integration of both transactional and master • data
Although based on asynchronous data integration, • IDOC, via SAP’s Application Link Enabling (ALE) protocol, can support near real time data movement as well as larger batches of data
Business Application Programming Interface (BAPI)
X X Library of standard SAP interfaces that are Remote • Function Call (RFC) enabled
Able to move data into and out of SAP•
Useful in data migration scenarios to do pre-validation, • or look-ups, of legacy data against the SAP application
Computer Aided Testing Tool (CATT)
X Tool that supports the testing of an SAP business • process
While designed for recording and automating QA test • scenarios, it can be used to load data into an SAP production environment
Legacy System Migration Workbench (LSMW)
X X SAP tool that helps migration teams orchestrate data • migration processes
Able to schedule and run techniques listed above, • such as BAPI, IDOC, or DMI processes to load data into SAP
Supports both Batch Input as well as Direct Input • techniques
Figure 2: Interface Techniques Available for SAP Data Migration
White Paper
9
In most cases, if just a portion of the data being loaded into SAP does not pass the SAP application validation, then SAP will reject the entire record. Examples of data validation performed at the SAP application layer include:
Syntactical.• Is the fi eld length and data type of the material master number valid?
Semantic.• What is the context of the data? Does this number identify a customer or vendor?
Structural.• Does the purchase order header and line item meet proper parent/child relationships or cardinality rules?
Dependency.• Is this bill of material valid even if one of the referenced material master records has not yet been created in SAP?
Supporting the Data Migration LifecycleData migration may be approached as a discrete activity that happens only once during an SAP implementation. However, data migration requirements commonly surface after the initial migration is complete. Subsequent data migration may be required beyond the initial SAP implementation due to such factors as:
Data synchronization.• SAP implementation teams do not always simply “turn off” their legacy systems immediately after data has been migrated to SAP. If the decision is made to maintain both legacy applications and SAP in parallel, data will need to fl ow in a bidirectional manner to synchronize the systems.
Implementation approach.• While a “big bang” or single cutover approach to an SAP implementation is not unheard of, it is considered aggressive. Phased implementations can offer benefi ts including a more manageable and controlled scope to cutover to new business process and applications. Examples of how or when data migrations are implemented in a phased approach include implementations by:
- Region or country
- Legacy system
- mySAP solutions (e.g., FI, SD, MM, etc.)
Treating data migration as a one-time event can be shortsighted and place a burden on subsequent migration phases. Without properly considering the full lifecycle of the SAP implementation, organizations risk duplicating efforts already invested in initial migration phases.
Audits and Data Lineage
The ability to audit and track the end-to-end migration logic, especially across multiple legacy systems, is often overlooked in data migration projects. Whether driven by the need to complete the user acceptance testing phase of a data migration project, or by the need to support audit and compliance requirements, the ability to prove the data was migrated properly can be just as critical as moving the data itself.
Conducting audits and establishing data lineage is particularly challenging when data migration teams use traditional custom coding, which may require tens or even hundreds of individual programs. Whether required to extract, prepare, or load data into SAP, each program contains logic to validate and support business rules about the data. Creating and maintaining these programs is challenging. Auditing and tracking the business logic embedded within and across the programs is even more challenging and is often not fully considered by data migration teams.
Implementing or Upgrading SAP
10
Data auditing has an enormous impact on data migration projects. Consider this example: A developer is responsible for converting material master data. Despite assurance that a conversion program reconciled and consolidated legacy material master data, an audit report shows that the data loaded into SAP still contains duplicate material master data. Identifying problems caused by the legacy extraction or the SAP conversion program requires cumbersome and time-consuming code review sessions and exhaustive testing.
Often migration teams try to address auditing requirements by creating custom reports or spot-checking the data being migrated into SAP. These approaches are incomplete and do not identify the root cause of any issues with the data conversion logic.
Using a Single, Unifi ed Enterprise Data Integration Platform to Address SAP Data Migration ChallengesThis section examines the role of a single, unifi ed enterprise data integration platform in addressing the challenges associated with migrating data into SAP.
The solution to address the challenges of migrating all enterprise data into an SAP solution is Informatica® PowerCenter®. PowerCenter is a single, unifi ed enterprise data integration platform that enables companies and government organizations of all sizes to access, discover, and integrate data from virtually any business system, in any format, and deliver that data throughout the enterprise at any speed.
PowerCenter provides powerful capabilities to help overcome data migration challenges. These capabilities include:
Data profi ling capabilities for identifying and analyzing source data•
Universal data access capabilities for accessing source data•
Built-in transformation and correction capabilities for addressing the quality of data in legacy • applications
Certifi ed connectivity to SAP to prepare and load data into SAP•
Single, unifi ed data integration platform to support the data migration lifecycle•
Data Profi ling Capabilities for Identifying and Analyzing Source DataWhile the objective of moving data from legacy systems to SAP seems straightforward, complications arise when “legacy” migration translates to n number of distinct business applications running on different platforms and data stores, and the context and relationship of the data may not meet or match SAP requirements.
Data profi ling is the analysis of data to understand its content, structure, quality, and dependencies. During SAP implementations, data migration teams typically try to profi le legacy data manually. Manual data profi ling ranges from spot inspections of actual legacy applications or sample data extracts, to analysis via custom-coded reports or elaborate and intertwined spreadsheets. These data profi ling methods typically sample data in a few key fi elds to get a sense of what the data is like in these columns, but the results are often inaccurate and incomplete.
An inadequate toolset and manual approach to profi ling often leads to a data migration project which underestimates the scope, schedule, and resources required to properly analysis source data systems. Figure 3 shows how a much more even distribution of project resources over the key project phases (e.g., analysis, build, and test) can promote savings. Relying on the build or development phase to identify and fi x data issues can increase the cost by ten times.
SAP NetWeaver®, the application platform for all SAP application modules, includes data integration functionality with its established ABAP™/Basis layer, interfaces such as SAP NetWeaver (Exchange Infrastructure) and enterprise service-oriented architecture. PowerCenter complements SAP NetWeaver capabilities with the ability to migrate non-SAP data.
It is important to note that data migration teams do not have to choose between NetWeaver and PowerCenter—it’s not an “either/or” proposition. NetWeaver and PowerCenter’s complementary capabilities help organizations signifi cantly reduce risks and improve productivity during the data migration effort in any SAP implementation.
White Paper
11
PowerCenter’s data profi ling capabilities provide comprehensive, accurate information about the content, quality, and structure of data in virtually any operational system. Organizations can automatically assess the initial and ongoing quality of data regardless of its location or type. With its comprehensive data profi ling capabilities, PowerCenter:
Reduces data quality assessment time with easy-to-use wizards and pre-built metric-driven • reports that comprise a single interface for the entire profi ling process
Addresses ongoing data quality in legacy applications with Web-based dashboards and reports • that illustrate changes in data content, quality, structure, and values over time
Ensures end user data confi dence by automatically and accurately profi ling any data accessible • to PowerCenter—virtually any and all enterprise data formats
Figure 4 shows an example of a PowerCenter data profi ling report. The report shows how PowerCenter automatically infers the primary and foreign key relationships across three tables in a legacy application.
Analysis10%
Build 60%
Test30%
Build
Analysis
Test
Typical Project Effort Ideal Project Effort
Analysis40%
Test 30%
Build30%
Figure 3: Proactive Analysis of Source Data Saves Both Time and Moneyv
Implementing or Upgrading SAP
Figure 4: PowerCenter Profi ling Report Inferring Primary Key and Foreign Key Relationships between Multiple Legacy Application Data Sources
PowerCenter’s data profi ling reports help migration teams determine if the legacy data has quality issues and how to properly address them.
12
PowerCenter’s data profi ling capabilities enable migration teams to do much more thorough analysis than manual profi ling of the legacy systems. The platform provides the tools to automatically scan all records across all columns and tables in a source system and dynamically generate reports that make it easy to understand the true state of the data. These reports help the migration teams help migration teams determine if the legacy data has quality issues and how to properly address them.
Universal Data Access Capabilities for Accessing Source DataAnalysis of legacy data is essential for creating accurate data migration mapping specifi cations with relevant data conversion requirements. However, a complex, ineffi cient migration process still lies ahead if the data migration team relies exclusively on manually extracting data from each legacy data source. According to a report from The Data Warehousing Institute (TDWI), on average, organizations extract data from at least 12 distinct data sources. This average will inexorably increase over time as organizations expand their enterprise application landscape to support more subject areas and groups in the organization2.
Furthermore, while most IT organizations have experience extracting data from relational databases, fl at fi les, and legacy systems, a signifi cant percentage of data required for an SAP data migration project comes from formats not traditionally considered to be a part of a legacy business application.
While many of today’s modern business applications are based on a relational database, many mature and established applications are still maintained on mainframe platforms. A signifi cant percentage of data for an SAP migration will need to be extracted from these systems, but the fact that much of the mainframe data is not stored in a relational format leaves the migration teams relying exclusively on mainframe developers to extract and replicate data.
Based on a 2003 TDWI survey of the types of data sources that ETL programs process, enterprise data may reside in XML fi les, Web-based data sources, payloads from message queues, as well as unstructured data formats such as Microsoft Excel and Adobe .pdf fi les3, as shown in Figure 5. The ability to readily access all enterprise data—structured, unstructured, and semi-structured—is vital to successful data migration.
0 20 40 60 80 100
89%
81%
65%
39%
15%
12%
15%
15%
4%
Relational databases
Flat files
Mainframe/legacy systems
Packaged application
Replication or change data capture utilities
EAI/messaging software
Web
XML
Other
Data Sources
Figure 5: Enterprise Data Resides in a Variety of Sources and Formats
2Eckerson, Wayne and Colin White. “Evaluating ETL and Data Integration Platforms.” TDWI Report Series, 20033Ibid
White Paper
13
PowerCenter provides universal data access, allowing the data migration team to source virtually any and all enterprise data formats, including:
Mainframe data•
Structured data•
Unstructured data (e.g., Microsoft Word documents and Excel spreadsheets, email, binary fi les, • .pdf fi les, etc.)
Semi-structured data (e.g., industry-specifi c formats such as HL7, ACORD, FIXML, SWIFT, etc.)•
Relational data (e.g., DB2, Oracle, Microsoft SQL Server, etc.)•
ERP (e.g., SAP, PeopleSoft, Siebel, etc.) and fi le data•
Message queues (e.g., Tibco, IBM MQ Series, JMS, MS MQ, etc.)•
Figure 6 shows the breadth of PowerCenter’s data access capabilities.
With PowerCenter, SAP data migration teams can source directly from a mainframe application as if it were a relational database. PowerCenter’s data access capabilities offer SAP migration teams the fl exibility to source these “softer” forms of data which traditionally would be left up to manually interpretation and processing—or worse, left unaccounted for in the migration process.
The fl exibility to access all types of enterprise data in a single data integration platform offers signifi cant advantages over hand-coded data migration approaches, including:
Increased productivity.• With the ability to centralize data access and management, PowerCenter frees data migration teams from having to maintain and be dependent on a cumbersome, time-consuming process where programs are developed to extract and stage data for each source of legacy data.
Reduced risk.• Sources of data for SAP implementations tend to be dynamic. Extracting data from a client/server-based legacy application today does not insulate the team from future requirements—for example, having to migrate over mainframe and mid-range applications from applications resulting from a corporate merger or acquisition. PowerCenter reduces the risk of both current and future data migration efforts by providing access to a broad range of enterprise data formats.
Unstructured DataPDF Word ExcelVertical Standards
(e.g., HL7, SWIFT, ACORD)Print Stream BLOBsAny proprietary data
format/standard
Open and RelationalData Sources
Oracle IBM MicrosoftSybase Informatix Teradata
Flat Files XML Web Logs
InformaticaPowerCenter
EnterpriseSoftware Sources
Mainframe AS/400 JDEPeopleSoft Siebel SAP
SAS Essbase Lotus Notes
Across the Firewall/WAN
Remote Data AccessRemote or OutsourcedBusiness Applications
Real-Time Data SourcesTIBCO IBM WebSphere MQJMS SAP MSMQ WEBM
Web Services
Figure 6: PowerCenter Provides Universal Data Access
Implementing or Upgrading SAP
Sources of data for SAP implementations tend to be dynamic. Extracting data from a relational database-based legacy application today does not preclude SAP data migration teams from having to meet future sourcing requirements, such as mainframe or mid-range applications.
14
Built-In Data Transformation and Correction Capabilities to Address Data Quality in Legacy ApplicationsDespite the diverse types of business entities being migrated (e.g., master or transactional, fi nancial, or manufacturing-related), there are common requirements for converting the data from the context of legacy applications to SAP. In SAP implementations, this process is often referred to as data conversions. Examples of data conversion tasks include:
Filtering data for subset processing (e.g., only move European customer data, or material • masters for specifi c company codes)
Doing lookups to translate values (‘$’ to ‘USD’)•
Aggregating data for sum and averages•
Routing data for branching and case logic•
Calling custom functions or stored procedures•
Joining disparate tables and fi les•
Name and address standardization•
While a custom coding approach to data migration initially offers some fl exibility, this approach has its limitations. For example, if 100 programs are required to convert data across 10 legacy data sources (a conservative number of sources), custom coding becomes complicated, ineffi cient, and a challenge to scale. Developers working on different platforms and development tools may add to the complexity, and sharing and reusing the coding effort across the migration team, even with the best intentions, is unrealistic.
PowerCenter helps SAP data migration teams by enabling the team to focus on the data and not code. PowerCenter was originally developed to address data transformation and conversion requirements associated with data warehousing. That core capability has evolved into a single, unifi ed, scalable enterprise data integration platform with a robust library of transformation and data services capable of handling all data conversion on any SAP data migration project. By leveraging PowerCenter’s codeless and wizard-driven approach for SAP data conversion, teams can focus more on the business rules and data, and less on the code.
White Paper
15Implementing or Upgrading SAP
Certifi ed Connectivity SAP to Prepare and Load Data into SAPSAP data migration teams can realize signifi cant benefi ts from using PowerCenter to prepare legacy data to be loaded into SAP applications vs. the combination of custom code.
Native Connectivity to SAP NetWeaver
PowerCenter offers tight integration with SAP at both the data and metadata level. PowerCenter provides broad, metadata-driven, high performance data interchange capabilities for SAP NetWeaver. More than 600 SAP customers have leveraged this native extension to the PowerCenter data integration platform to enable bi-directional data integration with the SAP NetWeaver platform.
PowerCenter’s SAP connectivity features include:
Import of metadata for any SAP data dictionary object, including transparent and clustered • tables, as well as IDOC, BAPI and DMI object types
Bi-directional data integration support for BAPI interface•
Bi-directional data integration support for ALE/IDOC interface•
Flexibility to source or target IDOC data in real-time or in batch•
Ability to prepare legacy data into well formed and valid DMI fi les, ready for upload with the • SAP NetWeaver LSMW utility
Figure 7 shows a simple PowerCenter data migration mapping in which the customer master data from multiple mainframe sources are being sourced and prepared in a valid SAP DMI format.
Note how a single PowerCenter transformation within the mapping replaces the traditionally coding-intensive effort for preparing a well-formed and valid fi le ready for loading into SAP. All of the detail shown in the DMI customer master object is entirely imported directly from the SAP
Figure 7: PowerCenter Preparing SAP DMI File for Data Migration
16
application layer.
The ability to access and manage metadata makes PowerCenter unique. When using PowerCenter for SAP data migration, the SAP metadata is automatically interpreted within the transformation to validate all legacy data. This means that as the legacy data fl ows through the DMI transformation, it is validated against rules inherent within the SAP metadata such as:
Mandatory vs. option segments•
Cardinality•
Minimum and maximum occurrence of segments•
Field level data type and precision requirements•
PowerCenter Complements SAP NetWeaver
As both PowerCenter and SAP NetWeaver offer data integration capabilities, a logical question SAP customers and implementation partners may have is which product to use and when? A good rule of thumb is this:
Leverage SAP NetWeaver XI for business process integration•
Leverage Informatica PowerCenter for data migration•
According to a 2005 TDWI report, broadly speaking, enterprise business integration can occur at four different levels in an IT system: data, application, business process, and user interaction.4 Figure 8 shows how the four layers of an integrated enterprise are jointly supported by the
combination of PowerCenter and NetWeaver.
PowerCenter can access all non-SAP types of enterprise and legacy application data. It also provides the application integration to prepare and move all non-SAP data in a bi-directional manner with SAP.
SAP NetWeaver enables transactional based data integration requirements with the capabilities offered with SAP NetWeaver XI. SAP NetWeaver XI also orchestrates all data integration related to business process integration requirements, including SAP-to-SAP data integration (e.g., mySAP ERP to my SAP APO). Finally, from an end user perspective, SAP NetWeaver Portal provides a
Informatica
Powe
rExc
hang
e
PowerCenter
Source data profiling
Data cleansing
Mappings
Complex transformations
UserInteraction
Business Process
Application
Data
Applicationintegration
Busineesprocess
management
+
Data integration
Businessportal
Collaboration+
SAP Netweaver
METADATA SERVICES
Com
posi
t App
licat
ion
Fram
ewor
k
Life
cycl
e M
anag
emen
t
Multi-channel access
Portal Collaboration
INFORMATION INTEGRATION
Master data mgmt.
Knowledge mgmt.
PROCESS INTEGRATION
APPLICATION PLATFORM
Bus. intelligence
Integrationbroker
Business Process mgmt.
DB and OS abstraction
ABAPJ2EE
Figure 8: Complementary Data Integration between SAP NetWeaver and Informatica PowerCenter
4White, Colin. “Data Integration: using ETL, EAI, and EII Tools to Create an Integrated Enterprise” TDWI Report Series, November 2005
White Paper
17Implementing or Upgrading SAP
proven front-end platform for business intelligence and visibility across the enterprise.
Single, Unifi ed, Metadata-Based Data Integration Platform to Support the Data Migration LifecycleWhen data migrations projects are driven by teams that are focused exclusively on the target system, not in the end-to-end data migration process, a common outcome is the “code, load, and explode” phenomenon5. This occurs when developers code the extraction and conversion logic thought to be required for migration, then attempt to load it to the target business application, only to discover an unacceptably large number of errors due to unanticipated values in the source data fi les. They fi x the errors and rerun the conversion process, only to fi nd more errors, and so on. This ugly scenario repeats itself until the project deadlines and budgets become imperiled and angry business sponsors halt the project.
PowerCenter breaks this “code, load, and explode” cycle. PowerCenter provides all the capabilities that are essential for the lifecycle of SAP implementations from a single, unifi ed platform based on a metadata-driven architecture.
The foundation for all of PowerCenter’s data integration components is the shared metadata. When changes are made anywhere in the profi ling, data access, data conversion, or SAP loading process, PowerCenter enables immediate visibility into those changes. With its metadata-driven architecture, PowerCenter promotes faster and more fl exible iterations in the data migration lifecycle.
Figure 9 shows how PowerCenter is used for migrating data.
PowerCenter’s metadata management capabilities provide visibility across the entire data migration process—from sourcing legacy applications and cleansing the legacy data, to preparing it in the format required for upload by SAP. PowerCenter enables data lineage problems to be
Iterate
Access source systems/data
Reusability/Team Productivity
Synchronize
Audit/Lineage
Access target/data1
2 3
5
4
7
9
6
10
8
Informatica Data Integration Platform
Execute Migration
Analyze/ Profile
Extract/ Transform
Validate/ Lead
Target Application
XML, Messaging, and Web Services
Packaged Applications
Relational and Flat Files
Mainframe and Midrange
Figure 9: PowerCenter Is the Ideal Platform for Migrating Data
5Eckerson, Wayne and Colin White. “Evaluating ETL and Data Integration Platforms.” TDWI Report Series, 2003
18
traced at a metadata level.
Figure 10 shows a PowerCenter data lineage diagram spanning multiple data migration mappings, as well as each component responsible for sourcing, converting or targeting the required data for SAP. This diagram shows the fl ow of migration logic. Once the suspect area of data migration
logic has been identifi ed, users can drill into the object and make the appropriate changes to the relevant data mapping object.
PowerCenter helps data migration teams trace and prove how data has been converted and moved. The enhanced data visibility and tracking helps organizations comply with reporting requirements. These capabilities also help with user adoption, instilling new SAP application users with confi dence that legacy application data has in fact been converted and moved.
Furthermore, PowerCenter alleviates the politics associated with data migration projects. Data migration activities, whether related to legacy applications or the target SAP application, can be centralized within a single, unifi ed data integration platform. This promotes effective and productive communication between legacy and SAP resources, and between technical and functional resources.
Informatica and SAP: Working Together for Joint Customer SucessInformatica and SAP are in partnership to ensure organizations successfully implement SAP applications. Evidence of this strong partnership is demonstrated through:
PowerCenter’s “Powered by SAP NetWeaver” certifi cation•
Informatica and SAP’s Master Relationship Agreement•
A long track record of proven joint customer success•
“Powered by SAP NetWeaver” Certifi cationInformatica is a preferred vendor in SAP’s partner ecosystem and has achieved a level of certifi cation unequalled by any other data integration platform provider. In addition to developing a growing library of SAP certifi ed interfaces, PowerCenter is a certifi ed member of the “Powered by SAP NetWeaver” program, which is a certifi cation level above typical software vendor certifi cations.
Figure 10: PowerCenter Data Lineage Diagram enables tracking and auditing of end-to-end migration from legacy applications to SAP
PowerCenter has been awarded “Powered By SAP NetWeaver” status by porting the PowerCenter platform and PowerCenter Web Service Hub to the SAP J2EE platform. SAP users can access PowerCenter’s Web Services capabilities directly through SAP NetWeaver Portal’s front-end.
The global Master Relationship Agreement between Informatica and SAP underscores and validates how PowerCenter offers a certifi ed and proven solution for SAP data migration projects.
White Paper
19Implementing or Upgrading SAP
“Powered by SAP NetWeaver” is a program where partners develop solutions directly on the NetWeaver application platform.
Master Relationship AgreementOn October 20, 2005, Informatica signed a global Master Relationship Agreement with SAP AG and a Go-to-Market Agreement with SAP America focused on accelerating the time-to-value of SAP data migrations and other strategic data integration initiatives for SAP customers. This three-year agreement provides a framework for the execution of local go-to-market agreements piloted by SAP. Informatica is working with SAP customers to mitigate the risks and increase the effectiveness of data migration, system consolidation, and other strategic data integration-driven projects.
Track Record of Joint Customer SuccessPerhaps the strongest indicator of partnership that works to the benefi t of customers is a robust track record of joint customer success. Informatica and SAP have more than 600 joint customers. These customers have relied on PowerCenter to consolidate instances of SAP, to migrate data into SAP, and to successfully implement SAP applications in their environments. Examples of joint SAP and Informatica data migration customer success include:
Major Utilities/Energy CompanyChallenge
Stovepipe application architecture across 15 legacy systems due to internal growth and growth • via M&A
Need to migrate 15,000 human resource records and 2.5 million customer data records into • SAP
Data quality across heterogeneous legacy applications not meeting SAP standards and • requirements
SolutionLeverage PowerCenter’s rich legacy and SAP integration •
Single, unifi ed platform to meet end-to-end data migration requirements•
Reuse migration components through multiple phases in project•
ResultsDelivered project on time and on budget•
Reused more than 50 percent of PowerCenter components for data migration•
Military Defense OrganizationChallenge
Multitude of mainframe legacy applications managing logistics•
Available migration project team inexperienced in SAP and Informatica•
Unsure of how to identify what legacy data required for SAP•
SolutionStandardize on PowerCenter for data migration, including extraction, conversion, and • preparation of data for SAP
Leverage PowerCenter’s built-in data cleansing capabilities to improve data quality and • validation
Results
20
Compressed SAP implementation project’s overall timeline by 66 percent•
Delivered expanded scope of original implementation with 80 percent less effort•
Large Reinsurance Enterprise:Challenge
Need to meet fi nance-related reporting requirements to ensure compliance•
Need to reconcile fi nancial business processes across disparate legacy systems (including • multiple SAP instances), leading to close periods of more than 30 days
SolutionCentralize sourcing of legacy and existing SAP systems within PowerCenter •
Consolidate application instances on PowerCenter•
ResultsReduced fi nancial close cycle from more than 30 days to fewer than fi ve days •
Simplifi ed fi nancial master data–75 percent fewer master journal entries in consolidated SAP • instance
Reduced chart of accounts from 34,000 to less than 2,000 entries•
Conclusion and Next StepsData migration is critical to the success of SAP implementations. But data migration should not be approached as singular event. There are data migration challenges to be met throughout an SAP implementation. The fi ve main data migration challenges are:
1. Identifying and analyzing source data
2. Accessing source data
3. Addressing the quality of the data within the legacy applications
4. Preparing and loading data into SAP
5. Supporting the data migration lifecycle
The best way to overcome these challenges is to rely on Information PowerCenter, a single, unifi ed enterprise data integration platform based on a metadata-driven architecture. PowerCenter offers data migration teams powerful capabilities to meet each of the fi ve data migration challenges. The capabilities include:
Data profi ling•
Universal data access•
Built-in data transformation and correction•
Certifi ed connectivity to SAP•
White Paper
21Implementing or Upgrading SAP
Furthermore, PowerCenter allows data migration teams to leverage all these capabilities from a single, unifi ed data integration platform. This increases productivity, ensures scalability, and reduces risk.
Now that you have a solid understanding of the challenges around data migration and how PowerCenter can help you overcome them, what is your next step? Informatica has developed an offering to show you how you can make your next data migration project a success. This offering is called the Data Migration Readiness Assessment.
The Data Migration Readiness Assessment demonstrates the value of leveraging Informatica for SAP data migrations. It also serves to jump start any SAP data migration project.
The Data Migration Readiness Assessment is designed to help any SAP customer understand the challenges and risk in a data migration project:
1. Identify data risks early
2. Scope and plan migrations effectively
3. Deliver SAP implementation on time, on budget, and in scope
Figure 11 shows how the Data Migration Readiness Assessment works. 1
4
SAPSAPStage
1
4
LegacyStage
Figure 11: The Data Migration Readiness Assessment Jump Starts Data Migration Projects
Worldwide Headquarters, 100 Cardinal Way, Redwood City, CA 94063, USAphone: 650.385.5000 fax: 650.385.5500 toll-free in the US: 1.800.653.3871 www.informatica.com
Informatica Offi ces Around The Globe: Australia • Belgium • Canada • China • France • Germany • Japan • Korea • the Netherlands • Singapore • Switzerland • United Kingdom • USA
© 2008 Informatica Corporation. All rights reserved. Printed in the U.S.A. Informatica, the Informatica logo, and PowerCenter are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.
6665 (09/17/2008)
