Session ID: SPC202Conversion of SAPSystems to Unicode
Michaela See, SAP AG
Dr. Christian Hansen, SAP AG
SAP AG 2005, SAP TechEd ’05 / SPC202 / 3
Agenda
Introduction
Conversion Preparation
Additional Information and Contacts
Database Export, Conversion and Import
The Unicode System
2
1
3
6
4Demo
Demo
5 Customer Experiences
SAP AG 2005, SAP TechEd ’05 / SPC202 / 4
„[Unicode is] the encoding standard [which] provides the basis forprocessing, storage and interchange of text data in any language in allmodern software and informationtechnology protocols.“
What is Unicode?
see: http://www.unicode.org
Each character has a unique number („Unicode scalar value“).Notation U+nnnnnn (where nnnnnn are hexadecimal digits)
SAP AG 2005, SAP TechEd ’05 / SPC202 / 5
Unicode…
…allows text data from different languages to be stored in onerepository.
…enables a single set of source code to be written to process datain virtually all languages.
…simplifies the addition of new language support to an e-businessapplication since character processing and storage remainsunchanged.
Lower costs of implementation!
Faster speed to market!
Better customer satisfaction!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 6
Who needs Unicode?
Companies running global business processes like Global HRSystems or Global Master Data Management
Companies offering Web Services to their customers: Global MasterData containing multiple local language characters!
Companies using Open Standards: J2EE and .NET integration(JAVA speaks Unicode!)
Collaborative Business: Integration of Third Party Products that runon different code pages
Acting in global business requiressupport of a Global Character Set!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 7
Application Impact: Global Master Data
Example: CRM Business Partner
SAP AG 2005, SAP TechEd ’05 / SPC202 / 8
Internationalized Software with Unicode
The Unicode Standard was adopted by industry leaders e.g. Apple,HP, IBM JustSystem, Microsoft®, Oracle, SAP, Sun™, Sybase,Unisys and many others.
Unicode is required by modern standards such as XML, Java™,ECMAScript (JavaScript™), LDAP, CORBA 3.0, and WML.
Unicode is the official way to implement ISO/IEC 10646 and issupported in many operating systems and all modern browsers.
SAP AG 2005, SAP TechEd ’05 / SPC202 / 9
SAP Pre-Unicode Solutions
Single Code Page SystemOne standard system code page only a limited set of languages is supported inthese systems.
Blended Code Page System (R/3 3.0D – R/3 4.6D)Multibyte blended code pages which contain characters from several standard codepages.Blended code pages are not standard code pages, but SAP-customized code pagesthat were devised to support an increased number of possible languagecombinations in a single code page.a) Ambiguous Blended Code Page System:
Two characters can share the same code point.b) Unambiguous Blended Code Page System:
Each code point refers exactly to one character.
MDMP System Configuration (R/3 3.1I – NetWeaver 2004)„Multiple Display/Multiple Processing“: MDMP Systems deploy more than onesystem code page on the application server. This allows languages to be usedtogether in one system although the characters of those languages are not coveredby the same code page.
SAP AG 2005, SAP TechEd ’05 / SPC202 / 10
Single Code Pages in SAP Systems
Turkish
CroatianCzechHungarianPolish
RumanianSlovakian
Slovene
English
GermanDanish
Dutch,Finnish
French, ItalianNorwegianPortugueseSpanish
Swedish
RussianUkrainian
Greek
Hebrew
Thai
Korean
Japanese Chinese
Taiwanese
Icel
andi
c
SAP AG 2005, SAP TechEd ’05 / SPC202 / 11
Why convert to Unicode?
Unicode defines the character set for……efficient text processing in any language…maintaining text data integrity
Unicode systems……need only one locale (Unicode locale = platform- independent)…integrate seamless in existing system landscape (SAP and non- SAPsystems)…provide all ISO 639-2 language keys and 86 additional country-specificlanguage keys
a total of 560 technically supported language keys!
Users can……enter and display any character from any script no matter which logonlanguage they use…print text data in multiple languages
SAP AG 2005, SAP TechEd ’05 / SPC202 / 12
Why convert to Unicode?
One character set in systemlandscape
Easy handling for endusers
Lower operation costs
SAP AG 2005, SAP TechEd ’05 / SPC202 / 13
SAP NetWeaver™ the integration platform?
Evolution of mySAP Technology
Unifies and aligns people,information and businessprocesses
Integrates across technologiesand organizational boundariesA safe choice with full .NET andJ2EE interoperability
The business foundation for SAPand partners
Powers business-readysolutions that reduce customintegrationIts Enterprise ServicesArchitecture increases businessprocess flexibility
DB and OS Abstraction.NET WebSphere
People Integration
Com
posi
te A
pplic
atio
n Fr
amew
ork
Process IntegrationIntegration
BrokerBusiness Process
Management
Information IntegrationBusiness
IntelligenceKnowledge
Management
Life Cycle M
anagement
Portal Collaboration
J2EE ABAP
Application Platform
Multi-Channel Access
SAP NetWeaverSAP NetWeaver™™
DB and OS Abstraction
Master Data Management
……
SAP AG 2005, SAP TechEd ’05 / SPC202 / 14
SAP NetWeaver™ with non-Unicode ABAP stack
Evolution of mySAP Technology
Unifies and aligns people,information and businessprocesses
Integrates across technologiesand organizational boundariesA safe choice with full .NET andJ2EE interoperability
The business foundation for SAPand partners
Powers business-readysolutions that reduce customintegrationIts Enterprise ServicesArchitecture increases businessprocess flexibility
DB and OS Abstraction.NET WebSphere
People Integration
Com
posi
te A
pplic
atio
n Fr
amew
ork
Process IntegrationIntegration
BrokerBusiness Process
Management
Information IntegrationBusiness
IntelligenceKnowledge
Management
Life Cycle M
anagement
Portal Collaboration
J2EE non-Unicode ABAP
Application Platform
Multi-Channel Access
SAP NetWeaverSAP NetWeaver™™
DB and OS Abstraction
Master Data Management
……
no
no
no
SAP AG 2005, SAP TechEd ’05 / SPC202 / 15
Only solution for full integration: Unicode
Evolution of mySAP Technology
Unifies and aligns people,information and businessprocesses
Integrates across technologiesand organizational boundariesA safe choice with full .NET andJ2EE interoperability
The business foundation for SAPand partners
Powers business-readysolutions that reduce customintegrationIts Enterprise ServicesArchitecture increases businessprocess flexibility
DB and OS Abstraction.NET WebSphere
People Integration
Com
posi
te A
pplic
atio
n Fr
amew
ork
Process IntegrationIntegration
BrokerBusiness Process
Management
Information IntegrationBusiness
IntelligenceKnowledge
Management
Life Cycle M
anagement
Portal Collaboration
J2EE Unicode ABAP
Application Platform
Multi-Channel Access
SAP NetWeaverSAP NetWeaver™™
DB and OS Abstraction
Master Data Management
……
yes
yes
yes
SAP AG 2005, SAP TechEd ’05 / SPC202 / 16
Unicode Conversion Project
Pre-Conversion
Unicode Conversion
Post-Conversion
Set up the UnicodeConversion Project
Check Prerequisites
Plan database downtimeduring conversion
Enable your CustomerDevelopments
Highly automatedprocess
Data Analysis – specialMDMP treatment
Unload /reload processfor small databases
Minimum downtime toolfor large databases
Unicode system is upand running
Verification ofData Consistency
Integration Testingfocused on
language handling
SAP AG 2005, SAP TechEd ’05 / SPC202 / 17
Agenda
Introduction
Conversion Preparation
Additional Information and Contacts
Database Export, Conversion and Import
The Unicode System
2
1
3
6
4Demo
Demo
5 Customer Experiences
SAP AG 2005, SAP TechEd ’05 / SPC202 / 18
Prerequisites
Related Workshops/Lectures at SAP TechEd 2005SPC250, Making Programs Unicode Enabled, Hands-onSPC251, Unicode Interfaces - Data Exchange between Unicodeand Non-Unicode Systems, Hands-on
Upgrade to Web AS 6.20 non-UnicodeFor more information, read the appropriate Upgrade Guide.
Unicode-based mySAP componentsFor current status, see SAP Note 79991.
Unicode-enabled ABAP and C/C++ programsFor more information, please visit the following sessions:
SAP AG 2005, SAP TechEd ’05 / SPC202 / 19
Upgrade Paths to Unicode (SAP R/3 Enterprise)
R/3 4.6c
Source System Target System
SAP R/3 EnterpriseExt. Set 2.00
non-Unicode
SAP R/3 EnterpriseExt. Set 2.00
Unicode
R/3 4.5b
R/3 3.1i
First upgrade, then conversionto Unicode!
R/3 4.6b
R/3 4.0b Conversion
Directupgrade
SAP AG 2005, SAP TechEd ’05 / SPC202 / 20
easyeasy
Conversion Preparation: Concept
Before the system conversion to Unicode is executed, all textdata stored in the database require language information.
Single Code Page Systems/Unambiguous Blended Code Page Systems(ca. 90% of all customer installations)
MDMP Systems/Ambiguous Blended Code Page Systems(ca. 10% of all customer installations)
WHY?
complexcomplex
CODE PAGE ASSIGNMENT
SAP AG 2005, SAP TechEd ’05 / SPC202 / 21
Old solution for multiple languages: MDMP*
West European View Japanese View Korean View
* Check your system type with report RSCPINST current configuration
SAP AG 2005, SAP TechEd ’05 / SPC202 / 22
Old solution for multiple languages: MDMP
West European View Japanese View Korean View
End of support with NetWeaver ‘04
(see notes 838402 and 79991)
As of release NetWeaver 04s MDMP will no longer be supported!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 23
Content in Unicode System
UnicodeUnicode
… we need to knowthe language/code page ofevery character …
SAP AG 2005, SAP TechEd ’05 / SPC202 / 24
MDMP -> Unicode Database Conversion
MDMP System(logon language DE)
Unicode System
RU
JA
KO
1500
8000
8500
4103
4103
4103
DE1100 4103
Code Page Switch
SAP AG 2005, SAP TechEd ’05 / SPC202 / 25
Conversion Preparation: Concept
Problem:Code page assignment in MDMP/Ambig. Blended Code PageSystems is complex, because…
ALL table rows in the database must beassigned a code page before the conversion!
CommonCharacter
Set
Solution:You can...1. use existing Language Keys2. create and maintain a Vocabulary
(collect words* assign language)
*‘Word’:entry in Vocabulary, default length 3-30 bytes,and contains characters outside range ofcommon character set
SAP AG 2005, SAP TechEd ’05 / SPC202 / 26
Problem Description
MDMP system is a “Mixed code page solution“Database contains textual data from several code pages.
Conversion of MDMP systems like a Single Code Page systemleads to data destruction
Example:MDMP System with Latin-1 (SAP 1100) and Latin-2 (SAP 1401)
Hex-Code on database: „0x50657472F9” could bePetr (1401)orPetrù (1100)
To Do:Clarify the original code page of all entries with specialcharacters before the conversion.
SAP AG 2005, SAP TechEd ’05 / SPC202 / 27
Example: Russian text data in MDMP system
The Hex-code entry 0xE2D5DAE1E2 is found in adatabase table which has no language key.
MDMP system configuration
ISO-8859-1 (Western Europe)ISO-8859-2 (Eastern Europe)ISO-8859-5 (Cyrillic)
The entry could be either:
âÕÚáâ EnglishÚáâ Polish
Russian
Worst case: Russian speakers have to identify the data manually.
E2 D5 DA E1 E2ISO-8859-1 â Õ Ú á âISO-8859-2 â Ú á âISO-8859-5
SAP AG 2005, SAP TechEd ’05 / SPC202 / 28
Table Categories
Code page dependent with Language Key (I)
The table contains text data with non-ASCII characters:• Single code page dependent• Multiple code page dependent
Code page dependent without Language Key (II)
The table contains text data with non-ASCII characters.The data is therefore code page dependent.The Language Key has to be identified.
ASCII (III)
The table contains only text data with ASCII characters. The data is not codepage dependent. It is irrelevant which code page is used for the conversion.
After the conversion preparation each database table is assignedto one table category!
Vocabulary
Vocabulary
SAP AG 2005, SAP TechEd ’05 / SPC202 / 29
Conversion Preparation: Transaction SPUMG
Pre-ConversionUnicode Conversion
Post-ConversionSet up theConversionProject
CheckPrerequisites
Plan databasedowntime duringconversion
Enable CustomerDevelopments
Unicode system isup and running
Verification ofData Consistency
IntegrationTesting focusedonlanguagehandling
SUMG
ConversionCompletion
in theUnicodesystem
SAPinst(R3load)
DatabaseExport,
Conversion& Import
SPUMG
ConversionPreparation
innon-Unicode
system
SAP AG 2005, SAP TechEd ’05 / SPC202 / 30
SPUMG
Consistency CheckTables without Language InformationTables with Ambiguous LanguageInformationTables with Language InformationReprocessINDX Table AnalysisINDX Table Repair
SPUMG consists of several Scan Levels:
Database analysis tool for collecting words withoutlanguage/code page information. SPUMG createscontrol information for the database conversion.
SingleCode Page
andMDMP!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 31
SPUMG Language Lists
Languages installed in your system:
a) activeb) inactive
Active flag must beset for all languageswhich shall be usedfor code pageassignment!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 32
SPUMG Settings
Maintain the SPUMG settings and then initialize the worklist forthe Consistency Check!
GlobalFallbackCode Page
Settings for theVocabulary:MDMP only!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 33
Consistency Check
Classifies tables into tables with or without languageinformation (Table Category 1 or 2)
Checks table consistency (existence, access)
Writes control information to SPUMG control tables(Table Category, Language field)
ExportControlTable
Vocab.
ConsistencyCheck
WithoutLang. Info
With Lang.Info
ReprocessWith Amb.Lang. Info
Single CodePage andMDMP!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 34
Tables without Language Information
ControlInfo
Vocab.
All tables without language information(Table Category 2) are scanned.
This scan adds words to the Vocabulary!
ConsistencyCheck
WithoutLang. Info
With Lang.Info
ReprocessWith Amb.Lang. Info
MDMPonly!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 35
Tables with Ambiguous Language Information
All tables with language information (TableCategory 1) are scanned. Words with anambiguous language are added to theVocabulary.
Only active if ambiguous language list hasbeen maintained!
ConsistencyCheck
WithoutLang. Info
With Lang.Info
ReprocessWith Amb.Lang. Info
ControlInfo
Vocab.
MDMPonly!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 36
Vocabulary
The Vocabulary must be maintained before thedatabase export is executed.
You can:Insert language keys automatically on Scan Level Tables with LanguageInformation.Create Vocabulary Hints to assign a language based on other tablefields.Manually assign language to word in the Vocabulary.Reuse existing language code page assignmentsimported from other systems or delivered by SAP.Automatic language assignment with CharacterStatistics.Use of language patterns. SAP Notes
756534756535871541
SAP Notes756534756535871541
MDMPonly!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 37
Vocabulary
1.
MDMPonly!
2.3.
SAP AG 2005, SAP TechEd ’05 / SPC202 / 38
Tables with Language Information
Voc-
abulary
All tables with language information (Table Category 1)are scanned.
This scan assigns a language to words in theVocabulary based on the values of other tables in thedatabase.
Problem: Vocabulary Collisions
ConsistencyCheck
WithoutLang. Info
With Lang.Info
ReprocessWith Amb.Lang. Info
MDMPonly!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 39
Reprocess
This scan simulates R3load behavior.
It checks if the code page information in the Vocabularyis sufficient for the conversion. If not, this scan createsa Reprocess Log for the table.
(row identifier + language assignment)
ControlInfo
Voc-abulary
ConsistencyCheck
WithoutLang. Info
With Lang.Info
ReprocessWith Amb.Lang. Info
Re-processLog
MDMPonly!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 40
Reprocess Log
The Reprocess Logcontains
table name,key values,reason why nocode page couldbe assigned.
Users can assign alanguage to eachentry here. Thisinformation is usedin the Unicodesystem to repairwrongly converteddata.
MDMPonly!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 41
INDX-type tables consist of:
a transparent part which is treated like the other transparent tables inthe database during the conversion preparation.
a binary part which contains the code page information used during theEXPORT TO DATABASE /IMPORT FROM DATABASE statement.
In MDMP Systems the handling of INDX-type tables is improper in away that a wrong code page might be stored in the binary part of thetable.
Why must INDX-type tables be scanned separately?
INDX-type Tables: Special Treatment
SAP AG 2005, SAP TechEd ’05 / SPC202 / 42
LRAWINT2INT4C(n)C(2)C(3)
CLUSTDCLUSTR…….SRTF2SRTFDRELIDMANDT
key
Client Area ID Line cnt User data (opt) lng of data Data cluster
Binary PartTransparent Part
Character area
INDX-type Tables: Structure
SAP AG 2005, SAP TechEd ’05 / SPC202 / 43
MDMPonly!
INDX Table Analysis/ Repair
INDX TableAnalysis
INDX TableRepair
AdjustCode Page Infoin INDX clusterdata
Voca-
bulary
ConsistencyCheck
These two scans treat the binary partof INDX-type tables:
1. All INDX-type tables without language information arescanned and all text hidden in the INDX-cluster part isanalyzed. Words are added to the Vocabulary.
2. The Vocabulary is used for assigning correct codepages before the database export.
Voca-
bulary
Maintenance
SAP AG 2005, SAP TechEd ’05 / SPC202 / 44
Final Steps in non-Unicode Systems
Newly created tables can be added toSPUMG by updating the Worklist. Youcan display the new tables in theUpdate Log.
Perform additional preparation stepsas described in the documentation„Unicode Conversion Guide“.
MDMPonly!
Single CodePage andMDMP!
One important additionalpreparation steps isGenerating the Unicode nametab.
Note:Afterwards you must not import newor modify existing DDIC objects!Otherwise the Unicode nametabwill become invalid.
SAP AG 2005, SAP TechEd ’05 / SPC202 / 45
Agenda
Introduction
Conversion Preparation
Additional Information and Contacts
Database Export, Conversion and Import
The Unicode System
2
1
3
6
4Demo
Demo
5 Customer Experiences
SAP AG 2005, SAP TechEd ’05 / SPC202 / 46
Demo
Demo
SPUMG
SAP AG 2005, SAP TechEd ’05 / SPC202 / 47
Agenda
Introduction
Conversion Preparation
Additional Information and Contacts
Database Export, Conversion and Import
The Unicode System
2
1
3
6
4Demo
Demo
5 Customer Experiences
SAP AG 2005, SAP TechEd ’05 / SPC202 / 48
Database Export, Conversion and Import
Pre-ConversionUnicode Conversion
Post-Conversion
SUMG
ConversionCompletion
in theUnicodesystem
SPUMG
ConversionPreparation
innon-Unicode
system
SAPinst(R3load)
DatabaseExport,
Conversion& Import
Unicode system isup and running
Verification ofData Consistency
IntegrationTesting focusedonlanguagehandling
Set up theConversionProject
CheckPrerequisites
Plan databasedowntime duringconversion
Enable CustomerDevelopments
SAP AG 2005, SAP TechEd ’05 / SPC202 / 49
Database Export, Conversion and Import
When you perform a Unicode Conversionyou perform a System Copy!
You can download the applicable System Copy Guide from SAPService Marketplace Quick Link / instguides.
Documentation:“Homogeneous and Heterogeneous System Copy for SAP Systemsbased on”:
For information about Java System Copy, see SAP Note 795267
Preparation tasksExport the database with SAPinstInstall database on target systemImport the database dump with
SAPinstPost-Processing tasks
SAP Web AS 6.20SAP Web AS 6.40SAP Web AS ABAP 6.40 SR1
SAP AG 2005, SAP TechEd ’05 / SPC202 / 50
Database Export, Conversion and Import: Tools
R3load performs the database export including conversion toUnicode by using the Export Control Table and theVocabulary. R3load writes a log in case code pageinformation is not available, and it performs the databaseimport.
The system setup tool SAPinst is used for the entire systemcopy - internally SAPinst uses the program R3load.
The Distribution Monitor is an additional tool for reducing thesystem downtime to an acceptable limit by taking advantageof additional hardware (e.g. application servers) of thecustomer’s system landscape (pilot project).
SAP AG 2005, SAP TechEd ’05 / SPC202 / 51
SAPinst Export (SAP R/3 Enterprise Ext. 2.00)
Intro
Advanced Options
General Parameters
SAP AG 2005, SAP TechEd ’05 / SPC202 / 52
R3load - Export
SAPinst is used as „Frontend-tool“Logs are written to the SAPinst DirectoryExport Files are written to the Export Directory
SAP Systemnon-Unicode
Export DIR:SAPAPPL0.001SAPAPPL1.001
.....
Conversion (Non-UC UTF-16) during Export:
DB independent format(depending on platform
big or little endian)
R3load.exe -datacodepage 4103 –e ....
Space needed on Export Dir:~10-30% of DB size
Multiple processespossible
Default: One process (STR file) per dataclass (see SE11 Technical Settings)Advanced Export options: 50 largesttables get own process each
Example: „für“ (0x66 FC 72 0x0066 00FC 0072)
Non-UC R3load
SAP AG 2005, SAP TechEd ’05 / SPC202 / 53
R3load - Import
SAPinst is used as „Frontend-tool“Non-UC DB can be deleted (not recommended) or UC DB can beinstalled on different serverImport Procedure nearly same as New Installation
SAPSystemUnicode
Export DIR:SAPAPPL0.001SAPAPPL1.001
.....
Multiple processespossible.Number of processessame as during export(default)
UC R3loadR3load.exe -i ....
SAP AG 2005, SAP TechEd ’05 / SPC202 / 54
Database Export, Conversion and Import: Overview
SPUMG
R3load
ControlTable
Voca-bulary
EXPORT IMPORT
nonUnicode
Conversion
Re-process
Log
UnicodeData R3load
R3loadLog
SAP AG 2005, SAP TechEd ’05 / SPC202 / 55
EXPORT
Database Export, Conversion and Import: Overview
R3load
IMPORT
nonUnicode UnicodeData R3load
SPUMG
ControlTable
Voca-bulary
Re-process
Log
R3loadLog
SAP AG 2005, SAP TechEd ’05 / SPC202 / 56
Agenda
Introduction
Conversion Preparation
Additional Information and Contacts
Database Export, Conversion and Import
The Unicode System
2
1
3
6
4Demo
Demo
5 Customer Experiences
SAP AG 2005, SAP TechEd ’05 / SPC202 / 57
Unicode System – First Steps
Install Unicode Kernel + otherUnicode Executables
Single CodePage andMDMP!
Start Unicode System
Logon to Unicode System
1
2
3
SAP AG 2005, SAP TechEd ’05 / SPC202 / 58
Database Export, Conversion and Import: Overview
R3load
EXPORT IMPORT
nonUnicode UnicodeData R3load
R3loadLog
SPUMG
ControlTable
Voca-bulary
Re-process
Log
ControlTable
Voca-bulary
Re-process
Log
Conversion
SAP AG 2005, SAP TechEd ’05 / SPC202 / 59
R3load
EXPORT IMPORT
nonUnicode UnicodeData R3load
SPUMG
ControlTable
Voca-bulary
Re-process
Log
SUMG
ControlTable
Voca-bulary
Re-process
Log
R3loadLog
Database Export, Conversion and Import: Overview
SAP AG 2005, SAP TechEd ’05 / SPC202 / 60
SPUMG Statistics
Run report UMG_SCAN_STATISTICS. This report collectsand analyzes SPUMG statistics, e.g. database size,number of Vocabulary entries, R3load runtime, ReprocessLog size.
For enhancement monitoring, SAP kindlyrequests customers to send an e-mail with the followinginformation to [email protected]:
UMG_STATS_<SID>.xml (UMG_SCAN_STATISTICS file)database size before and after the conversionexport and import runtimes
Single CodePage andMDMP!
SAP AG 2005, SAP TechEd ’05 / SPC202 / 61
Unicode System: Transaction SUMG
Pre-ConversionUnicode Conversion
Post-Conversion
SPUMG
ConversionPreparation
innon-Unicode
system
SAPinst(R3load)
DatabaseExport,
Conversion& Import
SUMG
ConversionCompletion
in theUnicodeSystem
Unicode system isup and running
Verification ofData Consistency
IntegrationTesting focusedonlanguagehandling
Set up theConversionProject
CheckPrerequisites
Plan databasedowntime duringconversion
Enable CustomerDevelopments
SAP AG 2005, SAP TechEd ’05 / SPC202 / 62
SUMG: Conversion Completion
In the Unicode system you might recognize data which have notbeen converted correctly. In order to ‘complete’ the conversionprocedure, the data can be converted once again in transactionSUMG, using the correct language information (code page).
SUMG provides two Completion Types
Automatical Completion:
Using the R3load Log and thelanguage information stored in theReprocess Log
Manual Repair:
Repair Hints
Manual Language (CP) Assignment
time
Down
Uptime
SAP AG 2005, SAP TechEd ’05 / SPC202 / 63
Automatical Completion
Enter path and name of thefile which contains thepath(s) of the XML-file(s).
1.
2.
R3load Log
table name, key values, codepage used for the conversion,error messages
Reprocess Log
table name, key value, lang.which were manually assignedin SPUMG
SAP AG 2005, SAP TechEd ’05 / SPC202 / 64
Manual Repair: Repair Hints
1. CreateCondition
2. Create Hint
3. Save Hint
Once created you can reuseHints in other systems.
4. ExecuteHint
SAP AG 2005, SAP TechEd ’05 / SPC202 / 65
1. Assign current lang.2. Assign correct lang.3. Save entry to
database
Manual Repair: Manual Language Assignment
Tables can be manuallyadded. Then for eachbroken entry the correctlanguage must berecognized and assigned.
SAP AG 2005, SAP TechEd ’05 / SPC202 / 66
Agenda
Introduction
Conversion Preparation
Additional Information and Contacts
Database Export, Conversion and Import
The Unicode System
2
1
3
6
4Demo
Demo
5 Customer Experiences
SAP AG 2005, SAP TechEd ’05 / SPC202 / 67
Demo
Demo
SUMG
SAP AG 2005, SAP TechEd ’05 / SPC202 / 68
Agenda
Introduction
Conversion Preparation
Additional Information and Contacts
Database Export, Conversion and Import
The Unicode System
2
1
3
6
4Demo
Demo
5 Customer Experiences
SAP AG 2005, SAP TechEd ’05 / SPC202 / 69
Customer Experiences
Customer Experiences: Hardware Requirements
Conversion Project: Timeline
Customer Experiences: Runtimes
Customer Project Example
SAP AG 2005, SAP TechEd ’05 / SPC202 / 70
Customer Experiences: Hardware Requirements
The figures are based on parallel benchmarking of Unicode /non-Unicode test systems and on customer experiences.
Database Size GrowthUnicode Encoding Form
up to +60...70%UTF-16
up to +10%UTF-8
SAP AG 2005, SAP TechEd ’05 / SPC202 / 71
Conversion Project: Timeline
The project timelinedepends on a number of factors:
How long does it taketo converta system?
SAP AG 2005, SAP TechEd ’05 / SPC202 / 72
Dependencies
Single Code Page or MDMP Conversion?MDMP requires more preconversion tasks and postconversion handlingin the Unicode system
Biggest Tables – Optimize parallelization of Export/Importprocesses (splitting of R3load packages)
Processing of Cluster TablesSizes of cluster tables (compared totransparent tables)
HardwareNumber and speed of CPUsPerformance of disksSeparate server available for Unicode System?
Time spent on optimization (problems not related to runtime)
SAP Note784118857081
SAP Note784118857081
SAP AG 2005, SAP TechEd ’05 / SPC202 / 73
Customer Experiences: Runtimes
Unicode conversion throughput
0
20
40
60
80
100
120
140
160
180
200
0 500 1000 1500 2000 2500 3000 3500 4000
DB size in GB
Spee
d in
GB
/h
Depending on hardware andparallelization effort, theR3load runtimesthroughput is 100 – 200 GB/h.
SAP AG 2005, SAP TechEd ’05 / SPC202 / 74
Distribution Monitor: Pilot Project
00:00
01:12
02:24
03:36
04:48
06:00
07:12
0 5 10 15 20 25
total paralle l jobs
elap
sed
time limited CPU power (dual
CPU host)scaled CPU power (1 or2 jobs per CPU)
Runtime measurements show that using additionalhardware (CPU power) speeds up the overall runtime of thesystem copy process.
Exportruntimes
Motivation
SAP AG 2005, SAP TechEd ’05 / SPC202 / 75
Distribution Monitor
R3load processes are distributed among multiple applicationservers.
Hardware 1
Hardware 2
Hardware 3
Hardware 4
DB ServerNUC
DB ServerUC
R3load Exp. & Imp.
R3load Exp. & Imp.
R3load Exp. & Imp.
R3load Exp. & Imp.
SAP AG 2005, SAP TechEd ’05 / SPC202 / 76
Distribution Process
PREPARATION PHASE
Generation of control information and Migration Monitor properties.
Preparation is performed only on one designated machine.
EXPORT AND IMPORT PHASE
Both are started on each machine. Export and Import can run inparallel.
Preparation Phase must be completed first!
SAP NotePilot Release
855772
SAP NotePilot Release
855772
SAP AG 2005, SAP TechEd ’05 / SPC202 / 77
Customer Project Example
Paul Hartmann AG
Conversion of MDMP system to Unicode:
3 Standard single code pages with 16 installed system languagesISO-8859-1 (1100): DA, DE, EN, ES, FR, IT, NL, NO, PT, SVISO-8859-2 (1401): CS, HU, PL, SK, SLISO-8859-7 (1700): EL
SAP AG 2005, SAP TechEd ’05 / SPC202 / 78
Customer Project Example: Schedule
01/23/2004 Kick-Off-Meeting
02/27 – 04/01/2004 SPUMG tests (afterwards RESET_ALL)
04/02 – 08/04/2004 Conversion of first sandbox system
08/10 – 10/14/2004 Conversion of second sandbox system
10/12 – 10/30/2004 Conversion of development system (DEV)
10/12 – 11/02/2004 Conversion of production system (PRD)
10/31/2004 Go-Live
11/02/2004 Systemcopy PRD quality system (QAS)
The schedule also includes: application and integration tests,preconversion data reduction (archiving and deletion of obsoletedata), runtime measurements and optimization)
SAP AG 2005, SAP TechEd ’05 / SPC202 / 79
Customer Project Example: PRD System
SAPKB62040SAP Basis 6.20 Support Package
133.600.000PSAPTEMP size (in KB)
44.467.397Export size (in KB)
523.101.248Database size (UC, in KB)
551.775.684Database size (non-UC, in KB)
5x PowerPC_POWER4(24 CPUs, 1.4-1.5 GHz, 127 GB RAM)
Hardware (Unicode)
5x PowerPC_POWER4(24 CPUs, 1.4-1.5 GHz, 47 GB RAM)
Hardware (non-Unicode)
DB2 UDB V8 FP3Database version
AIX 5.2Platform (Unicode)
AIX 5.2Platform (non-Unicode)
Production System (PRD)System type
R/3 Enterprise 4.7 Ext. Set 1.10SAP Release
SAP AG 2005, SAP TechEd ’05 / SPC202 / 80
Customer Project Example: Runtimes
*7,9 hSUMG: Automatical Completion1,5 hGenerate the Unicode-Nametab
*11 minScan: INDX Table Repair*107,2 hScan: Reprocess
8 minImport of maintained Vocabulary*-Scan: Tables with Language Information*1,7 hScan: INDX Table Analysis*6,3 hScan: Tables with Ambiguous Language Information*144,8 hScan: Tables without Language Information*20,7 hCheck cluster tables*24,5 hScan: Consistency Check
47 minSet missing Language Flags7 minFill TRDIR Language Keys
RuntimeStep
„*“ = Parallelization possible by scheduling several Jobs.
Runtimes of automated conversion steps
SAP AG 2005, SAP TechEd ’05 / SPC202 / 81
Sandbox System <–> Production System
-...Application tests
12 h12 hSUMG: Manual Repair
0,5 h0,5 hINDX Log Maintenance
5,5 days5 daysReprocess Log Maintenance
-10 daysResolve collisions
6 h33 days*Vocabulary Maintenance
0,5 days3 daysDeletion of obsolete data
´-200 daysUnicode-enabling of customer programs
Runtime(PRD)
Runtime(Sandbox/DEV)Action
*new Vocabulary Maintenance Tools (see SAP Notes)have not been used.
SAP Note756534
SAP Notes756534, 756535
871541
SAP Notes756534, 756535
871541
SAP AG 2005, SAP TechEd ’05 / SPC202 / 82
01:00 – 10:15 Import (incl. create all indexes and views)10:15 – 14:45 Update statistics14:45 – 15:30 First steps in UC15:30 – 17:40 Offline Backup of UC17:40 – 23:00 SUMG; RFC destinations; key-user tests
21:30 – 23:00 Generate Unicode nametab23:00 – 00:00 Offline Backup of non-UC; preparation tasks
00:00 – 01:00 Offline Backup of non-UC; preparation tasks01:00 – 20:00 Export20:00 – 22:00 Drop database, delete syslog files22:00 – 22:30 Preparation of Import22:30 – 01:00 Create tablespaces
23:00 Release for endusers
09:00 – 23:00 Finish SPUMG Preparation (Repairlog),Export of proposal pool, stop of transports)
Conversion of Production System: Schedule
FR
SA
SU
Dow
ntim
eUptime
Up time
SAP AG 2005, SAP TechEd ’05 / SPC202 / 83
Agenda
Introduction
Conversion Preparation
Additional Information and Contacts
Database Export, Conversion and Import
The Unicode System
2
1
3
6
4Demo
Demo
5 Customer Experiences
SAP AG 2005, SAP TechEd ’05 / SPC202 / 84
Additional Information and Contacts
Printing in Unicode Systems
Upgrade and Conversion Paths
Platforms and Databases
Documentation
References and Contacts
SAP AG 2005, SAP TechEd ’05 / SPC202 / 85
Printing in Unicode Systems
1. HP Unicode Printer + HPUTF8 device type + SAP Andale_* Fonts2. Lexmark Unicode Printer + LEXUTF8 device type + SAP Andale_* Fonts
SAPscript formsSmart FormsABAP Lists
High-speed printing of documents with single-byte and multi-byte native languagecharacters.
Printing of all Unicode characters on any printer with SAPlpd + cascading fonts-enabled SAPWIN device type. No modification of forms required!
SAPscript formsSmart FormsABAP Lists
not yet supported
supported for 6.40; 6.20 SP55
development in progress
Unicode UTF-8 Printing Solutions
Cascading Fonts Printing Solution
215015750219
215015750219
812821812821
SAP AG 2005, SAP TechEd ’05 / SPC202 / 86
6.20
Upgrade and Unicode Conversion paths
4.6 6.40 7.0Non-
Unicode:Single
Code Page
Non-Unicode:
MDMP4.6 6.20 6.40
Unicode 6.20 6.40 7.0
Upgrade Path Unicode Conversion Path Upgrade&Conversion Path
SAP AG 2005, SAP TechEd ’05 / SPC202 / 87
Upgrade and Conversion 4.6C/D -> ERP 2005
ERP 2005
Release Action
Toolimport
Upgrade Preparation
Unicode Conversion Preparation
Upgrade
Unicode Conversion
4.6C or4.6D
Upt
ime
Dow
ntim
e
Timeline
SAP AG 2005, SAP TechEd ’05 / SPC202 / 88
Platforms and Databases
SAP supports Unicode systems on the following platforms:
Default = 64 bit versions132 bit versions still available
²There will be no support for Informix and Tru64!
Operating System
Dat
abas
esy
stem
SAP Note379940
--x-xxxxSAP DB
--------Informix²
-xxxxxxxDB/2
--x-xxxxOracle
-------xSQLServer
Tru64²AS/400AIXOS/390HPSolarisLinux1W2K1
SAP AG 2005, SAP TechEd ’05 / SPC202 / 89
Documentation
The following documents are required for the conversionof non-Unicode SAP systems to Unicode:
Unicode ConversionGuide; available forWeb AS 6.20/6.40.
System Copy Guide;available for Web AS6.20/6.40/6.40SR1
SAP Note 548016:application-specificinformation.
SingleCode Pageand MDMP!
SAP Note548016
SAP Note548016
SAP AG 2005, SAP TechEd ’05 / SPC202 / 90
References and Contacts
Unicode Conversion Library
https://service.sap.com/unicode@sap
SAP AG 2005, SAP TechEd ’05 / SPC202 / 91
References and Contacts
https://service.sap.com/globalization
SAP AG 2005, SAP TechEd ’05 / SPC202 / 92
Q&A
Questions?
SAP AG 2005, SAP TechEd ’05 / SPC202 / 93
Further Information
Public Web:NetWeaver Developer‘s Guide: www.sdn.sap.com/sdn/developersguide.sdn
SAP AG 2005, SAP TechEd ’05 / SPC202 / 94
Please complete your session evaluation.
Be courteous — deposit your trash,and do not take the handouts for the following session.
Feedback
Thank You !
SAP AG 2005, SAP TechEd ’05 / SPC202 / 95
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The informationcontained herein may be changed without prior notice.Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.Microsoft, Windows, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries, xSeries, zSeries, z/OS, AFP,Intelligent Miner, WebSphere, Netfinity, Tivoli, and Informix are trademarks or registered trademarks of IBM Corporation.Oracle is a registered trademark of Oracle Corporation.UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.Java is a registered trademark of Sun Microsystems, Inc.JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.MaxDB is a trademark of MySQL AB, Sweden.SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, and other SAP products and services mentioned herein as well as their respective logos aretrademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentionedare the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.
The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purposewithout the express prior written permission of SAP AG.This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intendedstrategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, productstrategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice.SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics,links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limitedto the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use ofthese materials. This limitation shall not apply in cases of intent or gross negligence.The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use ofhot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Webpages.
Copyright 2005 SAP AG. All Rights Reserved
SAP AG 2005, SAP TechEd ’05 / SPC202 / 96
Cascading Fonts: Customizing
SAP AG 2005, SAP TechEd ’05 / SPC202 / 97
Cascading Fonts: Output example
Both documents were printed out with font ‚HELVE‘.
no Cascading Fonts Cascading Fonts
SAP AG 2005, SAP TechEd ’05 / SPC202 / 98
ISO 8859-1 (Latin-1)
0 1 2 3 4 5 6 7 8 9 A B C D E F
Example: MDMP with Latin-1 and Latin-2
Common Character Set
ISO 8859-2 (Latin-2)
ABCDEF
ABCDEF
CommonCharacters