SAP Sybase Replication Server Change DATA … Sybase Replication Server Change DATA Capture Configuration 13 16. After completion of Replication Wizard, Power Designer will create

SAP Sybase Replication Server Change DATA Capture Configuration

Example Configuration


2

TABLE OF CONTENTS 1 INTRODUCTION ............................................................................................................................................ 3 1.1 Scope .......................................................................................................................................................... 3 1.2 Purpose ...................................................................................................................................................... 3

2 CONFIGURATION ......................................................................................................................................... 4 2.1 Requeriments ................................................................................................................................... 4 2.2 Notes ........................................................................................................................................................... 4 2.2 Configuration steps ......................................................................................................................... 4


3

1 INTRODUCTION

New SAP solution called “SAP Sybase Replication Server Change Data Capture” combines SAP

Sybase Replication Server, one of the most used and powerful tools for real-time data replication, with SAP Data Services, one of the most used and powerful ETL tools. Implementing this new solution gives you the ability to perform real-time replication with complex data transformations for your business processes.

1.1 Scope

This document is useful for people interested in installing this solution and checking how to configure SAP Sybase Replication Server to capture data changes and send them to a CDC staging database to be processed by SAP Data Services. 1.2 Purpose

The basic purpose of the document is be used as cookbook to show how to model a SRS process with PowerDesigner 16.5 to create a CDC staging database is ASE 15.7. PowerDesigner will be able to generate data model for the CDC staging DB and also data movement model and will create all scripts needed (both at DB level and at SRS level) to active data replication to the CDC staging DB.


4

2 CONFIGURATION

2.1 Requeriments

SAP Sybase Repliation Server 15.7.1

SAP ASE 15.7

SAP Data Services 4.2

PowerDesigner 16.5 2.2 Notes

For this example/demo primary DB (called “testdb”) will be located in an SAP ASE 15.7 (called ASE157) but it can be any source DBMS supported by SRS.

CDC staging database will be located in the same ASE and will be called “testdb_stage”.

Replication server is supposed to be installed, running and with the connections to the primary DB and staging DB created.

This doc describes configuration related to SAP Sybase related products and not Data Services.

2.2 Configuration steps

1. “testdb” and “testdb_stage” will be located in same ASE (ASE157).

In “testdb” there is a table t1 that will be replicated (noticed that the table must have a primary key). testb_stage database has just been created and no user tables in it (only SRS tables).


5

2. Replication server is configured, up and running with connections to both “testdb” and

“testdb_stage” created:


6

3. Power Designer automates the process of configuring Sybase Replication Server, which includes

marking the source table as replicated, creating data model for the CDC staging database and

creating replication definitions and subscriptions.

The extended PD model for Data Services CDC distributed with Data Services 4.2 has to be copied to the extended models definitions folder in PowerDesigner (the .xem file is called SybaseRepServerDataServicesCDC100.xem):


7

4. PDM model created for the primary db:

5. Create a new Data Movement Model:


8

6. Attach extensions “SAP Data Services CDC” and “Replication Server 15.7.1 SP100” (see “select

extensions” button on the right-bottom side of the window):

7. Run the Replication Wizard. It creates the replication server process, source physical data model

and replicated physical data model. In this case physical model for the source has been created

previously.

From PowerDesigner Tools Replication Wizard Source Database is ASE157.testdb


9

8. Choose “Select existing Physical Data Model” and check ASE157-testdb that this the model for the

source database

9.

10. Use the repserver name as name for the replication process, in this case r157:


10

11. In Replication Wizard: Publication Selection, selects “Create or select a single publication to gather

replicated tables” and give a name for the publication:

12. Select objects to replicate, in this case only table t1:


11

13. Connection to the CDC staging database:

14. Create a new physical data model for CDC staging DB for ASE 157:


12

15. Finish and get the Data Movement Model:


13

16. After completion of Replication Wizard, Power Designer will create default replication definitions

(called Articles) and PDM for staging database with all selected tables from source PDM and

corresponding table columns and properties.

Double click on Replication Server process Publications & Replication Definitions to see the publication:


14

17. Check properties for the publication to see the articles included in it:

Go further to check article properties (columns, where clauses …etc), it is possible to check the generated code for the publication and articles in the “preview” tab:


15


16

18. Configure a connection to CDC staging DB:

On the object Browser, right click ASE157.testdb_stage (CDC staging DB) and select properties

19. Inside Database Properties window select “Database Connection” tab, click “Select a Data Source”

and select the DSN for the connection to staging DB or create it if it does not exists:


17

Click OK

20. Save DMM and PDM.


18

21. Select Replication Process icon in DMM diagram and choose “Add Data Services CDC Definitions”

from context menu to run Data Services extension to create:

a. CDC replication definitions,

b. CDC staging tables ,

c. CDC system tables (DS_CDC_TABLES_MAP, DS_CHANGE_RETENTION and

DS_PURGE_TIMESTAMP),

d. PURGE_OLD_CHANGE_DATA stored procedure and

e. User SAPDS.


19

22. Check CDC system tables, stating tables, PURGE_OLD_CHANGE_DATA stored proc, user

SAPDS and publications/replication definitions have been created, it is possible to check function

strings too:


20

23. Running “Add Data Services CDC Definitions” command automatically changes owner of replicated

tables, Purge stored procedure, CDC staging table and CDC system tables to SAPDS. It defines CDC

staging table name same as Remote Table name of article with suffix “_SAPCDC”. It creates or

overrides function strings - rs_delete, rs_insert, rs_update and rs_get_textptr for blob columns. The

CDC staging table defines additional columns in addition to RemoteTable columns to store Replication

Server system variables:

a. DS_SRCDB - stores system variable “rs_origin_db”

b. DS_SRCDB_SRV – stores system variable “rs_origin_ds”

c. DS_SRCDB_ID – stores system variable “rs_origin”

d. DS_SRCDB_COMMIT_TIME - stores system variable “rs_origin_commit_time”

e. DS_SRCDB_ORIGIN_QID – stores “rs_origin_qid”

f. DS_SRCDB_OP_TYPE – stores change operation types ‘I’ – insert , ‘B’- Before image of

update,’ U’- After image of update, ‘D’-Delete

g. DS_CDCDB_PUBLISH_TIME – stores publish datetime of CDC database when record get

inserted by Replication Server


21

24. “SAP Data Services CDC” extension added two extended attributes to Article object. They are

“Retention Period” and “CDC Staging Table”. Default value “Retention Period” extended attributes is 5

days, You can change this value. “Add Data Services CDC Definitions” command sets “CDC Staging

Table” attributes with CDC staging table name for each article.

Physical data models and Data Movement model are now ready. To finish Data Services Sybase Replication Server CDC environment configuration these 4 steps are needed:

1. Create objects in the CDC staging DB.

2. Run Data Services Staging database runtime script.

3. Deploy replication definitions and subscriptions into SRS.

4. Configuring primary DB, mark source tables for replication.


22

1. CREATE OBJECTS IN THE CDC STAGING DB

1.1 Create SAPDS login in ASE157 and add SAPDS user in CDC staging DB:

1.2 Generate staging DB DDL script

Select the Data Services staging database PDM in Power Designer Data Movement Model and right click the context to select "Generate Database" to generate the CDC database DDL script. This automatically generates DDL of all the database objects that are required for Data Services CDC Staging database. The CDC database script is generated as a file:


23

1.3 Execute the script with isql and check that objects are created properly and give permissions to

maintenance user in staging tables:


24

2. RUN DATA SERVICES STAGING DATABASE RUNTIME SCRIPT

2.1 Generate script

Select the Data Services staging database PDM in Power Designer Data Movement Model and right click the context to select "Generate Script". In the pop-up window select “SAP Data Services CDC” extension and proceed. It generates script file suffixed with “_DataServices_CDC_DB_Runtime_Script.sql“ which creates Data Services CDC staging database runtime environment. This script provides following functionalities: a. Validates the staging database to find out if this database is added to replication server by

Replication Server rs_init utility.

b. Validates if Data Services CDC staging database create script is run.

c. Creates Replication Server maintenance user that defined in Data Movement Model if it is

not created before. DBA can change the password after creation to protect un-authorized

access.

d. Adds maintenance user to database.

e. Add TargetTable name and CDC staging table name to DS_CDC_TABLES_MAP table for

each replication definition.

f. Add CDC staging table name and retention period to DS_CHANGE_RETENTION table for

each replication definition.

g. Grants select, insert, update and delete privileges to CDC staging tables.

h. Grant execute privilege to “PURGE_OLD_CHANGE_DATA” stored procedure.

i. Grants Replication Server maintenance user privileges to execute “Purge Old Change Data”

scheduled job in Sybase ASE Job Scheduler database.

j. Creates “Purge Old Change Data” scheduled job in Sybase ASE Job Scheduler database.

“Purge Old Change Data” job name is SAPDS_<CDC Staging Database

Name>_PURGE_CHANGE_OLD_DATA.

2.1 Execute the script with isql:


25

3. DEPLOY REPLICATION DEFINITIONS AND SUBSCRIPTIONS INTO SRS

3.1 Define a primary DB for the subscriptions:

Double click Replication Server process in Power Designer Data Movement Model and select subscriptions, double click in subscription to get its properties and set Primary Connection to ASE157.testdb:


26

3.2 In Publications & Replication definitions (general tab) check that type is “publication”:


27

3.3 Generate script:

To generate Replication Server RCL script file select the Replication Process in Power Designer Data Movement Model and right click the context to select "Generate Script", set the folder and click OK to generate Replication Definitions, CDC Function Strings definitions and subscriptions. Connection has been created previously so Create Connection should be set to false:


28

3.4 Execute script with isql:

4. CONFIGURING PRIMARY DB, MARK SOURCE TABLES FOR REPLICATION

4.1 Generate script:

Select the primary databases in Power Designer Data Movement Model and right click the context to select "Generate Script" and select “Replication Server 15.7SP100”:

Only the script with sp_setreptable is needed, no need to generate the other ones:


29

Click Edit to check and be sure that owner on in set in the sp_setreptable call:

4.2 Execute script in primary DB testdb (removing // lines if they appear) to avoid syntax error:


30

Everything is now configured and changes in t1 at primary DB should be captured into the CDC staging table:

© 2013 SAP AG. All rights reserved.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP

BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP

products and services mentioned herein as well as their respective

logos are trademarks or registered trademarks of SAP AG in Germany

and other countries.

Business Objects and the Business Objects logo, BusinessObjects,

Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and

other Business Objects products and services mentioned herein as

well as their respective logos are trademarks or registered trademarks

of Business Objects Software Ltd. Business Objects is an SAP

company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL

Anywhere, and other Sybase products and services mentioned herein

as well as their respective logos are trademarks or registered

trademarks of Sybase Inc. Sybase is an SAP company.

Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are

registered trademarks of Crossgate AG in Germany and other

countries. Crossgate is an SAP company.

All other product and service names mentioned are the trademarks of

their respective companies. Data contained in this document serves

informational purposes only. National product specifications may vary.

These materials are subject to change without notice. These materials

are provided by SAP AG and its affiliated companies ("SAP Group")

for informational purposes only, without representation or warranty of

any kind, and SAP Group shall not be liable for errors or omissions

with respect to the materials. The only warranties for SAP Group

products and services are those that are set forth in the express

warranty statements accompanying such products and services, if

any. Nothing herein should be construed as constituting an additional

warranty.

www.sap.com

Documents

SAP Sybase Replication Server Change DATA … Sybase Replication Server Change DATA Capture Configuration 13 16. After completion of Replication Wizard, Power Designer will create