65
Data Replication Service FAQs Date 2020-03-31

Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Data Replication Service

FAQs

Date 2020-03-31

Page 2: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Contents

1 Product Consulting..................................................................................................................11.1 Region and AZ.......................................................................................................................................................................... 11.2 What Is DRS?............................................................................................................................................................................ 31.3 Which Databases Can DRS Migrate?................................................................................................................................61.4 Can DRS Migrate Data to Primary/Standby RDS DB Instances?............................................................................ 61.5 What Constraints Does DRS Have for a Source Database?......................................................................................71.6 What Requirements Does DRS Have for a Destination Database?....................................................................... 71.7 Does DRS Support Migration Between the Public Cloud and the Dedicated Cloud?......................................81.8 Can DB2 Be Migrated to PostgreSQL Using DRS?....................................................................................................... 81.9 Can Microsoft SQL Server Database Synchronize Data with Local Databases in Real Time?......................81.10 Does DRS Support Data Replication in a Specified Time Period?........................................................................81.11 Does DRS Support Resumable Uploads?...................................................................................................................... 81.12 What Is Single-Active/Dual-Active Disaster Recovery?............................................................................................9

2 Network and Security.......................................................................................................... 122.1 What Security Protection Policies Does DRS Have?................................................................................................. 122.2 What Can I Do If the Network Is Disconnected During the Migration?............................................................122.3 Which Database Accounts Are Required During Migration?................................................................................. 122.4 How Can I Set a VPC Security Group to Access to an Outside EIP?................................................................... 13

3 Permissions Management................................................................................................... 153.1 How Do I Set an Independent Oracle Account That Has the Least Privilege and Uses DRS?...................153.2 Which MySQL Permissions Are Required for DRS?................................................................................................... 163.3 How Can I Import Users and Permissions from the Source to the Destination Database?........................19

4 Data Migration.......................................................................................................................214.1 When Can I Stop a Migration Task?...............................................................................................................................214.2 How Do I Maintain the Original Service User Permission System After Definer Is Forcibly ConvertedDuring MySQL Migration?........................................................................................................................................................ 224.3 What Can I Do If the Invoking Permission Problem Occurs After the MySQL Stored Procedure IsMigrated to the Cloud?.............................................................................................................................................................. 234.4 How Do I Ensure that All Services on the Database Are Stopped?.....................................................................244.5 What Can I Do When Scheduled Task Failed to Start Leaving Message "can not get agency token"...254.6 What Do I Do If the Maximum Index Length Has Been Reached During Migration from Oracle toMySQL?............................................................................................................................................................................................ 25

Data Replication ServiceFAQs Contents

2020-03-31 ii

Page 3: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

4.7 Why Is the Collation of Heterogeneous or Oracle Databases Converted to utf8mb4_bin After ThoseDatabases Are Migrated to MySQL?..................................................................................................................................... 274.8 What Can I Do If MyISAM Tables Are Not Supported by RDS for MySQL?..................................................... 274.9 What Are the Precautions for Migrating Data from an Earlier Version MySQL to MySQL 8.0?...............284.10 What Can I Do When OOM Occurs During the Migration of MongoDB Databases?............................... 384.11 Which Plug-ins Can Be Migrated During PostgreSQL Database Migration?.................................................394.12 How Do I Disable the Balancer?................................................................................................................................... 404.13 How Do I Export and Import Events and Triggers in Batches?.......................................................................... 414.14 How Can I Migrate Databases or Tables Whose Names Contain Uppercase Letters?...............................424.15 What Should I Do If the Last Backup File Is Incorrectly Selected in the Backup Migration Scenario?............................................................................................................................................................................................................ 43

5 Data Synchronization........................................................................................................... 445.1 Can DRS Sync Tables of Different Schemas to the Same Schema?.................................................................... 44

6 Data Subscription.................................................................................................................. 456.1 How Long Does It Take for SDK to Consume Database Changes?..................................................................... 45

7 Multi-Active DR..................................................................................................................... 467.1 What Are RPO and RTO of DRS Disaster Recovery?................................................................................................ 467.2 How Do I Select Active Database 1 and 2 for Dual-Active DR?...........................................................................467.3 Common Multi-Active DR Scenarios.............................................................................................................................. 48

8 General Operations...............................................................................................................518.1 What Can I Do When Information Overlaps on the DRS Console?.................................................................... 518.2 How Do I Set Global binlog_format=ROW to Take Effect Immediately?..........................................................518.3 How Do I Change the Destination Database Password to Meet the Password Policy?...............................528.4 How Do I Configure the Shard Key for a MongoDB Sharded Cluster?.............................................................. 548.5 Does Bandwidth Expansion Affect the Running DRS Tasks?................................................................................. 568.6 Why Data in MariaDB and SysDB Cannot Be Migrated or Synchronized?....................................................... 568.7 Constraints and Operation Suggestions on Many-to-One Scenario................................................................... 568.8 Where Can I View DRS Operation Logs?...................................................................................................................... 59

A Change History...................................................................................................................... 60

Data Replication ServiceFAQs Contents

2020-03-31 iii

Page 4: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

1 Product Consulting

1.1 Region and AZ

ConceptA region and availability zone (AZ) identify the location of a data center. You cancreate resources in a specific region and AZ.

● Regions are divided from the dimensions of geographical location andnetwork latency. Public services, such as Elastic Cloud Server (ECS), ElasticVolume Service (EVS), Object Storage Service (OBS), Virtual Private Cloud(VPC), Elastic IP (EIP), and Image Management Service (IMS), are sharedwithin the same region. Regions are classified as universal regions anddedicated regions. A universal region provides universal cloud services forcommon tenants. A dedicated region provides services of the same type onlyor for specific tenants.

● An AZ contains one or multiple physical data centers. Each AZ hasindependent cooling, fire extinguishing, moisture-proof, and electricityfacilities. Within an AZ, computing, network, storage, and other resources arelogically divided into multiple clusters. AZs within a region are interconnectedusing high-speed optical fibers to allow you to build cross-AZ high-availabilitysystems.

Figure 1-1 shows the relationship between regions and AZs.

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 1

Page 5: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Figure 1-1 Regions and AZs

HUAWEI CLOUD provides services in many regions around the world. You canselect a region and AZ as needed. For more information, see HUAWEI CLOUDGlobal Regions.

How to Select a Region?

When selecting a region, consider the following factors:

● LocationYou are advised to select a region close to you or your target users. Thisreduces network latency and improves access rate. However, Chinesemainland regions provide basically the same infrastructure, BGP networkquality, as well as operations and configurations on resources. Therefore, ifyou or your target users are in the Chinese mainland, you do not need toconsider the network latency differences when selecting a region.– If you or your target users are in the Asia Pacific region, except the

Chinese mainland, select the AP-Hong Kong, AP-Bangkok, or AP-Singapore region.

– If you or your target users are in Africa, select the AF-Johannesburgregion.

– If you or your target users are in Europe, select the EU-Paris region.– If your target users are in Latin America, select the LA-Santiago region.

The LA-Santiago region is located in Chile.

● Resource priceResource prices may vary in different regions. For details, see Product PricingDetails.

How to Select an AZ?

When determining whether to deploy resources in the same AZ, consider yourapplications' requirements on disaster recovery (DR) and network latency.

● For high DR capability, deploy resources in different AZs in the same region.● For low network latency, deploy resources in the same AZ.

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 2

Page 6: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Regions and EndpointsBefore using an API to call resources, specify its region and endpoint. For moredetails, see Regions and Endpoints.

1.2 What Is DRS?DRS is a stable, efficient, and easy-to-use cloud service for database onlinemigration and synchronization.

It simplifies data migration processes and reduces migration costs.

You can use DRS to quickly transmit data between databases in various scenarios.

DRS provides multiple capabilities, including online migration, backup migration,active-active DR, data synchronization, and data subscription.

Online MigrationDRS allows you to directly migrate data from the source database to thedestination database on the condition that they are connected. In addition, thesource DB instances, destination DB instances, and migration objects must beconfigured.

Online migration supports migration through multiple types of networks, such aspublic networks, VPCs, VPNs, and direct connections. With these networkconnections, migration can be performed between different cloud platforms, fromon-premises databases to cloud databases, or on cloud databases across regions.

DRS supports incremental migration, which ensures service continuity whileminimizing the impact of service downtime and migration. Databases can therebybe smoothly migrated to the cloud, and all database objects can be migrated.

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 3

Page 7: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Figure 1-2 Online migration

Backup MigrationIt often becomes necessary to hide the real IP address of your database for thesake of security. Migrating data through direct connections is an option, but costly.DRS supports backup migration, which allows you to export data from your sourcedatabase for backup and upload the backup files to OBS. Then, you can restorethe backup files to the destination database to complete the migration. Using thismethod, data migration can be realized without exposing your source databases.

This method applies to on-premise to cloud migration scenario.

Without connecting to your sources databases, DRS can help you complete datamigration.

Figure 1-3 Backup migration

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 4

Page 8: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Data SynchronizationData synchronization refers to the real-time flow of key service data from onesource to another while consistency of data can be ensured.

It is different from data migration. Migration means moving your overall databasefrom one platform to another. Synchronization refers to the continuous flow ofdata between different services.

It can be used in many scenarios such as real-time analysis, report system, anddata warehouse environment.

Data synchronization focuses on tables and data. It can meet variousrequirements, such as many-to-one, one-to-many synchronization, dynamicaddition and deletion of tables, and synchronization between tables with differentnames.

Figure 1-4 Many-to-one data synchronization

Data SubscriptionData subscription helps you obtain incremental information of key services in adatabase. This type of information is often required by downstream services. Withdata subscription, incremental information is cached. Therefore, downstreamservices can use a unified SDK API provided by DRS to subscribe to, obtain, andconsume this information. In this way, the database and downstream systems aredecoupled.

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 5

Page 9: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Data subscription can be used by Kafka to subscribe to MySQL incremental data.

Figure 1-5 Data subscription

Multi-Active Disaster RecoveryTo prevent service unavailability caused by regional faults, DRS provides disasterrecovery to ensure service continuity. You can easily implement disaster recoverybetween on-premises and cloud, without the need to invest a lot in infrastructurein advance.

The disaster recovery architectures, such as two-site three-data-center and two-site four-data center, are supported. A primary/standby switchover can beimplemented by promoting a standby node or demoting a primary node in thedisaster recovery scenario.

1.3 Which Databases Can DRS Migrate?For details about the supported database versions, see Functions and Features.

1.4 Can DRS Migrate Data to Primary/Standby RDS DBInstances?

Yes. With the high availability mechanism, DRS can migrate data to single orprimary/standby RDS DB instances. For migration of primary/standby RDS DBinstances, DRS automatically rebuilds the databases connection after a shortinterruption. In addition, DRS provides a resumable data transfer feature to ensurethe continuity and consistency of data synchronization based on the point wherethe transfer is interrupted.

If the high availability (HA) design of the source database suits the requirementsof floating IP address connections and RPO is 0 during a switchover, DRS supportsmigration of primary/standby DB instances without manual intervention.

If the HA design does not meet the requirements of floating IP addressconnections and RPO is 0 during a switchover, the following situations may occur:● The floating IP address is used and RPO may be 0 during a switchover. In this

situation, the database can be connected, but DRS will identify datainterruption (if data loss occurs during the switchover) and display a messageindicating that the task fails. You can only reset the migration task.

● A fixed IP address is used and RPO is 0 during the switchover. In this situation,the migration is supported only when the instance is running properly.

● The floating IP address is used and zero RPO cannot be ensured during aswitchover. In this situation, the database can be connected, but DRS will

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 6

Page 10: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

identify data interruption (if data loss occurs during the switchover) anddisplay a message indicating that the task fails. You can only reset themigration task.

If the destination database is primary/standby DB instances, DRS can ensure thatthe source data is completely migrated to the destination database. However, theswitchover of the destination database cannot ensure zero RPO. As a result, datain the destination database may be interrupted.

1.5 What Constraints Does DRS Have for a SourceDatabase?

● For details about online migration constraints, see Before You Start.● For details about backup migration constraints, see Before You Start.● For details about data synchronization constraints, see Before You Start.● For details about data subscription constraints, see Before You Start.

1.6 What Requirements Does DRS Have for aDestination Database?

When you create a MySQL migration task, the destination database user musthave the following permissions:

● SELECT● CREATE● DROP● DELETE● INSERT● UPDATE● INDEX● EVENT● CREATE VIEW● CREATE ROUTINE● TRIGGER● WITH GRANT OPTION

When you create a Microsoft SQL Server migration task, the destination databaseuser must have the following permissions:

● PROCESSADMIN ROLE● SETUPADMIN ROLE

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 7

Page 11: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

1.7 Does DRS Support Migration Between the PublicCloud and the Dedicated Cloud?

DRS supports database migration between the DeC and public cloud. The dataflow from the public cloud to the dedicated cloud is outbound, and from thededicated cloud to the public cloud is inbound. For details about the supportedmigration scenarios, see Online Migration. For details about the operation, seeGetting Started.

1.8 Can DB2 Be Migrated to PostgreSQL Using DRS?DRS does not support the migration from DB2 to PostgreSQL. For details, seeFunctions and Features.

1.9 Can Microsoft SQL Server Database SynchronizeData with Local Databases in Real Time?

The Microsoft SQL Server database does not support real-time datasynchronization with the local database. If data is migrated to the cloud, you canuse the backup migration mode. Backup migration supports incremental migrationof Microsoft SQL Server databases, which effectively shortens the servicemigration time.

1.10 Does DRS Support Data Replication in a SpecifiedTime Period?

Currently, DRS supports only full and full+incremental migration scenarios.

1.11 Does DRS Support Resumable Uploads?In database migration and synchronization scenarios, if a migration orsynchronization task fails due to unavoidable problems (such as networkfluctuation), DRS records the current parsing and replay point (which is the basisof database internal consistency) and then resumes data transfer from the pointto ensure data integrity.

For incremental migration and synchronization, DRS automatically retries formultiple times. For full migration of MySQL databases, the system automaticallyresumes the migration for three times by default. After the number of automaticretry failures reaches a specified value, the task becomes abnormal. You need toanalyze the cause based on logs and try to rectify the blocking point (for example,the database password is changed). If the environment cannot be restored and therequired logs have been eliminated, you can use the reset the task.

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 8

Page 12: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

1.12 What Is Single-Active/Dual-Active DisasterRecovery?

With the rapid development of information technologies, data and informationplay an increasingly important role in modern enterprises. Loss and damage ofdata will cause inestimable losses to enterprises. How to defend against large-scale disasters has drawn increasing attention. Currently, remote disaster recovery(DR) is the only feasible solution. The backup and restoration of key data is animportant part of the routine operation and maintenance of the system.

The dual-AZ, HA DB instances of RDS can meet the requirements of intra-citydisaster recovery. DRS provides cross-region and cross-cloud DR capabilities,including single-active DR and dual-active DR.

Single-Active DRIn single-active DR mode, one active database and one standby database aredeployed. When a disaster occurs, the DR database functions as the servicedatabase to ensure service continuity. DRS supports active/standby switchover.Before a switchover, services are running properly in the service database and datais synchronized to the DR database in real time. In this case, data cannot bewritten into the DR database. After an active/standby switchover, the DR databasebecomes readable and writable, services can be switched to the DR database, anddata cannot be written to the service database.

Figure 1-6 Single-active DR

Dual-Active DRThe dual-active DR mode is used in scenarios where the two databases work inactive/standby mode and share services. Dual-active DR contains two roles, active

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 9

Page 13: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

database 1 and active database 2. Before performing dual-active DR, you need todetermine the RDS role in the current cloud (region). A complete dual-active DR isperformed in two directions, one in the forward direction and the other in thebackward direction. The two directions must be created in sequence. At thebeginning, active database 1 is readable and writable, and active database 2 isread-only. The backward DR can be started only after the initial data is fullysynchronized from active database 1 to active database 2 in the forward direction.In this case, both active database 1 and active database 2 are readable andwritable, and incremental data is continuously synchronized to active database 2and active database 1 in the forward and backward directions, respectively.

Figure 1-7 Dual-Active DR

Features and constraints:● The dual-active DR architecture has high requirements on the environment.

Before deploying the dual-active DR solution, view Before You Start.● The dual-active DR deployment poses strict requirements on the procedure.

Perform the following steps to ensure that the dual-active DR task can besuccessfully deployed.a. Create a DR task. For details, see Create a DR Task. After the creation is

complete, two subtasks are generated, that is, the forward DR task andreverse DR task. In this case, the reverse DR task is in the configurationstate.

b. When the forward DR task is in the DR state (the reverse task is displayedin the Operation column), configure and start the reverse task.On the Disaster Recovery Management page, select the backward DRtask and click Edit in the Operation column. The Create DisasterRecovery Task page is displayed. Continue to create the backward task.You are advised to perform the verification on the active database 2 andstart the backward task after the expected result is met.

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 10

Page 14: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Figure 1-8 Forward and backward DR task

Data Replication ServiceFAQs 1 Product Consulting

2020-03-31 11

Page 15: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

2 Network and Security

2.1 What Security Protection Policies Does DRS Have?

Network● Uses security groups to ensure that the sources of access are trusted.● Uses SSL channels to encrypt data during transmission.

ManagementUse the Identity and Access Management (IAM) service to manage DRSpermissions.

2.2 What Can I Do If the Network Is DisconnectedDuring the Migration?

If the network is disconnected during the migration, you can view the task statusfirst. If a full or incremental task fails, click Retry in the Operation column.

Full migration

Incremental migration

Full synchronization

Incremental synchronization

2.3 Which Database Accounts Are Required DuringMigration?

MySQLTo ensure your database can be successfully migrated to RDS MySQL DB instanceson the current cloud, DRS automatically creates temporary accounts drsFull and

Data Replication ServiceFAQs 2 Network and Security

2020-03-31 12

Page 16: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

drsIncremental for a destination database when you create a migration task.After the task is complete, DRS automatically deletes the temporary account.

NO TICE

Attempting to delete, rename, or change the passwords or permissions for theseaccounts will cause task errors.

Table 2-1 Temporary accounts created for RDS MySQL

Account ApplicationScenario

Host Description

drsFull Fullmigration

RDS MySQL DBinstance

When you start a fullmigration task, DRS will addthis account to thedestination RDS database fordata migration.

drsIncremental

Incrementalmigration

RDS MySQL DBinstance

When you start anincremental migration task,DRS will add this account tothe destination RDSdatabase for data migration.

2.4 How Can I Set a VPC Security Group to Access to anOutside EIP?

By default, a VPC on the current cloud is isolated from external networks forsecurity reasons. You cannot use an EIP outside a VPC (for example, an EIP ofanother cloud database or an on-premise database) to access DB instances insidethe VPC. However, the replication instance or destination database in a VPC needsto connect to an external EIP to migrate data.

Therefore, you need to add an outbound rule to a security group to allow accessfrom specific external EIPs and ports outside the VPC. The outbound rule allowsthe replication instance EIP to access the destination database EIP. The followingfigure shows how to add an outbound rule.

Data Replication ServiceFAQs 2 Network and Security

2020-03-31 13

Page 17: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Figure 2-1 Adding an outbound rule

Data Replication ServiceFAQs 2 Network and Security

2020-03-31 14

Page 18: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

3 Permissions Management

3.1 How Do I Set an Independent Oracle Account ThatHas the Least Privilege and Uses DRS?

To perform a full migration for an Oracle database, you must grant the CREATESESSION, SELECT ANY TRANSACTION, SELECT ANY TABLE, and the SELECT ANYDICTIONARY permissions to the user. If you need to perform an incrementalmigration, the user must have the log parsing permission. If the destinationdatabase is a PostgreSQL database, the SELECT ANY SEQUENCE permission is alsorequired. This section describes how to set an independent Oracle account thathas the least privilege and uses DRS.

● Full migration

a. Create a user for migration. User1 is used as an example.Example command: CREATE USER User1 IDENTIFIED BY pwd

User1 indicates the username and pwd indicates the password.

b. Run the following statement as user sys or as user who has the DBApermission to grant the required permissions to User1:Example command: GRANT CREATE SESSION, SELECT ANYTRANSACTION, SELECT ANY TABLE, SELECT ANY DICTIONARY TOUser1

● Full+incremental migration

a. Create a user for migration. User1 is used as an example.Example command: CREATE USER User1 IDENTIFIED BY pwd

b. Run the following statement as user sys or as user who has the DBApermission to grant the required permissions to User1:Example command: GRANT CREATE SESSION, SELECT ANYTRANSACTION, SELECT ANY TABLE, SELECT ANY DICTIONARY TOUser1

c. Run the following statement as user sys or as user who has the DBApermission to grant the log parsing permission to User1:

Data Replication ServiceFAQs 3 Permissions Management

2020-03-31 15

Page 19: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

▪ If Oracle version is earlier than 12c, run the following statement:GRANT EXECUTE_CATALOG_ROLE TO User1

▪ If Oracle version is later than 12c, run the following statement:Example command: GRANT EXECUTE_CATALOG_ROLE TO User1Example command: GRANT LOGMINING TO User1

3.2 Which MySQL Permissions Are Required for DRS?DRS has certain permission requirements on accounts during migration,synchronization, and DR. This section describes the permission requirements onthe MySQL engine.

Permission● You must have the login permission of the source and destination database

connection accounts. If you do not have the account, perform the followingoperations to create one. user1 is used as an example.Reference statement: CREATE USER'user1'@'host'IDENTIFIED BY'password'

● The following table uses user1 as an example and lists the permissionsrequired in DRS online migration, data synchronization, and disaster recovery.

Data Replication ServiceFAQs 3 Permissions Management

2020-03-31 16

Page 20: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Table 3-1 Permission requirements and reference statements

FunctionModules

Source/Service Database Destination/DRDatabase

Onlinemigration

Full migration:SELECT, SHOW VIEW, EVENTReference statement: GRANT SELECT,SHOW VIEW, EVENT ON *.* TO'user1';Full+incremental migration:SELECT, SHOW VIEW, EVENT, LOCKTABLES, REPLICATION SLAVE,REPLICATION CLIENTReference statement: GRANT SELECT,SHOW VIEW, EVENT, LOCK TABLES,REPLICATION SLAVE, REPLICATIONCLIENT ON [Databases to bemigrated].* TO 'user1';

Full migration:SELECT, CREATE, ALTER,DROP, DELETE, INSERT,UPDATE, INDEX,EVENT, CREATE VIEW,CREATE ROUTINE,TRIGGER, WITH GRANTOPTIONReference statement:GRANT SELECT,CREATE, ALTER, DROP,DELETE, INSERT,UPDATE, INDEX,EVENT, CREATE VIEW,CREATE ROUTINE,TRIGGER ON *.* TO'user1' WITH GRANTOPTION;Full+incrementalmigration:SELECT, CREATE, ALTER,DROP, DELETE, INSERT,UPDATE, INDEX,EVENT, CREATE VIEW,CREATE ROUTINE,TRIGGER, WITH GRANTOPTIONReference statement:GRANT SELECT,CREATE, ALTER, DROP,DELETE, INSERT,UPDATE, INDEX,EVENT, CREATE VIEW,CREATE ROUTINE,TRIGGER ON[Databases to bemigrated].* TO 'user1'WITH GRANTOPTION;

Data Replication ServiceFAQs 3 Permissions Management

2020-03-31 17

Page 21: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

FunctionModules

Source/Service Database Destination/DRDatabase

Datasynchronization

SELECT, SHOW VIEW, EVENT, LOCKTABLES, REPLICATION SLAVE,REPLICATION CLIENTReference statement: GRANT SELECT,SHOW VIEW, EVENT, LOCK TABLES,REPLICATION SLAVE, REPLICATIONCLIENT ON [Databases to bemigrated].* TO 'user1';

SELECT, CREATE, DROP,DELETE, INSERT, andUPDATEReference statement:GRANT SELECT,CREATE, DROP, DELETE,INSERT, UPDATE ON[Databases to bemigrated].* TO 'user1';

Multi-activeDR

SELECT, CREATE, DROP, DELETE,INSERT, UPDATE, SHOW VIEW, EVENT,LOCK TABLES, REPLICATION SLAVE,REPLICATION CLIENTReference statement: GRANT SELECT,CREATE, DROP, DELETE, INSERT,UPDATE, SHOW VIEW, EVENT, LOCKTABLES, REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO'user1';

SELECT, CREATE,INSERT, UPDATE,ALTER, DROP, DELETE,SHOW VIEW, EVENT,INDEX, LOCK TABLES,REPLICATION SLAVE,REPLICATION CLIENT,WITH GRANT OPTIONReference statements:GRANT SELECT,CREATE, INSERT,UPDATE, ALTER, DROP,DELETE, SHOW VIEW,EVENT, INDEX, LOCKTABLES, REPLICATIONSLAVE, REPLICATIONCLIENT, WITH GRANTOPTION ON *.* TO'user1'@'%' WITHGRANT OPTION;

Run flush privileges; after executing the preceding reference statements. Make theauthorization take effect.

● Account migration:If the source database is not on the current cloud, the account must have theSELECT permission of mysql.user. If the source database is on the currentcloud, the account must have the SELECT permission of mysql.user andmysql.user_view.Reference statement:GRANT SELECT ON mysql.user.* TO 'user1';GRANT SELECT ON mysql.user_view.* TO 'user1';The destination database account must have the SELECT, INSERT, UPDATE,DELETE, and WITH GRANT OPTION permissions of the MySQL database.

Data Replication ServiceFAQs 3 Permissions Management

2020-03-31 18

Page 22: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Reference statement: GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO'u1' WITH GRANT OPTION

Actions● Create a user.

Operation:CREATE USER 'username'@'host' IDENTIFIED BY 'password';· username: indicates the account to be created.· host: indicates the host that allows the account to log in. If the account isallowed to log in to the database from any host, use%.· password: indicates the password of the account.For example, run the following command to grant the drsmigration accountwith all permissions on all databases and tables and allow the drsmigrationaccount to log in to the database from any host:CREATE USER 'drsmigration'@'%' IDENTIFIED BY 'Drs123456';

● Grant corresponding permissions.Operation:GRANT privileges ON databasename.tablename TO 'username'@'host' WITHGRANT OPTION;flush privileges;· privileges: indicates the operation permissions granted to the account, suchas SELECT, INSERT, and UPDATE. To grant all permissions to the account, useALL.· databasename: indicates the database name. To grant the account with alldatabase operation permissions, use *.· tablename: indicates table name. To grant the account with all tableoperation permissions, use *.· username: indicates the account to be authorized.· host: indicates the host that allows the account to log in. If the account isallowed to log in from any host, use%.· WITH GRANT OPTION: indicates that the permission to use the GRANTcommand is granted to the account. This parameter is optional.For example, run the following command to create an account drsmigrationwith the password Drs123456 and allow the account to log in to the databasefrom any host:GRANT ALL ON *.* TO 'drsmigration'@'%';

3.3 How Can I Import Users and Permissions from theSource to the Destination Database?

Step 1 Log in to an ECS that can access the source database.

Step 2 Run the following command, enter the password as prompted, and press Enter toexport the source database users to the users.sql temporary file:

Data Replication ServiceFAQs 3 Permissions Management

2020-03-31 19

Page 23: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

mysql -h 'host' -u 'user' -p -N $@ -e "SELECT CONCAT('SHOW GRANTS FOR ''',user, '''@''', host, ''';') AS query FROM mysql.user" > /tmp/users.sql

host indicates the IP address of the source database and user indicates theusername of the source database.

Step 3 Run the following command to export the authorization information of the usersfrom the source database to the grants.sql file:

mysql -h 'host' -u 'user' -p -N $@ -e "source /tmp/users.sql" > /tmp/grants.sql

sed -i 's/$/;/g' /tmp/grants.sql

host indicates the IP address of the source database and user indicates theusername of the source database.

Step 4 After the preceding command has been executed successfully, open the grants.sqlfile. Information similar to the following is displayed:-- Grants for root@%GRANT ALL PRIVILEGES ON *.* TO 'root'@'%';

-- Grants for testt@%GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO 'testt'@'%';

-- Grants for debian-sys-maint@localhost GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' WITH GRANT OPTION;

-- Grants for mysql.session@localhostGRANT SUPER ON *.* TO 'mysql.session'@'localhost'; GRANT SELECT ON `performance_schema`.* TO 'mysql.session'@'localhost';GRANT SELECT ON `mysql`.`user` TO 'mysql.session'@'localhost';

-- Grants for mysql.sys@localhostGRANT USAGE ON *.* TO 'mysql.sys'@'localhost'; GRANT TRIGGER ON `sys`.* TO 'mysql.sys'@'localhost'; GRANT SELECT ON `sys`.`sys_config` TO 'mysql.sys'@'localhost';

-- Grants for root@localhostGRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION; GRANT PROXY ON ''@'' TO 'root'@'localhost' WITH GRANT OPTION;

Step 5 The information displayed in Step 4 shows all users of the source database andtheir permissions. Add the required users one by one to the RDS MySQL databaseon the current cloud. For details, see the Creating an Account section in theRelational Database Service User Guide.

----End

Data Replication ServiceFAQs 3 Permissions Management

2020-03-31 20

Page 24: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

4 Data Migration

4.1 When Can I Stop a Migration Task?After service cutover is successfully completed, you can stop the migration task toprevent operations in the source database from being synchronized to thedestination database to overwrite the data. Before stopping the task, ensure that:

1. At least one complete data comparison is performed during off-peak hours.

2. Service cutover is completed.

a. Interrupt services first. If the workload is not heavy, you may do not needto interrupt the services.

b. Run the following statement on the source database (MySQL is used asan example. For other DB engines, see DRS Best Practices). If no SQLstatement is executed within 1 to 5 minutes, the service is stopped.show processlist

The process list queried by the preceding statement includes the connection ofthe DRS replication instance. If no additional session executes SQL statements,the service has been stopped.

c. When the delay is 0s and remains stable for a period, you can perform adata-level comparison between the source and destination databases. Fordetails about the time required, refer to the comparison results of theprevious comparison.

▪ If there is enough time, compare all objects.

▪ If there is not enough time, use the data-level comparison tocompare the tables that are frequently used and that contain keybusiness data or inconsistent data.

d. Determine a proper time to cut the services over to the destinationdatabase. Then, services can be used externally again.

3. Stopping a task only deletes the replication instance, and the migration task isstill in the task list. You can choose whether or not to delete the task.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 21

Page 25: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

4.2 How Do I Maintain the Original Service UserPermission System After Definer Is Forcibly ConvertedDuring MySQL Migration?

Definer is used in views, stored procedures, triggers, and events. Definer does notrestrict the permission to invoke objects, instead the permission to access thedatabase. If you select Yes for Migrate Definer to User during MySQL migration,the Definers of all source database objects will be migrated to the user. The usercontinues to use the original services without authorization. (Users, permissions,and passwords are migrated). Other users do not have permissions on databaseobjects unless these users are authorized.

The following procedures describe how to use database commands to authorizeusers.

Step 1 Ensure that the new user (Definer uses the specified account) has sufficientpermission to execute view- and stored procedure-related SQL statements.

Step 2 Log in to the destination database using the MySQL official client or other tools.

Step 3 Run the following command to view details about permissions of the user to beauthorized:show grants for 'user'@'host';

Step 4 To ensure that the original service does not report an error, run the followingcommand to grant the user the operation permissions the involved databaseobjects do not have:grant select,insert,update,delete on db_name.* to 'user'@'host';

Generally, the permissions to access the database are as follows: SELECT, CREATE,DROP, DELETE, INSERT, UPDATE, INDEX, EVENT, CREATE VIEW, CREATE ROUTINE,TRIGGER, and EXECUTE. You need to check the permissions that are missing basedon the database object, and then perform the authorization operation.

For stored procedures and functions, ensure that the user has the EXECUTEpermission. The authorization command is as follows:

grant execute on db_name.function_name to 'user'@'host';

Step 5 Use the authorized account to access the destination database. If the access issuccessful, the authorization is successful. Note: If the following information isdisplayed when a stored procedure or function is invoked in a Java project, themysql.proc database must be authorized: Java.sql.SQLException: User does nothave access to metadata required to determine stored procedure parameter types.If rights can not be granted, configure connection with"noAccessToProcedureBodies=true" to have driver generate parameters thatrepresent INOUT strings irregardless of actual parametertypesgrant select on mysql.proc to 'user'@'host';

----End

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 22

Page 26: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

4.3 What Can I Do If the Invoking Permission ProblemOccurs After the MySQL Stored Procedure Is Migratedto the Cloud?

After the MySQL stored procedure is migrated to the cloud, an error may occurwhen the stored procedure or function is invoked due to permission problems.

The method varies with Definer policies. This section uses user1 as an example todescribe how to solve this problem in two Definer policies.

Policy 1On the Destination Database page, enter the database username user1, andselect OK for Migrate Definer to User.

Figure 4-1 Policy 1

In this policy, after the Definers of all stored procedures and methods in the sourcedatabase are migrated to the destination database, the account is automaticallychanged to user1, and the value of host is automatically changed to %. If a storedprocedure fails to be invoked in the destination database, perform the followingoperations:

Step 1 Log in to the RDS MySQL DB instance of the destination database as the user1.

Step 2 Grant the execute permission to the account that you want to use to invoke astored procedure.

Step 3 Run the following statement to use user1 to grant other accounts the permissionto execute stored procedures:

user indicates other accounts that need to invoke the stored procedure.GRANT EXECUTE ON db.* TO user;

Step 4 To invoke a stored procedure using Java, run the following statement to use user1to grant other accounts the permission to query the mysql.proc table:

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 23

Page 27: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

The following is the authorization statement, in which user indicates the accountthat needs to invoke the stored procedure:GRANT SELECT ON mysql.proc TO 'user'@'%';

----End

Policy 2On the Destination Database page, enter the database username user1, andselect Cancel for Migrate Definer to User.

Figure 4-2 Policy 2

In this policy, the account and host in the source database remain unchangedafter the Definers of all stored procedures and methods are migrated to thedestination database. You need to migrate all users in the source database byreferring to Migrating Accounts. In this way, the permission system of the sourcedatabase remains unchanged.

If you do not migrate account permissions or some accounts cannot be migrated,you are advised to use Policy 1.

4.4 How Do I Ensure that All Services on the DatabaseAre Stopped?

To ensure that all services on the database are stopped, perform the followingsteps:

Step 1 Run the following statement on the source database to check whether activeconnections exist:show processlist

Figure 4-3 Checking active connections

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 24

Page 28: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Step 2 Optional: If there are active connections, locate the service processes based onthe values in the Host column in the command output and stop the serviceprocesses.

Step 3 Run the following statement in the source database to check the binlog position.Then, record the two values in the file and position columns as ckpt1:show master status

Figure 4-4 Viewing the binlog position

Step 4 Wait for more than 30s. Run the following statement in the source database tocheck the binlog position again. Then, record the two values in the file andposition columns as ckpt2. If ckpt1 and ckpt2 are equal, no data is written to thesource database.show master status

----End

4.5 What Can I Do When Scheduled Task Failed toStart Leaving Message "can not get agency token"

When you use a subaccount to use scheduled task startup function, the accountentrustment function must be used. Otherwise, the scheduled task fails to bestarted, leaving message "can not get agency token".

SolutionTwo solutions are provided as follows:

● Method 1: Use your primary account to create a task and select Start at aspecified time for Start Time.

● Method 2: Use the primary account to add the security administratorpermission to the user group to which the subaccount belongs. Then, create atask again, and select Start at a specified time for Start Time.

● Method 3: Create a task again and select Start upon task creation for StartTime.

4.6 What Do I Do If the Maximum Index Length HasBeen Reached During Migration from Oracle toMySQL?

Index LengthThe maximum length of each MySQL index is limited, which depends on the typeof DB engine and character set. For example, each utf8 character set contains a

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 25

Page 29: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

maximum of three bytes, and each utf8mb4 character set contains a maximum of4 bytes.● For a column index, the length of each column cannot exceed the value given

in Max. Characters of a Column Index in Table 4-1. For example, in MySQL5.7.6 that uses the InnoDB storage engine, the length of a column indexcannot exceed 767 bytes (Max. Characters of a Column Index = 767 / Max.Bytes).

● For a multiple-column index, the length of each column cannot exceed thevalue given in Max. Characters of a Column Index in Table 4-1, and thetotal length of columns cannot exceed the value given in Max. Characters ofa Multiple-Column Index. For example, in MySQL 5.7.6 that uses the InnoDBstorage engine, each column index length cannot exceed 767 bytes (Max.Characters of a Column Index = 767 / Max. Bytes), and the total column indexlength cannot exceed 3072 bytes (Max. Characters of a Column Index =3072 / Max. Bytes)

Table 4-1 Index length description

StorageEngine

MySQLVersion

CharacterSet

Max. Bytes Max.Charactersof a ColumnIndex

Max.Charactersof aMultiple-ColumnIndex

InnoDB MySQL5.7.6 orearlier

utf8mb4 4 191 768

MySQL5.7.7 orlater

utf8mb4 4 768 768

Solution● Method 1

Do not migrate tables that contain indexes of which the maximum length hasbeen reached.

● Method 2Changes to index length may cause data consistency problems. Exercisecaution when performing this operation. For example, if the destinationdatabase version is MySQL 5.7.6 or earlier and uses utf8mb4 character set,run the following command to change the index length:alter table tablename modify columnname varchar2 (768) ;

Replace tablename with the actual table name and columnname with the actualcolumn name.

● Method 3Delete the index and its constraints from the source database. For example, ifthe destination database version is MySQL 5.7.6 or earlier and uses utf8mb4

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 26

Page 30: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

character set, run the following commands to delete the index and itsconstraints:drop index indexname;alter table tablename drop constraint constraintname;

Replace indexname with the actual index name, tablename with the actualtable name, and constraintname with the actual constraint name.

4.7 Why Is the Collation of Heterogeneous or OracleDatabases Converted to utf8mb4_bin After ThoseDatabases Are Migrated to MySQL?

Different databases support different types of character sets. After databases suchas Oracle databases are migrated to the MySQL database, their encoding will beconverted to the utf8mb4 character set to support more bytes per character. Thedefault collation of utf8mb4 is utf8_general_ci, which is case insensitive. That is,abc and ABC are the same data. The migration from case-sensitive databases suchas Oracle databases to MySQL databases may cause migration failures (primarykey conflicts) or have impact on services (incorrect query results). You can refer tothe following suggestions:

1. When DRS is used to perform heterogeneous migration or synchronization toMySQL, DRS automatically sorts the character set of the database (only thedatabase to be migrated) with the utf8mb4_bin collation. This ensures thatcollation settings of new tables and columns in the same database are the sameas those of the migrated tables and columns, facilitating associated queries andindex queries.

2. You can set collation_server to utf8mb4_bin at the instance level. This settingensures that the default character set of all instances is utf8mb4_bin, you can setthis parameter based on service requirements.

4.8 What Can I Do If MyISAM Tables Are NotSupported by RDS for MySQL?

Currently, RDS for MySQL does not support the MyISAM engine due to thefollowing reasons.

● MyISAM engine tables do not support transactions and support only table-level locks. As a result, read and write operations conflict with each other.

● MyISAM has a defect in protecting data integrity, which may cause databasedata damage or even data loss.

● If data is damaged, MyISAM does not support data restoration provided byRDS for MySQL and requires manual restoration.

● Data can be transparently migrated from MyISAM to InnoDB, which does notrequire code modification for tables.

During migration, DRS automatically converts MySIAM to InnoDB. The MyISAMengine table does not support transactions. To ensure data consistency of theMyISAM table, DRS uses primary keys to ensure final data consistency. If you need

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 27

Page 31: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

to migrate MyISAM tables without primary keys, you are advised to start themigration task when no service is running to ensure data consistency.

4.9 What Are the Precautions for Migrating Data froman Earlier Version MySQL to MySQL 8.0?

Based on MySQL 5.7, some new features have been added to MySQL 8.0. Thereare performance differences between the two versions. Before migration, you needto analyze compatibility and provide a corresponding solution. The followingshows the analysis:

● Compatibility analysisMySQL 8.0 and MySQL 5.7 Community Edition are analyzed as follows:

a. Compatibility does not affect migration, but the solutions are different.

Compatibility

Check Item Function

Status Solution

Datatypesorfunctions

ENCODE() Encryption

Deleted Replaced byAES_ENCRYPT()

DECODE() Decryption

Deleted Replaced byAES_DECRYPT()

ENCRYPT() Encryption

Deleted Replaced by SHA2()

DES_ENCRYPT()

Encryption

Deleted Replaced byAES_ENCRYPT()

DES_DECRYPT()

Decryption

Deleted Replaced byAES_DECRYPT()

JSON_APPEND()

AddsJSONelements.

Deleted Replaced byJSON_ARRAY_APPEND()

PASSWORD() Changes a userpassword.

Deleted ALTER USER userIDENTIFIED BY'auth_string';

JSON_MERGE()

MergesmultipleJSONs.

Discarded

Replaced byJSON_MERGE_PERSERVE()

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 28

Page 32: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

SQLMODE

NO_AUTO_CREATE_USER,DB2, MAXDB,MSSQL,MYSQL323,MYSQL40,ORACLE,POSTGRESQL,NO_FIELD_OPTIONS,NO_KEY_OPTIONS,NO_TABLE_OPTIONS

- Deleted -

Foreign keyconstraintlength

The constraintname cannotbe greaterthan 64characters.

- - SELECTTABLE_SCHEMA,TABLE_NAME FROMINFORMATION_SCHEMA.TABLES WHERETABLE_NAME IN(SELECTLEFT(SUBSTR(ID,INSTR(ID,'/')+1),INSTR(SUBSTR(ID,INSTR(ID,'/')+1),'_ibfk_')-1)FROMINFORMATION_SCHEMA.INNODB_SYS_FOREIGN WHERELENGTH(SUBSTR(ID,INSTR(ID,'/')+1))>64);Use the ALTER TABLEstatement to adjust thelength.

Features

Use theGRANTstatement tocreate users.

- Deleted CREATE USER

Use theGRANTstatement tomodify userinformation.

- Deleted ALTER USER

IDENTIFIED BYPASSWORD'auth_string'

Setsnewpasswords

Deleted IDENTIFIED WITHauth_plugin AS'auth_string'

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 29

Page 33: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

\N in a SQLstatement

NULL Deleted Replaced by NULL

PROCEDUREANALYSE()syntax

Specifies therecommendedfieldtype isprovided aftertheMySQLfieldvalue isanalyzed.

Deleted -

Spatialfunctions

- - -

mysql_install_db

Initialization

Deleted mysqld --initialize or --initialize-insecure

b. The following items affect the migration. You need to check in advance.

Compatibility

Check Item Function

Status Solution OriginalUsage

Reservingkeywords

cume_dist,dense_rank,empty, rank,first_value,grouping,groups,json_table,lag,last_value,lateral, lead,nth_value,ntile, of, over,percent_rank,rank, andrecurs ive,row_number,system, andwindow

- Added SETsql_mode='ANSI_QUOTES'

Name:database,table,index,column,alias, view,storedprocedure,partition,andtablespace

Character set

utf8mb3 - Discarded

Replacedbyutf8mb4.

-

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 30

Page 34: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Partition tablename

Partitiontables ofstorageengines thatdo notsupport localpartitions arenot allowed.

- Deleted SELECTTABLE_SCHEMA,TABLE_NAME FROMINFORMATION_SCHEMA.TABLESWHEREENGINENOT IN('innodb','ndbcluster') ANDCREATE_OPTIONSLIKE'%partitioned%';You canuse eitherof thefollowingmethods:(1) ALTERTABLEtable_nameENGINE=INNODB;(2) ALTERTABLEtable_nameREMOVEPARTITIONING;

MyISAM isnotsupported.

Syntax group by...asc/desc

Ascending/Descending

Deleted Replacedby theORDER Byclause.

View andfunction

Namelength

The viewname cannotbe greaterthan 64characters.

- - ALTER The valuecancontain amaximumof 255characters.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 31

Page 35: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

The enum orset elementcontains amaximum of255characters.

- - Handledby users.

The valuecancontain amaximumof 64 KB.

Upperandlowercaseletters

lower_case_table_names

Specifieswhether ornot toset theMySQLtablenamecasesensitive.

- If thisparameteris set to 1during theupgrade,ensurethat theschemaand tablenames areinlowercase.SELECTTABLE_NAME FROMINFORMATION_SCHEMA.TABLESWHERETABLE_NAME !=LOWER(TABLE_NAME) ANDTABLE_TYPE ='BASETABLE';SELECTSCHEMA_NAMEFROMINFORMATION_SCHEMA.SCHEMATAWHERESCHEMA_NAME !=LOWER(SCHEMA_NAME);

-

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 32

Page 36: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Triggers Checkwhether thereis an emptydefinition orinvalidcreationcontext.

- - Use theSHOWTRIGGERSstatementto checkthecharacter_set_client,collation_connection, andDatabaseCollationattributes.

-

● Change the default value of the system variable.

The analysis of default values of MySQL 5.7 and MySQL 8.0 CommunityEdition shows that default values do not affect the migration but affectservices after the migration.

No.

Parameter/Option

Community Function Remarks

OriginalDefaultValue

NewDefaultValue

Server

1 character_set_server

latin1 utf8mb4 - Beconsistentwith theorigindefaultvalue.

2 collation_server latin1_swedish_ci

utf8mb4_0900_ai_ci

- Beconsistentwith theorigindefaultvalue.

3 explicit_defaults_for_timestamp

OFF ON Specifieswhether toupdate thetimestampcolumnwhen arow isupdated.

Beconsistentwith theorigindefaultvalue.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 33

Page 37: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

4 optimizer_trace_max_mem_size

16KB 1MB - Beconsistentwith theorigindefaultvalue.

5 validate_password_check_user_name

OFF ON - Beconsistentwith theorigindefaultvalue.

6 back_log -1(autosize)changedfrom :back_log =50 +(max_connections / 5)

-1(autosize)changedto :back_log =max_connections

Specifiesthenumber ofrequeststhat canbe storedin thestack in ashortperiodbefore theMySQLdatabasestopsrespondingto newrequests.

Beconsistentwith theorigindefaultvalue.

7 max_allowed_packet

4194304(4MB)

67108864(64MB)

Limits thesize ofdatapacketsreceivedby theserver

Use thedefaultvalue.

8 max_error_count

64 1024 Controlsthenumber ofalarms tobedisplayed.

Beconsistentwith theorigindefaultvalue.

9 event_scheduler

OFF ON - Beconsistentwith theorigindefaultvalue.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 34

Page 38: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

10 table_open_cache

2000 4000 - Beconsistentwith theorigindefaultvalue.

11 log_error_verbosity

3 (Notes) 2(Warning)

- Use thedefaultvalue.

INNODB

1 innodb_undo_tablespaces

0 2 - Use thedefaultvalue.

2 innodb_undo_log_truncate

OFF ON - Use thedefaultvalue.

3 innodb_flush_method

NULL fsync(Unix),unbuffered(Windows)

Controlstheenablingandwritingmodes ofInnoDBdata filesand redologs.

Use thedefaultvalueO_DIRECTforHWSQL.

4 innodb_autoinc_lock_mode

1(consecutive)

2(interleaved)

Controlsthebehaviorof relatedlocks whendata isinsertedinto atable withtheauto_incrementcolumn.

Beconsistentwith theorigindefaultvalue.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 35

Page 39: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

5 innodb_flush_neighbors

1 (enable) 0 (disable) Checkswhetherother dirtypages inthe samerange arerefreshedwhenrefreshingthe pagefrom thebufferpool.

Beconsistentwith theorigindefaultvalue.

6 innodb_max_dirty_pages_pct_lwm

0 (%) 10 (%) Affects theInnoDBdirty pagerefreshingoperation.

Use thedefaultvalue.

7 innodb_max_dirty_pages_pct

75 (%) 90 (%) Affects theInnoDBdirty pagerefreshingoperation.

Use thedefaultvalue.

PERFORMANCESCHEMA

Enabledglobally.

- - - Beconsistentwith theorigindefaultvalue.

REPLICATION

1 log_bin OFF ON - Enabled bydefault

2 server_id 0 1 - If thevalue is 0,change itto 1.

3 log-slave-updates

OFF ON - Enabled bydefault.

4 expire_log_days 0 30 - Use thedefaultvalue.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 36

Page 40: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

5 master-info-repository

FILE TABLE - Use thedefaultvalueTABLE.

6 relay-log-info-repository

FILE TABLE - Use thedefaultvalueTABLE.

7 transaction-write-set-extraction

OFF XXHASH64 - Use thedefaultvalue.

8 slave_rows_search_algorithms

INDEX_SCAN,TABLE_SCAN

INDEX_SCAN,HASH_SCAN

- Use thedefaultvalue.

● Remove system variables.

The analysis of MySQL 5.7 and 8.0 Community Edition shows that removingsystem variables does not affect migration.

System variables

innodb_locks_unsafe_for_binlog

log_builtin_as_identified_by_password

old_passwords

query_cache_limit

query_cache_min_res_unit

query_cache_size

query_cache_type

query_cache_wlock_invalidate

ndb_cache_check_time

ignore_db_dirs

tx_isolation

tx_read_only

sync_frm

secure_auth

multi_range_count

log_error_verbosity

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 37

Page 41: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

sql_log_bin

metadata_locks_cache_size

metadata_locks_hash_instances

date_format

datetime_format

time_format

max_tmp_tables

ignore_builtin_innodb

innodb_support_xa

innodb_undo_logs

innodb_undo_tablespaces

internal_tmp_disk_storage_engine

4.10 What Can I Do When OOM Occurs During theMigration of MongoDB Databases?

ScenariosOut of memory (OOM) occurs during the migration of MongoDB databases,causing migration failures.

Possible CauseThe possible causes are as follows:

● If the mongod service of the source database is deployed on a single server,OOM occurs when the migration process consumes large amounts of memorythrough operations such as creating indexes and sorting queries.

● If the monogod service is deployed on a server with other services and thecacheSizeGB value is not specified, OOM occurs when all available memoryhas been allocated to other services, so the WiredTiger engine does not havesufficient memory.

By default, the memory used by the WiredTiger engine of mongod is calculated asfollows: (Memory in GB-1)x50% for version 3.2 or (Memory in GB-1)x50% for version3.4 and later.

Solution● If the mongod service is deployed on a single server, do not perform any

operations that consume large amounts of memory during the migration.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 38

Page 42: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

● If the mongod service and other services are deployed on the same server, setthe value of cacheSizeGB to the half of the minimal idle memory to ensurethat memory used in peak hours will not be allocated to WiredTigerexcessively.

4.11 Which Plug-ins Can Be Migrated DuringPostgreSQL Database Migration?

Currently, DRS supports the migration of plug-ins in the whitelist. Plug-ins not inthe whitelist may have security risks and cannot be migrated.

If you need the support for plug-ins not in the whitelist, contact technical supportengineers to for processing.

For details about the supported plug-ins, see Table 4-2.

Table 4-2 Plug-ins information

Name Description

postgis When postgis is created, the following pluginsare created at the same time:postgispostgis_topologyfuzzystrmatchpostgis_tiger_geocoderaddress_standardizeraddress_standardizer_data_us

btree_gin -

btree_gist -

hstore -

pg_trgm -

tablefunc -

unaccent -

uuid-ossp -

cube -

dict_int -

dict_xsyn -

earthdistance -

intagg -

intarray -

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 39

Page 43: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Name Description

ltree -

pgcrypto -

timescaledb The timescaledb plugin does not supportPostgreSQL 9.5.5.

hll -

zhparser Provides the default dictionary only and doesnot support user-defined dictionaries. Thedefault values are as follows:zhparser.punctuation_ignore = offzhparser.punctuation_ignore = offzhparser.multi_short = offzhparser.multi_duality = offzhparser.multi_zmain = offzhparser.multi_zall = offzhparser.dict_in_memory = off

oracle_fdw, Supports PostgreSQL 10.3 to connect to Oracle12c or earlier versions only.

pg_pathman -

pg_stat_statements -

pg_hint_plan -

pg_jieba -

4.12 How Do I Disable the Balancer?Before using the DRS service to migrate collections between sharded clusters, youmust disable the balancer of the collections to be migrated.

After the migration is complete, enable the balancer. The balancer is disabled during themigration, generating different numbers of chunks on each shard of the source database.After the balancer is enabled, chunks will be distributed between shards in the cluster,which may affect the performance of the source database.

Procedure

Step 1 Log in to a database through mongo shell.

Step 2 Run the following command in the command window of the mongos node toswitch to the config database:

use config

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 40

Page 44: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Step 3 Run the following commands to check whether the balancer can be disabled:while( sh.isBalancerRunning() ) { print("waiting..."); sleep(1000);}

● If the command output is waiting, the balancer is migrating chunks. In thiscase, do not disable the balancer. Otherwise, data inconsistency may occur.

Figure 4-5 Viewing the command output

● If no command output is displayed, the balancer is not migrating any chunks.In this case, you can disable the balancer:

Step 4 Disable the balancer.● If you migrate the entire DB instance, run the following command to disable

the balancer.sh.stopBalancer()

● If you need to disable the balancer of the sharded collections to be migrated,run the following command:sh.disableBalancing("database.collection")

database.collection indicates the namespace of the collection to bemigrated.

----End

4.13 How Do I Export and Import Events and Triggersin Batches?

During the MySQL to MySQL migration, if the migration log indicates that themigration of events and triggers fails after the migration task is complete, you canmanually migrate the events and triggers.

This section describes how to export and import events and triggers in batches.

Step 1 Export triggers from the source database in batches.

1. Run the following statement in the source database to obtain values ofTRIGGER_SCHEMA and TRIGGER_NAME:SELECT TRIGGER_SCHEMA,TRIGGER_NAME FROM INFORMATION_SCHEMA.TRIGGERS WHERE TRIGGER_SCHEMA in ('DB1','DB2','DB3') order by TRIGGER_NAME;

In the preceding statements, DB1, DB2, and DB3 indicate the databases to bemigrated to the destination database.

2. Run the following statement in the source database to obtain the statementfor creating a trigger from the source database from the SQL OriginalStatement field:SHOW CREATE TRIGGER TRIGGER_SCHEMA.TRIGGER_NAME \G;

In the preceding statement, replace TRIGGER_SCHEMA and TRIGGER_NAMEwith the values obtained in Step 1.1.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 41

Page 45: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Step 2 Export events from the source database in batches.

1. Run the following statement in the source database to obtain values ofEVENT_SCHEMA and EVENT_NAME:SELECT EVENT_SCHEMA,EVENT_NAME FROM INFORMATION_SCHEMA.EVENTS WHERE EVENT_SCHEMA in ('DB1','DB2','DB3') order by EVENT_NAME;

In the preceding statements, DB1, DB2, and DB3 indicate the databases to bemigrated to the destination database.

2. Run the following statement in the source database to obtain the statementfor creating an event from the source database from the SQL OriginalStatement field:SHOW CREATE EVENT EVENT_SCHEMA.EVENT_NAME \G;

In the preceding statement, replace EVENT_SCHEMA and EVENT_NAME withthe values obtained in Step 2.1.

Step 3 Import triggers and events.

Execute the statements for creating triggers and events exported from the sourcedatabase in the destination database.

----End

4.14 How Can I Migrate Databases or Tables WhoseNames Contain Uppercase Letters?

ScenariosWhen the value of source database parameter lower_case_table_names is set to1, the databases or tables whose names contain uppercase letters cannot bemigrated.

Possible CauseWhen the value of lower_case_table_names in the source database is 1, theMySQL engine converts the database name or table name into lowercase letters.In this case, the database or table may not be found, resulting in query failure.Simply, if the value of lower_case_table_names is 1, the database or tablecontaining uppercase letters may be inaccessible.

SolutionsTwo solutions are provided as follows:

Solution 1Change the value of lower_case_table_names in the source database to 0 (case-sensitive) and ensure that the value of this parameter in the source database isthe same as that in the destination database.

Solution 2If the value of lower_case_table_names cannot be changed permanently, changethe value to 0, and then perform the following operations:

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 42

Page 46: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

● For a table, you can use the following statement to convert the table name tolowercase:alter table `BigTab` rename to `bigtab`

● For a database, you need to export the database data, change the databasename from uppercase to lowercase, and then import the data.

After changing the database name or table name, you need to maintain thepermission consistency without affecting application access.

Method 3Do not migrate the databases or tables that contain uppercase letters.

4.15 What Should I Do If the Last Backup File IsIncorrectly Selected in the Backup Migration Scenario?

During the backup migration, If Last Backup File is selected by mistake, performeither of the following operations:

● If you select Yes by mistake, the database receives a signal that the restore iscomplete, and then sets the database to available, making incrementalbackup migration impossible. In this case, you can only delete the backupdatabase and perform full and incremental backup restoration again.

● SQL Server does not have the last backup file in a strict sense. If you selectNo by mistake, you can perform an incremental backup (even if no data ischanged). During the incremental backup, select Yes to complete themigration. The related database becomes available.

Data Replication ServiceFAQs 4 Data Migration

2020-03-31 43

Page 47: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

5 Data Synchronization

5.1 Can DRS Sync Tables of Different Schemas to theSame Schema?

DRS can directly synchronize tables of different schemas to those of the sameschema if the tables do not conflict with each other.

Data Replication ServiceFAQs 5 Data Synchronization

2020-03-31 44

Page 48: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

6 Data Subscription

6.1 How Long Does It Take for SDK to ConsumeDatabase Changes?

Users can obtain database changes within 1s if they have consumed the existingsubscription information.

Data Replication ServiceFAQs 6 Data Subscription

2020-03-31 45

Page 49: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

7 Multi-Active DR

7.1 What Are RPO and RTO of DRS Disaster Recovery?● Recovery Point Object (RPO) refers to the difference between the time when

a transaction in the current service database is submitted and the time whenthe transaction is sent to DRS. Generally, the transaction is the latesttransaction received by DRS. RPO measures the difference between the datain the service database and the data in the DRS instance. When RPO equals 0,all the data in the service database has been migrated to the DRS instance.

● Recovery Time Object (RTO) refers to the time difference between the timewhen a transaction on the current DRS instance is transmitted to the DRinstance and the time when the transaction is successfully executed. (Thistransaction is usually the latest transaction received by DRS.) RTO measuresthe amount of data being transmitted. When RTO equals 0, all transactionson the DRS instance have been completed on the DR database.

7.2 How Do I Select Active Database 1 and 2 for Dual-Active DR?

In dual-active DR mode, at least one of the two DR databases must be an RDS DBinstance on the current cloud, and the other can be an RDS DB instance on the

Data Replication ServiceFAQs 7 Multi-Active DR

2020-03-31 46

Page 50: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

current cloud, other cloud database, self-built database on the ECS, or on-premises database. DRS uses active database 1 and active database 2 todistinguish RDS roles on the current cloud (region). After you determine the roleof RDS on the current cloud, the other role is also determined.

● Active database 1: Generally, service data is on the active database 1. If youselect active database 1 when creating a DR task, initial data is stored in theRDS DB instance.

● Active database 2: The database must be empty. If you select active database2 when creating a DR task, the RDS database on the current cloud is emptyand waits for receiving data.

When creating a DR task, comply with the given principles to select activedatabase 1 and 2 in the following scenarios:

● Both the DR and backup databases are on the RDS DB instances on thecurrent cloud.– If one of the instances is empty, the empty instance functions as the

active database 2, and the non-empty instance functions as the activedatabase 1.

– Both DB instances are empty. You are advised to select active database 2.

● On database is on the RDS DB instance on the current cloud, and the other isa self-built database on the ECS or on-premises database.– One database has initial data, and the other is empty.

▪ If the RDS DB instance on the current cloud is empty, select activedatabase 2.

▪ If the RDS DB instance on current cloud has initial data and theother is empty, select active database 1.

Data Replication ServiceFAQs 7 Multi-Active DR

2020-03-31 47

Page 51: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

– Both databases are empty. You are advised to select active database 2.

7.3 Common Multi-Active DR ScenariosDue to certain uncontrollable reasons, data may be inconsistent when data inboth the databases is changed at the same time in the multi-active DR scenario.This section describes common data exceptions. The dr1 and dr2 databases areused as examples in the following scenarios.

Scenario 1: In dual-active DR mode, operations are performed on the samerow in the two databases at the same time. As a result, multiple datarecords are generated.

● The following figure shows the initial data (seqno is the primary key andcolumn1 is the non-primary key).

Figure 7-1 Initial data in the dr1 and dr2 databases

● Perform the following operations on both databases:– dr1: update dr1 set seqno=5 where column1=8;– dr2: update dr2 set seqno=6 where column1=8;

● After the operations are performed, the data in the databases is consistentbut an additional row is generated.

Figure 7-2 Data in the dr1 and dr2 databases

Scenario 2: In dual-active DR mode, operations are performed on the samerow in both databases at the same time. As a result, data records becomeinconsistent.

● The following figure shows the initial data (seqno is the primary key andcolumn1 is the non-primary key).

Data Replication ServiceFAQs 7 Multi-Active DR

2020-03-31 48

Page 52: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Figure 7-3 Initial data

● Perform the following operations on both databases:– dr1: insert into dr1 values(101, 100);– dr2: insert into dr2 values(101, 102);

● After the operations are performed, the data in the databases is shown in thefollowing figure.

Figure 7-4 Data in the dr1 database

Figure 7-5 Data in the dr2 database

Scenario 3: In dual-active DR mode, DDL operations are performed. As aresult, data records become inconsistent.

● Perform the following operations on both databases:– dr1: truncate table dr1;– dr2: insert into dr2 values(5,5,5);

● After the operations are performed, the data in the databases becomesinconsistent.

Figure 7-6 Data in the dr1 database

Figure 7-7 Data in the dr2 database

Data Replication ServiceFAQs 7 Multi-Active DR

2020-03-31 49

Page 53: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

More scenarios are being added.

Data Replication ServiceFAQs 7 Multi-Active DR

2020-03-31 50

Page 54: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

8 General Operations

8.1 What Can I Do When Information Overlaps on theDRS Console?

Information often overlaps when you decrease the size of the page. You areadvised to set the page scale at 100%.

8.2 How Do I Set Global binlog_format=ROW to TakeEffect Immediately?

During an incremental migration or synchronization for MySQL databases, thesource database binlog must be in the ROW format. Otherwise, the task fails.After binlog_format=ROW at the global level is set in the source database, all theprevious service threads need to be stopped because these threads still connectthe binlog in the non-ROW format.

ProcedureStep 1 Log in to the source database using the MySQL official client or other tools.

Step 2 Run the following command for setting global parameters in the source database.set global binlog_format = ROW;

Step 3 Run the following command on the source database and check whether thepreceding operation is successful:select @@global.binlog_format;

Step 4 You can use either of the following methods to ensure that the modified binlogformat of the source database takes effect immediately:

Method 1

1. Select a non-service period to disconnect all service connections on thecurrent database.

a. Run the following command to query all service threads (excluding allbinlog dump threads and current threads) in the current database:

Data Replication ServiceFAQs 8 General Operations

2020-03-31 51

Page 55: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

show processlist

b. Stop all the service threads queried in the previous step.

Do not create or start a migration task before the preceding operations are complete.Otherwise, data may be inconsistent.

2. To prevent the binlog format of the source database from becoming invaliddue to database restart, add or modify the binlog_format parameter in thestartup configuration file (my.ini or my.cnf) of the source database and savethe modification.binlog_format=ROW

Method 2

1. To prevent the binlog format of the source database from becoming invaliddue to database restart, add or modify the binlog_format parameter in thestartup configuration file (my.ini or my.cnf) of the source database and savethe modification.binlog_format=ROW

2. Ensure that the binlog_format parameter is successfully added or modified.Then, restart the source database at a non-service period.

----End

8.3 How Do I Change the Destination DatabasePassword to Meet the Password Policy?

ScenariosWhen you set the password for the migration account in the destination database,you need to set the password based on the password strength requirements of thedestination database.

ProcedureThe following operations apply to the scenario where the target database is anRDS instance.

Step 1 Log in to the RDS console.

Step 2 Locate the target DB instance.

Step 3 Click the DB instance name.

Step 4 On the Basic Information page, click the Parameters tab.

Step 5 Enter the keyword password in the search box in the upper right corner of thepage and press Enter to view the search result.

Data Replication ServiceFAQs 8 General Operations

2020-03-31 52

Page 56: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Figure 8-1 Modifying parameters

Step 6 In the search result, change the values of the parameters listed in Table 8-1 basedon the password strength requirements. Ensure that the parameter values arewithin the password complexity range.

Table 8-1 Password description

Parameter Allowed Value Description

validate_password_length

0-2,147,483,647 Specifies the minimum passwordlength verified by thevalidate_password plugin.

validate_password_mixed_case_count

0-2,147,483,647 Specifies the minimum number oflowercase and uppercase letters in apassword when the password policylevel is MEDIUM or higher.

validate_password_number_count

0-2,147,483,647 Specifies the minimum number ofdigits in a password when thepassword policy level is MEDIUM orhigher.

validate_password_policy

LOW, MEDIUM,STRONG

Specifies the password policyexecuted by the validate_passwordplugin.

validate_password_special_char_count

0-2,147,483,647 Specifies the minimum number ofnon-alphanumeric characters in apassword when the password policylevel is MEDIUM or higher.

Step 7 After the parameter values are modified, save the modification.

Figure 8-2 Modification result

Data Replication ServiceFAQs 8 General Operations

2020-03-31 53

Page 57: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Step 8 Back to the Select Migration Type page and perform the next step.

----End

8.4 How Do I Configure the Shard Key for a MongoDBSharded Cluster?

MongoDB shards data at the collection level, distributing the collection data usingshard keys.

You choose the shard key when sharding a collection. Each record contains a shardkey, and the shard key is either an indexed field or indexed compound fields.MongoDB database distributes data in different chunks according to the shard key,and distributes chunks evenly among the shards. To divide data chunks by shardkey, MongoDB database uses two sharding methods: range-based sharding andhashed sharding.

Table 8-2 Shard key classification

Shard KeyType

Description Application Scenario

Range-basedsharding

Ranged-based sharding involvesdividing data into contiguous rangesdetermined by the shard key values.Range-based sharding is the defaultsharding methodology if no otheroptions are specified.This allows for efficient queries wherereads target documents within acontiguous range. The distributionroute determines which data chunkstores the data required and forwardsthe request to the correspondingshard.

It is recommended whenthe shard key has highcardinality with lowfrequency, and the shardkey value does notchange monotonically.

Hashedsharding

Hashed sharding uses a hashed indexto partition data across your sharedcluster and to create chunks.Hashed sharding provides more evendata distribution across the shardedcluster Hash values enable data to berandomly distributed in each chunk,and therefore are randomlydistributed in different shards.

If the shard key valuesthat have a highcardinality or changemonotonically, or thereare large number ofdifferent values, hashedsharding is an idealoption.

Once you shard a collection, the shard key and the shard key values areimmutable. If you need to modify the shard key of a document, you must deletethe document. Then modify the shard key and insert the document again.

Data Replication ServiceFAQs 8 General Operations

2020-03-31 54

Page 58: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

The shard key does not support array indexes, text indexes, geographical indexes, andspatial indexes.

Range-based Sharding

Step 1 Run the following command to enable database sharding:

sh.enableSharding(database)

database indicates the database for which the sharded collection is enabled.

Step 2 Configure the collection's shard key.

sh.shardCollection(namespace, key)

● namespace consists of a string <database>.<collections> specifying the full namespaceof the target collection.

● key indicates the index for the shard key.

● If the collection is empty, skip this step because the index on the shard keycan be created automatically.sh.shardCollection()

● If the collection is not empty, create an index key. Then, run the followingcommand to set the shard key:sh.shardCollection()

----End

Hashed Sharding

Step 1 Run the following command to enable database sharding:

sh.enableSharding(database)

database indicates the database for which the sharded collection is enabled.

Step 2 Set hashed shard keys.

sh.shardCollection("<database>.<collection>", { <shard key> : "hashed" } , false,{numInitialChunks: Number of preconfigured chunks})

The value of numInitialChunks is calculated as follows: db.collection.stats().size /10*1024*1024*1024.

If the collection contains data, run the following command to create a hashedindex for the hashed key:

db.collection.createIndex()

Run the following command to create a hashed shard key:

Data Replication ServiceFAQs 8 General Operations

2020-03-31 55

Page 59: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

sh.shardCollection()

----End

8.5 Does Bandwidth Expansion Affect the Running DRSTasks?

When the cloud connection bandwidth is expanded, the bandwidth link needs tobe re-established and the network is disconnected. Whether the networkdisconnection affects DRS tasks depends on the network disconnection durationand whether the source database IP address changes. For example, for the MySQLDB engine, if the network is disconnected for one day and the binlog of the sourcedatabase is cleared within this day (the binlog clearing policy of MySQL isconfigured by the user), the task cannot be resumed. In this scenario, you need toreset the task. If the network is interrupted for a short period of time and the IPaddress of the source database in the VPN remains unchanged after thebandwidth link is changed, the system can continue to resume the task.

8.6 Why Data in MariaDB and SysDB Cannot BeMigrated or Synchronized?

In some MariaDB versions, the SysDB database is used as a system database(similar to the sys database of MySQL 5.7). Therefore, DRS considers the SysDBdatabase as the system database of all MariaDB databases by default (similar tothe MySQL, information_schema, and performance_schema databases). If theSysDB is a service database, you can submit a service ticket for solving thisproblem.

8.7 Constraints and Operation Suggestions on Many-to-One Scenario

DRS supports many-to-one scenarios during migration or synchronization ofdifferent types of instances and tables to suit your service requirements.

Operation Suggestions● To ensure that there is sufficient space during task creation, you are advised

to calculate the total data volume of the source database and plan how toallocate the disk space of the destination instance. The remaining disk spacemust be greater than the total data volume of the source database. Forexample, if the data volume of source system1 is 1 GB, the data volume ofsource system2 is 3 GB, and the data volume of source system3 is 6 GB, theremaining disk space of the destination instance must be greater than 10 GB.

● To improve the performance of the destination MySQL database, you areadvised to use the Save Change function to configure common parameters(except max_connections). For performance parameters, you need tomanually change the parameter values based on the specifications of thedestination database.

Data Replication ServiceFAQs 8 General Operations

2020-03-31 56

Page 60: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Figure 8-3 Parameter comparison

Scenario 1: Many-to-One Data Migration

Data migration aims to migrate the entire database. Multiple databases can bemigrated at the instance level. Databases with the same name in the sourcesystem cannot be migrated and database name mapping is not supported.

Figure 8-4 Many-to-one data migration

Data Replication ServiceFAQs 8 General Operations

2020-03-31 57

Page 61: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

Scenario 2: Many-to-One Data SynchronizationUnlike data migration, data synchronization maintains continuous data flowbetween different services. It supports table-level, many-to-one synchronizationand database-level mapping.

Figure 8-5 Many-to-one data synchronization

Flow ChartWhen creating a task, ensure that the second task is created after the first taskhas entered the full migration state. For details, see Getting Started.

Figure 8-6 Flow chart

Data Replication ServiceFAQs 8 General Operations

2020-03-31 58

Page 62: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

8.8 Where Can I View DRS Operation Logs?You can view DRS operation logs on the Cloud Trace Service (CTS) console.

Data Replication ServiceFAQs 8 General Operations

2020-03-31 59

Page 63: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

A Change History

ReleasedOn

Description

2020-03-31 This issue is the fourteenth official release, which incorporatesthe following change:7.3 Common Multi-Active DR Scenarios

2020-02-29 This issue is the fourteenth official release, which incorporatesthe following changes:7.1 What Are RPO and RTO of DRS Disaster Recovery?3.2 Which MySQL Permissions Are Required for DRS?1.12 What Is Single-Active/Dual-Active Disaster Recovery?7.2 How Do I Select Active Database 1 and 2 for Dual-ActiveDR?7.3 Common Multi-Active DR Scenarios

2020-01-30 This issue is the thirteenth official release, which incorporates thefollowing changes:4.9 What Are the Precautions for Migrating Data from anEarlier Version MySQL to MySQL 8.0?1.11 Does DRS Support Resumable Uploads?4.8 What Can I Do If MyISAM Tables Are Not Supported byRDS for MySQL?7.1 What Are RPO and RTO of DRS Disaster Recovery?4.7 Why Is the Collation of Heterogeneous or OracleDatabases Converted to utf8mb4_bin After Those DatabasesAre Migrated to MySQL?

2019-11-30 This issue is the twelfth official release, which incorporates thefollowing change:4.6 What Do I Do If the Maximum Index Length Has BeenReached During Migration from Oracle to MySQL?

Data Replication ServiceFAQs A Change History

2020-03-31 60

Page 64: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

ReleasedOn

Description

2019-10-30 This issue is the eleventh official release, which incorporates thefollowing changes:1.10 Does DRS Support Data Replication in a Specified TimePeriod?4.4 How Do I Ensure that All Services on the Database AreStopped?

2019-09-30 This issue is the tenth official release, which incorporates thefollowing changes:8.6 Why Data in MariaDB and SysDB Cannot Be Migrated orSynchronized?4.5 What Can I Do When Scheduled Task Failed to StartLeaving Message "can not get agency token"

2019-08-30 This issue is the ninth official release, which incorporates thefollowing changes:4.2 How Do I Maintain the Original Service User PermissionSystem After Definer Is Forcibly Converted During MySQLMigration?8.5 Does Bandwidth Expansion Affect the Running DRS Tasks?1.8 Can DB2 Be Migrated to PostgreSQL Using DRS?1.9 Can Microsoft SQL Server Database Synchronize Data withLocal Databases in Real Time?

2019-06-30 This issue is the eighth official release, which incorporates thefollowing changes:4.12 How Do I Disable the Balancer?8.2 How Do I Set Global binlog_format=ROW to Take EffectImmediately?1.7 Does DRS Support Migration Between the Public Cloudand the Dedicated Cloud?5.1 Can DRS Sync Tables of Different Schemas to the SameSchema?

2019-05-30 This issue is the seventh official release, which incorporates thefollowing changes:6.1 How Long Does It Take for SDK to Consume DatabaseChanges?8.1 What Can I Do When Information Overlaps on the DRSConsole?4.11 Which Plug-ins Can Be Migrated During PostgreSQLDatabase Migration?4.10 What Can I Do When OOM Occurs During the Migrationof MongoDB Databases?

Data Replication ServiceFAQs A Change History

2020-03-31 61

Page 65: Data Replication Service - HUAWEI CLOUD · 2020-03-18 · Figure 1-2 Online migration Backup Migration It often becomes necessary to hide the real IP address of your database for

ReleasedOn

Description

2019-04-30 This issue is the third official release, which incorporates thefollowing change:4.13 How Do I Export and Import Events and Triggers inBatches?

2018-12-30 This issue is the second official release, which incorporates thefollowing changes:● 4.14 How Can I Migrate Databases or Tables Whose Names

Contain Uppercase Letters?● 8.4 How Do I Configure the Shard Key for a MongoDB

Sharded Cluster?● 8.3 How Do I Change the Destination Database Password

to Meet the Password Policy?

2018-10-31 This issue is the first official release.

Data Replication ServiceFAQs A Change History

2020-03-31 62