17
Santa Clara, California | April 23th – 25th, 2018 Brothers in Arms How to Make MySQL and PostgreSQL Work Together Charly Batista Senior Support Engineer

Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

Santa Clara, California | April 23th – 25th, 2018

Brothers in ArmsHow to Make MySQL and PostgreSQL Work Together Charly Batista Senior Support Engineer

Page 2: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!2

Agenda• Introduction • Replication - PostgreSQL Replication- MySQL Replication

• Challenges • Available solutions - SymmetricDS- Tungsten Replicator- Apache Kafka with Debezium

• Demo

Page 3: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!3

Introduction

Why replicate data between different databases?• Migrate data from one database to the another with minimal

downtime

• Feed legacy/in-house systems with web transactions

• DW and Reporting

• Use the correct technology assets (GEO, Sync Replication, json, etc)

Page 4: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

Replication

Page 5: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!5

Replication

PostgreSQL Streaming Replication• Physical replication (blocks, byte-by-

byte)• Not possible with different major

versions• Slaves readable, not writeable• Slaves have same structure as master• Complex timeline control• Re-sync slave often requires new BKP

Page 6: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!6

Replication

PostgreSQL Logical Replication (10+)• Uses a publish and subscribe model• Slaves can now be writable• Now possible with different major

versions• Transactional consistency is

guaranteed for publications within a single subscription

• Better granularity and control• Doesn’t replicate DDL

Page 7: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!7

Replication

MySQL• Logical Replication• Slaves can be writable• Possible with different major versions*• Good granularity and control• Replicates schema and data

Page 8: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

Available solutions

Page 9: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!9

Available solutions

SymmetricDS• Trigger based• Scalable• Asynchronous• One way and multi-master• Routing• Filtered Synchronization• Transformation Capabilities

Page 10: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!10

Available solutions

Tungsten Replicator• Row-based (heterogeneous)• Asynchronous• Transactionally Consistent• Flexible Extraction• Filtered Synchronization• Transformation Capabilities

Page 11: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!11

Available solutions

Debezium (Apache Kafka)• Row-based• Asynchronous/Queue (topics)• Publish/Subscriber model• Transactionally Consistent• Flexible Extraction• Filtered Synchronization• Transformation Capabilities

Page 12: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!12

Challenges

There are many challenges when replicating from different DB’s:• Data Type Differences• Provisioning• Database vs. Schema• SQL Dialect Differences• …

Page 13: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

DemoClick to add text

Page 14: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

Questions?Click to add text

Page 15: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!15

Rate My Session

Page 16: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

!16

Thank You Sponsors!!

Page 17: Brothers in arms · -SymmetricDS-Tungsten Replicator-Apache Kafka with Debezium • Demo!3 Introduction Why replicate data between different databases? • Migrate data from one database

Thank You!