Upload
keigo-suda
View
412
Download
8
Embed Size (px)
Citation preview
Apache Kafka & Kafka Connect
( ETL )
Future Architect ,IncKeigo Suda
2016/11/25D&S Data Night vol.04
27
Stream & Partition(RDB )
http://www.slideshare.net/KaufmanNg/data-pipelines-with-kafka-connect
28
N:1
Stream & Partition(RDB )
http://www.slideshare.net/KaufmanNg/data-pipelines-with-kafka-connect
32
$ bin/connect-standalone.sh config/connect-standalone.properties connector1.properties
(Standalone mode)
!
!
! Worker! Connector
Worker Connector
33
name=local-file-source connector.class=FileStreamSource tasks.max=1 file=test.txt topic=connect-test
bootstrap.servers=localhost:9092 key.converter=org.apache.kafka.connect.json.JsonConverter value.converter=org.apache.kafka.connect.json.JsonConverter ...
Worker
Connector
40
! Task Connector
!
Deploy
name=sample-sink connector.class=SampleSinkConnector tasks.max=1 topic=connect-test
Connector