Upload
caique-lima
View
334
Download
0
Embed Size (px)
Citation preview
Apache Kafka
distributed, partitioned, replicated commit log service
Apache Kafka
Apache Kafka
Apache Kafka
Kafka at Easy Taxi
• +600 mi unique events/day
• Currently on 0.9.0
• 3 brokers
• 10 topics
• X consumers
Architecture
Api Kafka
Consumer 1
Consumer 2
Consumer 3
S3
Redis
Consumer N
BQ
Api
…
Redash
Do’s
• Rsyslog as a scalable simple producer
• Use BigQuery to allow SQL
• Start building the event based culture
• Right-time before real-time
Dont’s
• Doesn't run stress tests
• Increase architecture’s complexity
• Produce useless data
• Go live with default configs
Easy Explorer
Easy Taxi Point
Supply demand analysis
• Allow us to understand the area behaviour
• Generate a “health indicator” based on
predefined parameters
• Take actions without any human interaction
References
• Confluent blog
• Kafka Summit
• Jay Kreps interview
• Jepsen: Kafka
• Documentation
• Dados Brasil Slack
THANKS!
QUESTIONS