Streaming Data Challenges

The Data Dossier

Choose a Lesson

Cloud Pub/Sub Overview

Return to Table of Contents

Pub/Sub Hands On

Text

Connecting Kafka to GCP

Monitoring Subscriber Health

What is Streaming Data?- "Unbounded" data- Infinite, never completes, always flowing

Examples

Fast action is often necessary- Quickly collect data, gain insights, and take action- Sending to storage can add latency- Use cases:

- Credit card fraud detection- Predicting highway traffic

Next

Streaming Data Challenges

Traffic Sensors Credit Card Transactions Mobile Gaming

Streaming Data Challenges

The Data Dossier

Choose a Lesson

Cloud Pub/Sub Overview

Return to Table of Contents

Pub/Sub Hands On

Text

Connecting Kafka to GCP

Monitoring Subscriber Health

Tight vs. Loose Coupling in Systems- Tightly (direct) coupled systems more likely to fail- Loosely coupled systems with 'buffer' scale have better fault

tolerance

Previous

Streaming Data Challenges

Sender Sender

Receiver

Sender Sender

Overloaded Receiver

Lost messages, delays in processing

Tightly-Coupled System

Publisher

Subscriber

Publisher

Buffer/Message Bus

Publisher

Subscriber

Buffer/Message Bus

Subscriber

Loosely-Coupled System- Fault tolerance- Scalability- Message queuing