Embed Size (px)
Citation preview
RabbitMQ for .NET Developers P2: Notes
Ngo Nguyen Chinh
Ha Noi 2016
(*) Here are my notes after completing a course of study offered by Pluralsight
1. Messages and Data
1.1 Introduction
Agenda
● Serialization & Real Data ● Message Format & Serialization ● Messages & Message Identification
1.2 Serialization & Real Data
What & Why
● Real world != “Hello World” ● Real World
○ Applications work with objects ○ Messaging systems work with messages ○ Objects are converted to messages ○ Messages are converted back to objects
Application Message Processing
1.3 Serialization Demo
PowerShell script
Client application
Server application
Contracts
1.4 Message Format and Serialization
What & Why
● Types of serialization ○ XML ○ JSON ○ Binary
● How do I communicate what the format of the message body is?
ContentType property!
1.5 Message Format Demo
PowerShell script
Client application
Server application
Contracts
1.6 Message Identification
What & Why
● Real world ○ Many different types of messages ○ Data structures can use inheritance or polymorphism ○ Messages aren’t always sent by the same technology as processes it
● To process a message the receiver needs to workout what the message is
○ From the content? ○ The sender indicates the message type
BasicProperties has a Type property intended for the message type
Application Message Processing
How Can We Do It
● Message Type = BasicProperties.Type ○ Custom String? ○ .net Fully Qualified Name ○ XSD namespace + Root Element
● Considerations ○ Should be application agnostic (if you want Interoperability) ○ What about versioning?
1.7 Shared Type Demo
PowerShell script
Client application
Server application
Contracts
1.8 Interoperable Demo
Using Xml Schema
1.9 Summary
● Messaging Challenges ○ Serialization & Real Data ○ Message Format & Serialization ○ Messages & Message Identification
This is one of the keys to interoperability
2. Large Messages
2.1 Introduction
Agenda
● Large Messages ○ Buffered ○ Chunked
2.2 Large Messages
Background
● What if I need to push large messages through Rabbit? ● Messaging systems
○ General recommendation = small messages ● AMQP
○ Supports very large messages ● RabbitMQ
○ You can use large messages ○ But you need to be careful
How Can We Do It
● Buffered ○ Small / medium files
● Chunked ○ Larger files
Buffered Sample
(1) Client reads entire file
(2) Then, this message is sent to a message queue
(3) Server receives this message from the message queue
(4) Server writes entire file
Chunked Sample
(1) Client reads a chunk file
(2) Then, the Client sends this chunk to a message queue
(3) Next step, the Client reads other chunks from file and send them to the message queue
(4) Server receives a chunk from the message queue
(5) Then, the Server write this chunk to file
(6) The Server received an other chunk from the message queue. After receiving new chunk, the Server writes this chunk to file.
(7) Repeat until write last chunk to file.
2.3 Buffered Demo
PowerShell script
Client application
Server application
2.4 Chunked Demo
PowerShell script
Client application
Server application
2.5 Considerations for Chunking
● A group of related messages ● Multiple Receivers ● Ordered Delivery and Resequencer
2.6 Summary
● Challenges of real world messaging solutions ○ Large Messages
● Solutions ○ Chunked ○ Buffered ○ Separation from other messages
3. Dealing with Errors
3.1 Introduction
Agenda
● Errors & Retries ● Dead Letter Queue ● Routing Failures
3.2 Basic Errors and Retry
Background
● Error Scenarios ○ Exception processing a message ○ Poison Message ○ A message we don’t understand
● Basic Error Handling ○ Reject and Retry ○ Reject and Discard
Retry Sample
(1) Client sends a message to a message queue
(2) Server receives this message from the queue
(3) There is an error processing. The message is rejected
(4) For RETRY mode. Server will return this message to queue
(5) The message is Retried
Reject Sample
(1) Client sends a message to a message queue
(2) Server receives this message from the queue
(3) There is an error processing. The message is rejected
(4) For DISCARD mode. The message will Discard
How Can We Do It
Demo Scenarios
3.3 Basic Retry Demo
PowerShell script
Client application
Server application
Run demo: DISCARD
(1) Client sends a message “1”
(2) Queue has an unackledged message
(3) Server can’t process this message “1”, so the server discards the message
Run demo: RETRY
(1) Client sends a message “2”
(2) The message is rejected but it is retried forever
(3) And queue still has an unack message
3.4 Advanced Errors and Retry
Background
● We can’t retry forever! ● We should retry n times and then reject ● We can decide if to reject sooner or later depending on the type of error
Scenarios Sample
How Can We Do It?
● Reject = BasicReject (no requeue) ● Retry = Custom Solution
○ Custom Property to Track no. Retries ○ No retries incremented with each attempt ○ Instead of rejecting we republish a new message with updated
properties ○ Application decides how many times it will allow a message to retry
3.5 Advanced Errors Demo
PowerShell script
Client application
Server application
3.6 Dead Letter Queue
Background
● When we reject messages we don’t just want to throw them away ● Store errors for review by Ops ● Advanced error handling? ● Repair & Resubmit
Dead Letter Sample
(1) Client sends a message to Queue
(2) Server receives this message from Queue
(3) The message is rejected and not requeuing message. This message is moved to DeadLetterQueue
(4) A DeadLetterQueueProcessor receives this message from the DeadLetterQueue
(5) The DeadLetterQueueProcessor repair and resubmit message
(6) Server receives the repaired message from Queue
How Can We Do It?
● Declare an Exchange for receiving messages ● On queue set the xdeadletterexchange property
● Routing Rules ○ Normal exchange routing rules can apply ○ You can setup a dead letter routing key
3.7 Dead Letter Queue Demo
PowerShell script
Client application
Server application
DeadLetterQueueProcessor
Run Demo
● Queues & Exchanges
● Send message “1”
● Send message “2”
3.8 Routing Failures
Background
● When processing messages sometimes a message will not be subscribed to
● Routing failure ○ Could be an error ○ Could be a malicious message ○ Could be fixable
Alternative Exchange!
Routing Failures
(1) Client sends a message with routing key ‘bananas’
(2) No one is interested!
(3) This message is moved to Alternative Exchange. Then it is moved to Backup Queue
How Can We Do It?
3.8 Routing Failures Demo
PowerShell script
Client application
Run Demo
● Exchanges and Queues
● Send correct routing (routing key: ‘apples’)
● Send correct routing (routing key: ‘oranges’)
● Send routing failures (routing key: ‘bananas’)
3.9 Summary
● Challenges of real world messaging solutions ○ Basic Error Handling ○ Advanced Error Handling ○ Dead Letter Queue ○ Routing Failures
4. Advanced Topics
4.1 Introduction
Agenda
● Scheduled Delivery ● Virtual Hosts
4.2 Scheduled Delivery
Background
● How can I publish a message but delay the processing of the message?
Holding Queue?
Scheduled Delivery
(1) Client sends a message to HoldingQueue
(2) Message schedule reached. It is moved to Exchange
(3) Then this message is moved to ProcessingQueue
How Can We Do It
4.3 Scheduled Delivery Demo
PowerShell script
Client application
Run Demo
● Exchanges & Queues
● Send messages (1) Client sends a message
(2) First, the HoldingQueue receives this message
(3) Then, after waiting the timeout (5000), this message is moved to the NormalQueue
4.4 Virtual Hosts
Background
● Can I sandbox messaging in RabbitMQ? ● Why
○ Application isolation ○ Data protection ○ Support separation ○ SLA
Virtual Host!
Virtual Host
How Can We Do It?
4.5 Virtual Hosts Demo
PowerShell script
Client application
Run Demo
● Exchanges & Queues
● Send a message to Host1
● Send a message to Host2
4.6 Summary
● Challenges of real world messaging solutions ○ Scheduled Delivery ○ Virtual Hosts
5. References
https://app.pluralsight.com/library/courses/rabbitmqdotnetdeveloperspt2
