Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
Client-server
syncronouscommunication
communicationinitiated by client
spikes in server load
messages get lost
client must knowserver‘s location
server must be up and running, ready for requests
client server
Indirect communication
Messaging
queue
broker
consumerproducer
syncronouscommunication
communicationinitiated by client
spikes in server load
messages get lost
client must knowserver‘s location
server must be up and running, ready for requests
message message
Messaging
queue
broker
consumerproducer
producer
producer
consumer
consumer
AMQPAdvanced Message Queuing Protocol
AMQP
producer consumerqueue AMQP connection
channel
channel
chan
chan
exchange
broker
virtual host
binding
RabbitMQ
• Erlang implementation of AMQP
• AMQP v0.9.1 with custom extensions
• Data persistance– In-memory/file-persisted embeded database
– Specific message storage and index files
• Clustering– Erlang´s capabilities
• Plugins– Web-based management
Tips
• Connecting to RMQ
– Your application should start irrespective ofwhether a connection to RMQ can be establishedor not
– If the connection to RMQ is lost, your applicationshould reconnect by itself
– If the connection is down, producing/consumingmessages should fail gracefully
AMQP Channels
• AMQP connection is a TCP connection
• A channel is a logical connection
– commands are sent via channels
– all channels reuse the same TCP connection
– hundreds of thousands of new channels a second
AMQP connection
channel
channel
chan
chan
Tips
• Open a channel, do something with thechannel, close the channel
• Channels instances are technically threadsafe– It is recommended to avoid having several
threads using the same channel concurrently
• Channels are terminated by exceptions– Cascade of failures: if some part of the program
tries to use a terminated channel, it will fail too
AMQP Message
properties:content-type::stringcontent-encoding::stringdelivery-mode::1|2priority::0-9correlation-id::stringreply-to::stringexpiration::stringmessage-id::stringtimestamp::timestamptype::stringuser-id::stringapp-id::stringreserved::nullheaders::map[string,object]
body:byte[]
• No information about the producer and consumer
RMQ Message
• Two parts
– Payload
• Anything
– Label
• Describes the payload
• Information about where the message should go– Exchange name
– Topic/routing key (optional)
AMQP Broker
Queue
• Place for messages ready to be consumed
• How to create a queue?– queue.declare
– Queue name• Used by consumers to subscribe to the queue• Used when creating a binding• Otherwise, RMQ assigns a random name
– Other properties• durable
• exclusive
• auto-delete
• arguments
Queue
• Who can create a queue?
– Both consumers and producers
– Consumers can’t create a queue while subscribed to another one on the same channel
• What if a queue already exists when trying to declare it?
– Idempotent if the properties are exactly the same
Tips
• Messages published into an exchange without a queue to be routed to are discarded by RMQ
• Who should create the queue?– Can you afford losing messages created by your
producers? If not, both consumers and producers should create the needed queue
• If you want to check the existence of a queue, set the passive option of queue.declare to true– check then act pattern discouraged
Queue and consumers
• Messages are sent immediately to consumers subscribed to the queue– basic.consume
– basic.get
• Consumer acknowledges a message iff it is done with processing (to RMQ and not to producer!)– basic.ack
– auto_ack parameter set to true when subscribing to a queue (acceptable risk of losing messages)
• Reject a message (rather than acknowledge it)– basic.reject
Tips
• Use basic.consume if the consumer is processing many messages out of a queue
• Use basic.consume if the consumer needs to automatically receive messages from a queue as soon as they arrive
• Don’t use basic.get in a loop as an alternative to basic.consume (performance overhead)
• Don´t forget to acknowledge a message, RMQ won´t send that consumer any more messages
Exchanges and bindings
• Exchange– Place where producers send messages
• Routing key– Rules upon which RMQ decides to which queue
it should deliver the message
• Binding– A queue is said to be bound to an exchange by a
routing key
Direct exchange
• Routing algorithm: if the routing key of a message matches therouting key of a queue, then the message is delivered to thatqueue
• If Q2 is bound to X by the routing key black, then all messageswith the routing key black will be deliverd to Q2
Direct exchange
• A default exchange
• Empty string as its name
• A newly declared queue will beautomatically bound to this exchangeusing the queue name as routing key
• channel.basic_publish(exchange=´´,
routing_key=´queue-name´, body=msg)
Fanout exchange
• Routing algorithm: a message isdelivered to all queues bound to theexchange (broadcast)
• Routing keys are ignored
Topic exchange
• Routing algorithm: if the routing key of a message matches the routing keyof one or more queues, then the message is deliverd to all the queues
• Routing key must be a list of words delimeted by dots• * can substitute for exactly one word• # can substitute for zero or more words
Topic exchange: example
• timeseries.<building>.<floor>.<room>.<area>.<type-id>.<instance-id>
• timeseries.iaas.0.353,window.light.12345
• Exchange– Name: timerseris.exchange.topic– Type: topic
• Queue– Name: timeseries.all– Type: durable– Routing key: timerseries.#
Virtual host
• Logical subdivision of RMQ´s execution space– Virtual message broker– Its own queues, exchanges and bindings– Its own persmissions
• Allow for multiple applications to one RMQ server
• Default vhost called /
• Vhosts are to RMQ what VMs are to physical servers
Tips
• Identify common functionality groups in your application and give each one itsown vhost
• Use the default one for testing
– guest/guest as credentials
– Guest has full permissions
Reliability: durability
• Queues and exchanges together with messagesdon´t survive a reboot of the RBM server
• The durable property of queues and exchanges isset to false by default
• For messages to survive reboot, queues and exchanges must be durable, but this is not sufficient– Messages must be persistent– Set the delivery-mode option of the message to 2
Tips
• Should you use persistent messaging forall messages?
– Writing a message to disk is much slower thansaving it in RAM
– Slows down RMQ in terms of the number ofmessages it can process per second
Reliability: redundancy
• Multiple RMQ instances
– Performance
– Scalability
– Fault tolerance
References
• Coulouris, G., Dollimore, J., Kindberg, T., Blair, G., Distributed Systems: Concepts and Designs. Fifth Edition.
• Videla, A. and Williams, J. W. RabbitMQ in Action: Distributed messaging for everyone. Manning Publications Co. 2012.
• Dossot, D. RabbitMQ Essentials. Packt Publishing. 2014.
• RabbitMQ Tutorial, https://www.rabbitmq.com/tutorials/.