Distributed Messaging Patterns

Cecil L. Phillip Senior Cloud Advocate @ Microsoft @cecilphillip Communication

How do we share information between systems?

Direct Brokered Broadcast

Distribution of messages Moving messages between Moving messages via from one source to anyone two endpoints. communication interested

“Phone” “Mail” “Radio” A Transfer B

Messaging is all about getting data from here to there (Getting data back from there to here is the just same thing) A A

A Transfer B

A A

Sometimes there’s a lot of “here” A Transfer B

Sometimes there’s A LOT of data B B

B A Transfer B

Sometimes there’s a lot of “there” B B C I

C A Transfer E

Sometimes the “there” are all different L L A Transfer B

Sometimes “there” isn’t currently paying attention A Transfer B

Sometimes “there” isn’t currently paying attention A Transfer B

Sometimes there’s trouble A Transfer B

Sometimes “there” is VERY BUSY Messaging Protocols

IETF HTTPS & WebSocket Protocol • Request-response application protocol • OSS: Nginx, Apache HTTP Server, Kestrel, YARP, Envoy

OASIS AMQP 1.0 • Symmetric, reliable message transfer protocol with support for multiplexing and flow control • OSS: Apache ActiveMQ, Broker-J, Apache Qpid Dispatch Router, , Pivotal RabbitMQ

OASIS MQTT 3.x/5.x • Reliable publish-subscribe protocol for telemetry transfer and state synchronization • OSS: Apache ActiveMQ, Hono, Pivotal RabbitMQ, Eclipse Mosquitto, HiveMQ, VerneMQ, etc. Synchronous vs. Asynchronous

Sender Receiver Sender Receiver

• Sender sends request and then waits for • Sender sends a message and proceeds to an immediate answer do other things. • May happen via asynchronous I/O, but • Replies may flow back on a separate path. the logical thread is preserved. Synchronous Patterns Throttling

• Request rates can be defined Client on per-client based on a given criteria

• Throttling state can be Client Server communicated via the transport protocol

• Clients should implement an adaptive retry strategy to get Client their work submitted Load balancing w/ a Reverse Proxy

Client Server • Messages are distributed between multiple servers

Client Proxy Server • Clients only need direct access access the proxy

Server • The proxy can be enhanced Client with additional functionality via middleware Retry Strategies

• Connectivity between Client systems can unreliable and faults can occur for various reasons

Client Server • Use reasonable delays between retries to prevent overloading the system

Client • Know when to stop retrying Asynchronous Patterns Long-Running Work reply queue

• Processing at the consumer may take very long (minutes to hours) correlationid:

replyto: • Producers entrust jobs into a queue. Producer Consumer • Bad jobs are moved into a dead- Retry 1 letter queue for inspection. Retry 2

• Flow back to the producer is performed through a reply queue dead-letter queue Load Leveling

requests/sec requests/sec • Queues act as an inbox for requests to a consumer. processing capacity

• Consumer pulls work when it time time has capacity for processing. • Consumers process at their Producer own pace. Producer Producer Consumer Producer Producer • No “too busy” errors, easier resource governance. Load Balancing (and Auto Scaling)

• Multiple consumers compete for messages queue length rolling average

• Truly load-aware job scale-up threshold

balancing scale-down threshold time +1 Producer Producer Consumer • Queue-length can be Producer Consumer Producer observed and more Producer Consumer consumers can be added to to manage load Publish-Subscribe

• Directing one input message Consumer to zero or more interested 1 2 3

parties (subscribers). filter

• Every subscriber can obtain a Producer Topic filter Consumer copy of every published 2 4 6 message.

filter • Subscribers may provide filters that select a subset of 1 3 5 Consumer the published messages. Sparse Connectivity

• Common scenario with mobiles devices and IoT applications. store and forward queue Push Producer

• Mobile users switch networks, move messages while online go out of range, hit Unreliable network boundary, e.g. mobile wireless networking bandwidth caps, etc.

Pull • Using a local store/forward Consumer queue makes communication paths more robust. Recap

Synchronous Asynchronous

Sender Receiver Sender Receiver Samples and Documentation

https://www.theurlist.com/distributed-messaging-patterns Thanks also to the sponsors. Without whom this would not have been posible. Thanks and … See you soon!