Communicate in microservices with Saga Pattern, Event Sourcing using RabbitMQ

6 min readJan 9, 2021

In this article, I want to summarize all of the parts (patterns and stacks) to implementing Inter-communication:

First, someone can communicate by transmitting messages. So what is a message and message pattern enterprise?
Second, transmitting messages isn’t simple between services. It makes sure good consistency of the database. So Saga pattern is a good solution.
Third, the Saga pattern has 2 popular ways to implement. But in our scenario, we will choose Events / Choreography, so we have an Event sourcing pattern.

1/ Message Patterns

There’re a lot of message patterns, but internal-communication between services in this article is built from some below popular ones.

In the scenario is finding how to communicate between micro-services, I will introduce some patterns must have and how to integrate technologies with them.

Messages

A Message is an atomic packet of data that can be transmitted on a channel. Thus to transmit data, a service must break the data into one or more packets, wrap each packet as a message, and then send the message on a channel. Likewise, a receiver service receives a message and must extract the data from the message to process it. The message system will try repeatedly to deliver the message (e.g., transmit it from the sender to the receiver) until it succeeds.

This is the duty of all services which need to send and receive messages to process its works.

Channels — Messaging services transmit data through a Message Channel, a virtual pipe that connects a sender to a receiver. A newly installed messaging system doesn’t contain any channels; you must determine how your applications need to communicate and then create the channels to facilitate it.

Routing

Message Router, an application component, and filter in the pipes-and-filters architecture will determine how to navigate the channel topology and direct the message to the final receiver, or at least to the next router.

Publish-Subscribe Channel

A Publish-Subscribe Channel works like this: It has one input channel that splits into multiple output channels, one for each subscriber. When an event is published on the channel, the Publish-Subscribe Channel delivers a copy of the message to each of the output channels. Each output channel has only one subscriber, which is only allowed to consume a message once. In this way, each subscriber only gets the message once, and consumed copies disappear from their channels.

2/ Saga Patterns

A saga is a sequence of local transactions where each transaction updates data within a single service. The first transaction is initiated by an external request corresponding to the system operation, and then each subsequent step is triggered by the completion of the previous one.

There are a couple of different ways to implement a saga transaction, but the two most popular are:

Events/ Choreography: When there is no central coordination, each service produces and listens to other service’s events and decides if an action should be taken or not.
Command/Orchestration: when a coordinator service is responsible for centralizing the saga’s decision-making and sequencing business logic.

Events / Choreography

In the Events/Choreography approach, the first service executes a transaction and then publishes an event. This event is listened to (subscribe) by one or more services that execute local transactions and publish (or not) new events.

Explaining:
Order Service -> (1): Create New Order published an event named ORDER_CREATED_EVENT.
(1) -> Payment Service-> (2): Payment Services subscribed event ORDER_CREATE_EVENT, so it received signal then process data, after that it publishes to event BILLED_ORDER_EVENT.
(2) -> Stock Service -> (3) -> Delivery Service -> (4) -> Order Service -> In the end.

Order Service saves a pending order and asks Order Saga Orchestrator (OSO) to start a create order transaction.
OSO sends an Execute Payment command to Payment Service, and it replies with a Payment Executed message
OSO sends a Prepare Order command to Stock Service, and it replies with an Order Prepared message
OSO sends a Deliver Order command to Delivery Service, and it replies with an Order Delivered message
With our scenario, we use Events / Choreography to apply
Next, we will approach Event Sourcing

3. Event Resourcing.

A service typically needs to atomically update the database and publish messages/events. For example, perhaps it uses the Saga pattern. To be reliable, each step of a saga must atomically update the database and publish messages/events. Alternatively, it might use the Domain event pattern to implement CQRS.

What is CQRS? CQRS is Command Query Responsibility Segregation. It means each command is responsible for each event that is subscribed.

Customers and Orders is an example of an application that is built using Event Sourcing and CQRS.

That’s all. We go to how to implement

4. How to implement intercommunication on services.

After finishing researching the necessary patterns, we need:

Event Bus works as a message broker, named “Channel” in Publish-Subscribe Pattern to receive and deliver messages.
Workers / Processes run along with services to receive signals of service then publish messages to Event Bus as well as always subscribe to events of Event Bus then send data to others.
For each event corresponding with each function, follow CQRS.

What is RabbitMQ? https://www.rabbitmq.com

RabbitMQ is lightweight and easy to deploy. It supports multiple messaging protocols.

RabbitMQ can be deployed in distributed and federated configurations to meet high-scale, high-availability requirements.

RabbitMQ allows services pub/sub using exchange, routing_key, and queue.
RabbitMQ defines routing as the combination of exchange, queue, and routing_key.
Ref: https://www.rabbitmq.com/tutorials/tutorial-four-python.html
X — is an exchange that has a topic type.
*.orange.*, *. * .rabbit, lazy.# — are routing keys which are patterns that help services determine the need to communicate.

So we use Rabbit for an Event Bus

What is Celery? http://www.celeryproject.org

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, supports scheduling, and supports well with RabbitMQ and Django ORM.

We use Celery works a Workers run along with services.