Message Queue in Magento2

The Core Problem: Synchronous Processing at Scale

In traditional Magento workflows, most operations are synchronous. Synchronous means “do everything right now and make the user wait until it finishes.”

A synchronous process executes all logic in the same request–response lifecycle. The HTTP request is blocked until every dependent operation finishes execution.

Example:

You order food at a restaurant, and the cashier refuses to give you a receipt until:

• The food is cooked
• The plates are washed
• The ingredients are restocked
• The chef submits a report

That’s inefficient and frustrating.

Magento faced the same issue with:

• Order placement
• Inventory updates
• Email notifications
• Third-party integrations
• Indexing operations

As Magento installations scaled (large catalogs, heavy traffic, multiple integrations), this approach became slow, fragile, and risky.

A message queue is a system that lets you store a task and process it later, instead of immediately.
It is an asynchronous communication mechanism where:

• A producer publishes a message
• The message is stored in a queue
• A consumer processes the message independently of the original request

Example:

• You write a letter (message)
• Drop it in a mailbox (queue)
• The post office delivers it later (consumer)
• You don’t wait at the mailbox until delivery is complete

Magento adopted this model to decouple heavy operations from user-facing requests.

1. Synchronous Flow (Traditional)

Execution flow:

1. Customer places an order
2. Magento:
   – Saves order
   – Sends emails
   – Updates inventory
   – Triggers ERP sync
   – Updates analytics
3. Response is returned

Problems:

• Slow checkout
• Higher timeout risk
• Any failure breaks the entire flow

2. Asynchronous Flow (With Message Queue)

Execution flow:

1. Customer places an order
2. Magento:
   – Saves order
   – Publishes messages for additional tasks
3. Response is returned immediately
4. Background consumers process tasks independently

Why this matters:

• Faster response times
• Fault isolation
• Better scalability

Magento’s message queue architecture is composed of well-defined roles.

Publisher — The Message Producer

1. What Is a Publisher?

A publisher is the part of the system that creates a message and sends it to the queue. It uses Magento’s PublisherInterface to send serialized data to a specific topic.

2. Publisher Responsibilities

• Prepare message payload
• Serialize data (JSON)
• Publish to a topic
• Remain unaware of who processes it

Publishers do not know about consumers. This ensures loose coupling.

Topics — The Communication Contract

1. What Is a Topic?

A topic is a named channel that describes what kind of message this is.
It defines:

• Message name
• Expected payload schema
• Communication contract between publisher and consumers

Topics are declared in:
etc/communication.xml

2. Why Topics Matter

Topics:

• Enforce data consistency
• Allow multiple consumers
• Act as stable integration points

Message Payload — What Data Is Sent

The payload is the actual data inside the message. Magento serializes the payload (usually JSON) before sending it to the queue.

Queues — The Holding Area

1. What Is a Queue?

A queue is a waiting line for tasks. Queues store messages until a consumer retrieves and processes them.

2. Queue Behavior

• FIFO (First In, First Out)
• Durable (messages survive restarts)
• Can retry failed messages

Consumers — The Workers

1. What Is a Consumer?

A consumer is a background worker that processes messages from a queue.

A consumer:

• Listens to a queue
• Deserializes the message
• Executes business logic
• Acknowledges success or failure

Defined in:

etc/queue_consumer.xml

Consumers are executed via CLI:

bin/magento queue:consumers:start consumer_name

Message Broker — RabbitMQ vs MySQL

1. What Is a Message Broker?

A broker is the system that physically stores and delivers messages.

Magento supports:

• RabbitMQ (recommended)
• MySQL (fallback)

2. RabbitMQ (Recommended)

Why Magento prefers RabbitMQ:

• High throughput
• Reliable message delivery
• Advanced routing
• Acknowledgement handling

3. MySQL Message Queue

When it’s used:

• RabbitMQ not installed
• Small or development environments

Limitations:

• Database load
• Lower performance
• Not ideal for scale

1. Create a Custom Module

Path: app/code/YourVendor/QueueExample

registration.php

<?php
\Magento\Framework\Component\ComponentRegistrar::register(
    \Magento\Framework\Component\ComponentRegistrar::MODULE,
    'YourVendor_QueueExample',
    __DIR__
);

etc/module.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="urn:magento:framework:Module/etc/module.xsd">
    <module name="YourVendor_QueueExample"/>
</config>

2. Define the Message Queue Topics

etc/communication.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">

    <topic name="notifycustomer.massmail" request="string">
        <handler name="notifycustomer.massmail"
                 type="YourVendor\QueueExample\Model\Consumer"
                 method="process"/>
    </topic>

</config>

3. Map Queue to Consumer

etc/queue_consumer.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/consumer.xsd">
    <consumer name="notifycustomer.massmail"
              queue="notifycustomer.massmail"
              connection="db"
              consumerInstance="Magento\Framework\MessageQueue\Consumer"
              handler="Webkul\MessageQueue\Model\Consumer::process"/>

</config>

4. Define Publisher Settings

etc/queue_publisher.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/publisher.xsd">

    <publisher topic="notifycustomer.massmail">
        <connection name="db" exchange="magento-db"/>
    </publisher>

</config>

5. Define Queue Topology

etc/queue_topology.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">

    <exchange name="magento-db" type="topic" connection="db">
        <binding id="binding_notifycustomer"
                 topic="notifycustomer.massmail"
                 destinationType="queue"
                 destination="notifycustomer.massmail"/>
    </exchange>

</config>

6. Write the Consumer Class

Model/Consumer.php

<?php
namespace YourVendor\QueueExample\Model;

class Consumer
{
    /**
     * The process() method is called by the message queue when a message arrives
     * @param string $message
     */
    public function process($message)
    {
        // Decode JSON and process
        $data = json_decode($message, true);

        // Example logic: perform notification or update database
        // replace with your own business logic
        foreach ($data as $item) {
            // your processing code here
        }
    }
}

7. Write the Publisher Logic

This is the code that will send messages to the queue during normal Magento operations.

Example Observer

Use an observer to publish messages when something happens (e.g., after product save).

namespace YourVendor\QueueExample\Observer;

use Magento\Framework\Event\ObserverInterface;
use Magento\Framework\MessageQueue\PublisherInterface;
use Magento\Framework\Json\Helper\Data as JsonHelper;

class AfterProductSave implements ObserverInterface
{
    private $publisher;
    private $jsonHelper;

    public function __construct(
        PublisherInterface $publisher,
        JsonHelper $jsonHelper
    ) {
        $this->publisher = $publisher;
        $this->jsonHelper = $jsonHelper;
    }

    public function execute(\Magento\Framework\Event\Observer $observer)
    {
        $product = $observer->getProduct();
        $data = [
            'id' => $product->getId(),
            'sku' => $product->getSku()
        ];

        $this->publisher->publish(
            'notifycustomer.massmail',
            $this->jsonHelper->jsonEncode($data)
        );
    }
}

Here, when a product is saved, a message is sent to the queue containing product details.

Magento 2’s message queue system is not an optional feature—it is a foundational architectural component designed for scale, reliability, and performance.

Understanding it deeply allows developers to:

• Design better modules
• Avoid performance bottlenecks
• Build enterprise-grade Magento solutions

Know more about Magento 2 Message Queue.

Looking to improve your store’s speed and overall performance? Check out our Magento 2 Speed & Optimization services.

For expert guidance or custom feature development, you may hire our Magento 2 developers to support your project.