Configure message queue with Redis

If several processes are to be used to process messages from the Shopware 6 message queue, it must be ensured that identical messages are not processed by the same consumers. This requires the use of a technology that supports the atomic blocking of messages. Redis](https://redis.com/) or RabbitMQ are suitable for this purpose.

Preparation in Shopware

First, the Admin Worker must be deactivated so that the message queue can be processed via a background process. More information on deactivating the Admin Worker and the initial setup via a Systemd Service can be found here: Set up Shopware 6 Background Queue Worker.

Configuration of the Shopware environment configuration

Create a new configuration file under the path: config/packages/messenger.yaml. This configuration file can be used to configure the Shopware Messenger transport service. Then add the following content to the file:

parameters:
  env(MESSENGER_CONSUMER_NAME): 'consumer'

framework:
  messenger:
    transports:
      default:
        dsn: "redis://%env(REDIS_HOST)%:%env(REDIS_PORT)%/messages/symfony/consumer-%env(MESSENGER_CONSUMER_NAME)%/?delete_after_ack=true&delete_after_reject=true&dbindex=%env(REDIS_DB_MESSENGER)%"

The following environment variables are used in this configuration file:

The variables used here must then be added to the Shopware environment configuration .env.local. To do this, open the .env.local file in the Shopware root directory and add the following part to the existing configuration:

# .env.local

# [...]

REDIS_HOST=127.0.0.1
REDIS_PORT=6379
REDIS_DB_MESSENGER=1

Then save the file and empty the Shopware cache via the console:

bin/console cache:clear

The Shopware message queue is then executed via Redis.

Configuration of the Systemd services

In the next step, the Systemd services must be prepared for the execution of the message queue consumer. To do this, an Environment directive must be added for each consumer.

Example for the Systemd service:

# consumer1

[Service]
# Unique Messenger Consumer Name
Environment=MESSENGER_CONSUMER_NAME=consumer1

# [..]

# consumer2

[Service]
# Unique Messenger Consumer Name
Environment=MESSENGER_CONSUMER_NAME=consumer2

# [..]

Then execute the command systemctl daemon-reload so that the changes take effect. Then restart the associated Systemd services with the command systemctl restart shopware_consumer_*.

Finally, check whether the messenger:consume processes are working properly.

systemctl status shopware_consumer_*

Example output:

● shopware_consumer.service - Shopware Message Queue Consumer #1
     Loaded: loaded (/etc/systemd/system/shopware_consumer.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-12-01 14:00:00 CET; 60s ago
   Main PID: 501963 (php)
      Tasks: 1 (limit: 19169)
     Memory: 111.8M
        CPU: 11.298s
     CGroup: /system.slice/shopware_consumer.service
             └─501963 php /var/www/domain.com/shopware/current/bin/console messenger:consume --time-limit=120 --memory-limit=512M

Dec 01 14:01:00 demo.creolineserver.com systemd[1]: Started Shopware Message Queue Consumer #1.
Dec 01 14:01:00 demo.creolineserver.com php[501963]:  [OK] Consuming messages from transports "default".
Dec 01 14:01:00 demo.creolineserver.com php[501963]: // The worker will automatically exit once it has exceeded 512M of memory, been
Dec 01 14:01:00 demo.creolineserver.com php[501963]: // running for 121s or received a stop signal via the messenger:stop-workers
Dec 01 14:01:00 demo.creolineserver.com php[501963]: // command.
Dec 01 14:01:00 demo.creolineserver.com php[501963]: // Quit the worker with CONTROL-C.
Dec 01 14:01:00 demo.creolineserver.com php[501963]: // Re-run the command with a -vv option to see logs about consumed messages.