Configuring the Message Queue with Redis

If multiple processes are to be used to process messages from the Shopware 6 message queue, you must ensure that identical messages are not processed by the same consumers. This requires the use of a technology that supports atomic locking of messages. Redis or RabbitMQ are suitable for this purpose.

Prerequisites

• Redis server
• Shopware server

Preparation in Shopware

First, the Admin Worker must be disabled so that the message queue can be processed via a background process. For more information on disabling the Admin Worker and the initial setup via a Supervisor process, see here: Disable the Shopware 6 Frontend Message Queue Worker.

Configuring the Shopware 6 Background Queue Worker

The configuration of Supervisor processes is covered in the following Help Center articles:

• Shopware 6 Background Queue Worker (Managed)
• Shopware 6 Background Queue Worker (Unmanaged)

Configuration in Redis

Since message queue jobs do not have a TTL by default, as they do in Redis, it is recommended to use appropriate key eviction policies, the configuration of which is described in the linked Help Center article.

If the message queue is running on a dedicated server used exclusively for this purpose, the default policy noeviction can be used without any issues.

Alternatively, you can run the message queue together with the app cache on the same Redis server. In this case, however, you should select the volatile-lru key eviction policy. This ensures that when the maxmemory limit is reached, only cache entries are removed, since only these have a TTL. The message queue jobs are thus preserved and will not be accidentally deleted.

Configuration in Shopware

Create a new configuration file at the following path: config/packages/messenger.yaml. This configuration file is used to configure the Shopware Messenger transport service.

Then insert the following content into the file:

Up to Shopware 6.4

parameters:
  env(MESSENGER_CONSUMER_NAME): 'consumer'

framework:
  messenger:
    transports:
 default:
 dsn: "redis://%env(REDIS_HOST_MESSENGER)%:%env(REDIS_PORT_MESSENGER)%/messages/symfony/consumer-%env(MESSENGER_CONSUMER_NAME)%/?delete_after_ack=true&delete_after_reject=true&dbindex=%env(REDIS_DB_MESSENGER)%"

Starting with Shopware 6.5

Stop the existing Shopware scheduler process

supervisorctl stop shopware-scheduled-tasks:shopware-scheduled-tasks_00

To check whether the message queue is empty, you can use the following command starting with Shopware 6.5:

bin/console messenger:stats

The output should show a 0 for each transport. In any case, wait until all jobs have been processed.

Then stop all Shopware message queue workers:

supervisorctl stop shopware-consumer:shopware-consumer_00

You can then switch the message queue to Redis in the Shopware configuration:

# File: config/packages/messenger.yaml

parameters:
  env(MESSENGER_CONSUMER_NAME): 'consumer'

framework:
  messenger:
    transports:
 # Shopware default transports
      async:
 dsn: "redis://%env(REDIS_HOST_MESSENGER)%:%env(REDIS_PORT_MESSENGER)%/messages_async?dbindex=%env(REDIS_DB_MESSENGER)%"
        options:
 consumer: "%env(MESSENGER_CONSUMER_NAME)%"
 failed:
 dsn: "redis://%env(REDIS_HOST_MESSENGER)%:%env(REDIS_PORT_MESSENGER)%/messages_failed?dbindex=%env(REDIS_DB_MESSENGER)%"
 options:
 consumer: "%env(MESSENGER_CONSUMER_NAME)%"
 low_priority:
 dsn: "redis://%env(REDIS_HOST_MESSENGER)%:%env(REDIS_PORT_MESSENGER)%/messages_low_priority?dbindex=%env(REDIS_DB_MESSENGER)%"
 options:
 consumer: "%env(MESSENGER_CONSUMER_NAME)%"

Description of a Transport

The transport DSN variant we use for Redis is structured as follows:

<transport-name>:
  dsn: "redis://<hostname>:<port>/<stream_key_name>?<options>"
  options:
    consumer: "custom-consumer-name-defined-in-environment-variable"    

Example for async

async:
  dsn: "redis://%env(REDIS_HOST_MESSENGER)%:%env(REDIS_PORT_MESSENGER)%/messages_async?dbindex=%env(REDIS_DB_MESSENGER)%"
  options:
    consumer: "%env(MESSENGER_CONSUMER_NAME)%"

The variables used here must then be added to Shopware’s .env.local environment configuration. To do this, open the .env.local file in the Shopware root directory and add the following section to the existing configuration:

# [...]

REDIS_HOST_MESSENGER=127.0.0.1
REDIS_PORT_MESSENGER=6379
REDIS_DB_MESSENGER=1

Then save the file and clear the Shopware cache via the console:

bin/console cache:clear

Restart all Supervisor processes with the following command:

supervisorctl start all

The message queue will then run via Redis.