Domain-based robots.txt

If you have several sales channels in your Shopware store, you can define your own robots instructions for the different domains that are used.

Requirements

• Shopware installation
• Multiple domains / sales channels

Customize directory structure

To begin with, another directory called robots must be created in the public directory of Shopware. Within the robots directory, the respective robots.txt files must then be created with the following naming convention, e.g: domain.tld.txt or sub.domain.tld.txt.

Example

An example of two different robots.txt in a Shopware installation:

public/robots/example.en.txt

public/robots/shop.example.en.txt

Terminal

mkdir -p /var/www/vhosts/<plesk-domain-name>/httpdocs/public/robots
touch /var/www/vhosts/<plesk-domain-name>/httpdocs/public/robots/shop.example.txt
touch /var/www/vhosts/<plesk-domain-name>/httpdocs/public/robots/example.com.txt

In the files shop.example.de.txt and example.com.txt you can now store the relevant robots instructions for the respective domain.

Plesk Control Panel

Navigate to the file browser of the corresponding subscription.

The robots directory must then be created in the public directory.

Confirm with the "OK " button and then change to the previously created directory.

In the robots directory, create a file with the following naming convention for each domain for which you want to define your own robots instructions, e.g: domain.tld.txt or sub.domain.tld.txt

Confirm the file name with the "OK " button and then select the file in the file browser, enter the desired robots instructions in the editor window that opens and save the change using the Save button highlighted in blue.

As instructions are to be defined for two domains in the example, the following file must also be created and filled with robots instructions: example.de.txt.

Define Apache2 rewrite rule

With Plesk, the Webserver Apache2 is used by default to deliver the websites of the respective subscription, which is why only an adjustment of the .htaccess is required here.

Insert the following at the beginning of the .htaccess file:

RewriteRule ^robots\.txt$ robots/%{HTTP_HOST}.txt [NS]

Store Plesk NGINX rewrite rule

If you have deactivated the NGINX proxy mode, the rewrite rule must be stored in the additional nginx instructions. To do this, navigate to the Hosting and DNS settings of your subscription and open the Apache and nginx settings.

Add the following nginx statement there and save the change by clicking on the OK button highlighted in blue.

\nAdditional nginx instructions:

# multi domain robots.txt
rewrite ^/robots\.txt$ /robots/$host.txt;

Standalone NGINX rewrite rule deposit

If NGINX is operated alone, the rewrite rule must be added to the corresponding server block. Depending on how you have configured NGINX, you will find the corresponding configuration file under /etc/nginx/conf.d or in /etc/nginx/sites-available.

\nExample:

server {
        listen 80;
        listen [::]:80;

        server_name domain.tld;
        root /var/www/domain.tld/public;

        # mutli domain robots.txt
+ rewrite ^/robots\.txt$ /robots/$host.txt

        index index.html index.htm;
}

The Nginx configuration must then be reloaded with the following command:

systemctl reload nginx