Domain-Based robots.txt

If you have multiple sales channels in your Shopware store, you can define separate robots.txt directives for the different domains used.

Prerequisites

• Shopware installation
• Multiple domains / sales channels

Customize the directory structure

First, create an additional directory named robots in the Shopware public directory. Within the robots directory, you must then create the respective robots.txt files using the following naming convention, for example: domain.tld.txt or sub.domain.tld.txt.

Example

An example of two different robots.txt files in a Shopware installation:

public/robots/example.de.txt

public/robots/shop.example.de.txt

Terminal

mkdir -p /var/www/vhosts/<plesk-domain-name>/httpdocs/public/robots
touch /var/www/vhosts/<plesk-domain-name>/httpdocs/public/robots/shop.example.de.txt
touch /var/www/vhosts/<plesk-domain-name>/httpdocs/public/robots/example.de.txt

You can now specify the robots directives relevant to each domain in the files shop.example.de.txt and example.com.txt.

Plesk Control Panel

Navigate to the File Browser for the corresponding subscription.

Next, create the robots directory inside the public directory.

Click the “OK” button to confirm, then navigate to the directory you just created.

In the robots directory, create a file for each domain for which you want to define custom robots directives, using the following naming convention, for example: domain.tld.txt or sub.domain.tld.txt

Confirm the file name by clicking the "OK" button, then select the file in the file browser, enter the desired Robots directives in the editor window that opens, and save the changes using the blue Save button.

Since the example requires defining instructions for two domains, you must also create the following file and populate it with Robots directives: beispiel.de.txt

Configure the Apache2 Rewrite Rule

By default, Plesk uses the Apache2 web server to serve the websites of the respective subscription, which is why only an adjustment to the .htaccess file is required here.

Insert the following at the beginning of the .htaccess file:

RewriteRule ^robots\.txt$ robots/%{HTTP_HOST}.txt [NS]

Configure the Plesk NGINX Rewrite Rule

If you have disabled NGINX proxy mode, the rewrite rule must be configured in the Additional NGINX Instructions. To do this, navigate to the Hosting and DNS settings for your subscription and open the Apache and nginx settings.

Add the following Nginx directive there and save the change by clicking the blue OK button.

\nAdditional Nginx Directives:

# multi-domain robots.txt
rewrite ^/robots\.txt$ /robots/$host.txt;

Configuring a Standalone NGINX Rewrite Rule

If you are running NGINX on its own, the rewrite rule must be added to the corresponding server block. Depending on how you have configured NGINX, you can find the corresponding configuration file under /etc/nginx/conf.d or in /etc/nginx/sites-available.

\nExample:

server {
 listen 80;
 listen [::]:80;

        server_name domain.tld;
 root /var/www/domain.tld/public;

 # multi-domain robots.txt
+ rewrite ^/robots\.txt$ /robots/$host.txt

 index index.html index.htm;
}

Next, the Nginx configuration must be reloaded with the following command:

systemctl reload nginx