Configure crawl directives with robots.txt in Shopware
Set out the crawl directives you want for your shop. FroggShop produces robots.txt content from the content providers and configuration applicable to the site.
Define directives for the relevant site
The configuration-based provider reads the robots.txt content associated with the sales channel. Rules for crawlers can therefore be prepared with the domains and environments actually configured in mind.
Control the rule that is ultimately served
The output uses the first applicable provider that returns non-empty content, in priority order. It does not automatically merge all content. Teams must therefore check the file actually accessible on each domain.
Do not confuse directives with access protection
Crawl directives are not a confidentiality mechanism. Private environments and sensitive data require their own access controls.
Robots.txt content must be consistent with the site’s other SEO decisions. Its configuration guarantees neither the removal of an already known page nor its indexing.
What is the scope of your project?
Tell us about your needs and the tools you use. We can clarify the possibilities, conditions, alternatives and scope of work suited to your project.
Discuss your needs