Free robots.txt generator
Tell search engines what to crawl. Pick a platform, add paths and a sitemap, then download your robots.txt file.
How to use
- Choose your platform or start with Custom.
- Add any paths you want to block and your sitemap URL.
- Download the file and upload it to the root of your website.
Frequently asked questions
Where does robots.txt go?
In the root folder of your site, so it opens at yourdomain.com/robots.txt.
Does robots.txt hide pages from Google?
No. It asks crawlers not to visit a page, but a blocked page can still be listed. Use a noindex tag to keep a page out of search results.
Want a site that ranks?
We build SEO into every website.
Websites, WordPress, Shopify, WooCommerce, SEO, domains and hosting.
Contact us on WhatsAppVisit TechnoWolves