Free robots.txt generator

Tell search engines what to crawl. Pick a platform, add paths and a sitemap, then download your robots.txt file.

How to use

  1. Choose your platform or start with Custom.
  2. Add any paths you want to block and your sitemap URL.
  3. Download the file and upload it to the root of your website.

Frequently asked questions

Where does robots.txt go?

In the root folder of your site, so it opens at yourdomain.com/robots.txt.

Does robots.txt hide pages from Google?

No. It asks crawlers not to visit a page, but a blocked page can still be listed. Use a noindex tag to keep a page out of search results.

Want a site that ranks?

We build SEO into every website.

Websites, WordPress, Shopify, WooCommerce, SEO, domains and hosting.

Visit TechnoWolves

More free tools

Business Name & Tagline Generator Password Generator Meta Tag & Open Graph Generator Privacy Policy Generator Terms & Conditions Generator Disclaimer Generator Lorem Ipsum Generator Slug & URL Generator QR Code Generator WhatsApp Link Generator UTM Link Builder