Build a valid robots.txt file with custom allow/disallow rules and a sitemap reference.
A correctly formatted robots.txt tells search engines what they can and can't crawl. Robots.txt Generator builds one with a live preview, so you can see exactly what you're about to publish before you copy or download it.
Enter your website URL and sitemap URL, add Allow/Disallow rules as needed, then copy the generated text or download it directly as robots.txt.
Where do I put the downloaded file?
robots.txt must be placed at the root of your domain, e.g. yoursite.com/robots.txt, for search engines to find it.
What happens if I don't add any rules?
With no rules, the generator defaults to allowing all crawling — a safe, common default for most sites.