Choose crawlers
Enter * or named crawlers such as Googlebot.
FREE ROBOTS.TXT GENERATOR
Free Robots.txt Generator
Build the standard file that tells search crawlers which parts of your site they may crawl.
Your rules and URLs are never uploaded or stored.
- 1Enter crawl rules
- 2Review the file
- 3Publish the file
1 · Enter crawl rules
Build crawl rules
Set allowed and blocked paths for each crawler group.
2 · Review the file
robots.txt preview
Enter rules on the left to preview the file here.
Upload the finished UTF-8 plain-text file to the site root at /robots.txt.
FREE ROBOTS.TXT GUIDE
Prepare crawl rules in three steps
Create the file for free without an account or installation.Set paths
Add Allow and Disallow paths, then review the generated file.
Publish at root
Place the downloaded file at your site’s /robots.txt URL.
CRAWLER CONTROL
Know these rules before publishing
Crawl control, indexing, and security are different concerns.
Grouped directives
Keep Allow and Disallow rules under the intended User-agent group.
Specific matches
Crawlers generally apply the most specific matching path rule.
Do not hide secrets
robots.txt is public and must not be used to protect private paths or data.
FREE ROBOTS.TXT GENERATOR FAQ
Robots.txt generator FAQ
Check free use, file placement, and scope.1Is the robots.txt generator free?
Yes. It is free with no account or usage limit, and everything stays in your browser.
2Where do I upload robots.txt?
Place it at the top-level path of the host, such as https://example.com/robots.txt.
3Does Disallow remove a page from search?
No. It limits crawling but does not guarantee deindexing or privacy.
4Can I add sitemap URLs?
Yes. You can add multiple complete http or https sitemap URLs.