Free Online Robots.txt Generator & Tester – Create SEO-Friendly Robots.txt for Blogger and WordPress

Robots.txt Generator & Tester

Create a clean robots.txt file and test your rules • 100% free • Runs in your browser

1. Start with a preset

Pick a platform to fill the fields below. You can edit everything afterwards.

Preset Blogger (blocks /search pages) WordPress Online store (cart, checkout, filters) Allow everything Block everything (staging site) Your website URL (used to build the sitemap line)

2. Rules for all crawlers (User-agent: *)

Put one path per line. Paths should start with / and can use * and $ wildcards.

Default access Use the allow / disallow lists below Block the whole site (Disallow: /) Disallow paths (blocked) Allow paths (exceptions) Crawl-delay in seconds (optional, ignored by Google) Give Google AdSense bot full access (Mediapartners-Google)

3. Sitemaps

One full URL per line. These are added at the bottom of the file.

4. Block specific bots (optional)

Each ticked bot gets its own group with Disallow: /. Compliant crawlers will honor it.

5. Extra custom lines (optional)

Anything typed here is added exactly as written, before the sitemap lines.

Generated robots.txt 0 lines

6. Test a URL against the file above

Enter a path and a bot name to see whether it would be crawled.

Path to test Bot name

All processing happens in your browser. Upload the file to the root of your domain (for example /robots.txt). In Blogger, paste the text into Settings > Crawlers and indexing > Custom robots.txt.