Robots.txt generator

Allow what should rank, block what shouldn't, and point crawlers at your sitemap.

Sitemap URL

Block admin pagesBlock login and signupBlock cart and checkoutBlock internal search resultsBlock /api/Block AI training crawlers (keeps ChatGPT search and Google AI answers)

Extra rules

Optional, added as written.

robots.txt

User-agent: *
Disallow: /admin/
Disallow: /login
Disallow: /signup
Disallow: /search
Disallow: /api/

Sitemap: https://example.com/sitemap.xml

Want this done every week?

Drumbly reads your site, plans the week, writes in your voice and shows what each piece earned.

Start free