OneCalcApp LogoOneCalcAppConvert. Calculate. Simplify.
SEO & Website Tools

🤖 Robots.txt Generator

Build a simple robots.txt file with user-agent, allow, disallow, crawl-delay and sitemap directives.

Build robots.txt

robots.txt

User-agent: *
Allow: /
Disallow: /account
Disallow: /result/

Sitemap: https://example.com/sitemap.xml

How to use Robots.txt Generator

  1. Choose the crawler user-agent. Use * only when the rule should apply to all compliant crawlers.
  2. Add only paths that genuinely need Allow or Disallow directives.
  3. Add the absolute sitemap URL if the site has an XML sitemap.
  4. Review the generated text carefully before placing it at /robots.txt on the website root.

Practical SEO notes

  • robots.txt controls crawling, not guaranteed indexing. A URL can still be known from links even when crawling is disallowed.
  • Do not use robots.txt to protect confidential data. Authentication and access controls are required for private content.

Privacy: this tool runs in your browser. OneCalcApp does not need to upload the text you enter here to a third-party SEO service.

Frequently asked questions

Does Disallow remove a page from Google?

Not reliably. Disallow is a crawl instruction. For pages that search engines can crawl, a noindex directive is the normal way to request removal from search results.

Where should robots.txt be hosted?

It should be available at the root of the exact host, for example https://example.com/robots.txt.

Can I list more than one sitemap?

Yes. A robots.txt file can contain multiple Sitemap directives using absolute sitemap URLs.