Robots.txt Generator

It must live at the root of your domain, served at https://yourdomain.com/robots.txt. Crawlers only look there — a robots.txt in a subfolder is ignored.

Disallow tells crawlers not to crawl a path; Allow re-permits a more specific path inside a disallowed one. An empty Disallow line means nothing is blocked, so the whole site is crawlable.

Not reliably. Disallow stops crawling, but a blocked URL can still appear in search results if other pages link to it. To keep a page out of the index, allow crawling and add a noindex meta tag, or protect it behind login.

No. The file is built entirely in your browser from what you type. Nothing is uploaded — copy or download the result and host it yourself.