Home / Robots.txt Generator
Robots.txt Generator
Build a valid robots.txt with crawl rules, disallowed paths and a sitemap reference — no syntax errors.
Configure your rules
Your robots.txt will appear here.
What is robots.txt for?
robots.txt tells well-behaved crawlers which parts of your site they may or may not request. It lives at the root of your domain, e.g. https://example.com/robots.txt, and is checked before a crawler fetches any other page.
Common mistakes to avoid
Don't use robots.txt to hide pages from search results — a disallowed page can still be indexed if it's linked elsewhere; use a noindex meta tag for that instead. Also make sure you don't accidentally disallow /, which blocks your entire site.
Where do I upload robots.txt?
Place it in the root directory of your domain via Hostinger's File Manager or FTP so it's reachable at yourdomain.com/robots.txt.