Robots.txt Tester
Test if a URL is allowed or blocked by a website's robots.txt file. Verify crawl rules for different user agents.
✨Sign in to use this tool — it's free!
About Robots.txt Tester
Test if a URL is allowed or blocked by a website's robots.txt file. Verify crawl rules for different user agents.
How to use
- Fill in the required fields in the form above.
- Click "Run Tool" to process your request.
- If you're not signed in, you'll be prompted to create a free account.
- View, copy, or download your results instantly.