Robots.txt Tester

Test robots.txt rules

Paste a robots.txt or fetch from a domain, list URLs to check, and see which user-agents can crawl them.