Robots.txt Checker

Fetch and analyze any website's robots.txt crawl rules.