Tools · SEO & marketing

Robots.txt analyzer

Parse a domain's robots.txt and simulate a crawler per User-Agent against a given path.

Domain

User-Agent

Test path

The link encodes your inputs in the URL fragment. Nothing is uploaded.