Crawl a website, depth-first, listing all unique paths found.

Change HTTP user agent.

Load a URL and list all documents found.

Look for unlisted files on a domain.

Load a URL and list all links found.

Search through a URL and find HTML comments.

Search through a URL and find mailto links with email addresses.

Perform an HTTP HEAD request on a URL and print out headers.

Perform an HTTP request to load a page and search for a string.