VerifyHostname verifies if the tls.ConnectionState certificate matches the hostname.

ErrNotMatchOrigin indicates that the end location is external to the host we were originally looking up.

ErrTooManyRedirects indicates that the requested origin redirected more than a considerable amount of times, indicating there may be a redirect loop.

Crawler is the higher level struct which wraps the entire threaded crawl process.

CrawlerConfig is the configuration which changes Crawler.

CustomClient is the state for our custom http wrapper, which houses the needed data to be able to rewrite the outgoing request during redirects.

CustomResponse is the wrapped response from http.Client.Do() which also includes a timer of how long the request took, and a few other minor extras.

Domain represents a url we need to fetch, including the items needed to fetch said url.

FetchResult -- struct returned by Crawl() to represent the entire crawl process.

HostnameError appears when an invalid SSL certificate is supplied.

Resource represents a single entity of many within a given crawl.

Response represents the data for the HTTP-based request, closely matching http.Response.

TLSResponse is the TLS/SSL handshake response and certificate information.