LinkScrubber Crawler

Q: Why have you crawled our site?
A: LinkScrubber only crawls sites on demand when requested by our customers and checks external links from those sites. If you're getting crawled and not link checked, without permission, please write to "support" @ this domain and tell us your URL and we'll investigate.

---

Q: What are the link checker UAs (user agents)
A: The crawler is "LinkScubber crawler" and the external link checker is "LinkScrubber link checker"

---

Q: Does LinkScrubber honor robots.txt?
A: The "LinkScrubber crawler" does honor basic robots.txt, none of the fancy Google extensions at this time, but the "LinkScrubber link checker" does not since it isn't a crawler, it's just a one link page validator.

---

Q: Do I have to crawl my site every time I want to check external links?
A: No! You can crawl any time, but it's not required as we save the external link list once your site is crawled. You may recheck your previous copy of your external links at any time, or recrawl your site to refresh that list.