|
|
|
|
|
User-Agents Database |
User Agent Details of NerdyBot
| Name: | NerdyBot |
| URL: | http://nerdybot.com/ |
| User Agent Key: | NerdyBot |
| User Agents: | |
| IP Ranges: | |
| Adheres robots.txt: | Yes |
| Adheres META tag: | N/A |
| Type(s): |
SearchEngine
|
| Verified: | | | Info: | NerdyBot is a web crawler used by NerdyData
Our search engine is different from search engines you've used before. We've visited over 140 million homepages and collected terabytes of HTML, Javascript, and CSS code. We allow users to search the full source code of web pages in our index, not just the plaintext.
NerdyBot runs on a cluster of several hundred servers, which is why you may see our web crawler access your site from many different IP addresses. | | Date Added: | 12/10/2015 16:44:53 | | |  |
User Agents - Search
|
|
