|User Agent|| ||Verified||Date Added|
|Nutch Engine||25/06/2011 13:25:41|
|Nutch Firefox||9/02/2013 13:09:54|
|Nutch spiderman||8/12/2009 23:36:29|
|When we crawl to populate our index, we advertise the "User-agent" string "NutchOrg". If you see the agent "Nutch" or "NutchCVS", that's probably a developer testing a new version of our robot, or someone running their own instance.|
|Squid - anonymize_headers|
|Started in 1995 to provide a comprehensive index to WWW pages within New Zealand. Now also used in Malaysia and other countries.|
|Objects Search aims to provide the users with best search results, free website submission and unbiased website ranking|
|Cobion Germany Brand Protection Services|
Cobion also does web content filtering, however, this bot is (was?) used for brand protection.
Cobion is now part of ISS.
|The robot takes high-level queries, breaks them down into multiple web requests, and answers them by combining disparate data gathered in one minute from numerous web sites, or from the robots cache.|
The robot is a descendant of Rodney, an earlier project at the University of Washington.
|Ocelli is a Web crawler owned and operated by GlobalSpec®, the leading specialized search engine and information resource for the engineering community. Ocelli's mission is to find and index web pages for The Engineering Websm from GlobalSpec, a unique slice of the World Wide Web focusing solely on engineering and technical content.|
|ODI3 Navigator||11/11/2009 0:03:06|
|ODP links test||7/04/2004 17:24:29|