| User Agent | | Verified | Date Added |
| Accoona-AI-Agent | [edit] | Yes | 5/11/2005 23:33:18 |
|
| Acoon | [edit] | Yes | 10/02/2004 1:12:10 |
|
| Ahoy! The Homepage Finder | [edit] | | 10/02/2004 20:36:27 |
| Ahoy! is an ongoing research project at the University of Washington for finding personal Homepages. |
| Alexa/Internet Archive | [edit] | Yes | 8/02/2004 19:47:49 |
| Alexa.com and archive.org (building the Internet Archive) |
| AltaVista Scooter | [edit] | Yes | 8/02/2004 16:47:05 |
| Search engine crawler for Altavista |
| ananzi | [edit] | | 11/02/2004 22:43:44 |
|
| AnswerBus | [edit] | Yes | 8/02/2004 15:11:03 |
|
| Anthill | [edit] | | 10/02/2004 20:38:54 |
| Anthill is used to gather priceinformation automatically from online stores.support for international versions. |
| antibot | [edit] | Yes | 8/02/2004 19:33:11 |
| Le robot utilisé par AFS est nommé AntiBot. Il parcourt l'Internet et met à jour les bases de données d'AFS tout en respectant la norme d'exclusion des robots sans surcharger les sites en espaçant ses requêtes. |
| Anzwers Crawl | [edit] | | 10/02/2004 1:15:24 |
|
| Aport | [edit] | Yes | 8/02/2004 15:13:03 |
|
| Appie spider | [edit] | Yes | 8/02/2004 19:36:24 |
| The Walhello search engine is based on a spider. The walhello spider (appie) follows links to web pages on the Internet and stores every new page found in the Walhello database. You can search the database via keywords. An advanced algoritm sorts pages, to ensure that the most relevant pages are shown first. |
| Araneo | [edit] | Yes | 10/02/2004 20:46:31 |
| Araneo is a web robot developed for crawling and indexing web pages written in the international language Esperanto. The database will be used to build a web search engine and auxiliary services to be published at esperantisto.net. |
| arks | [edit] | | 10/02/2004 20:57:37 |
| The Arks robot is used to build the database for the dpsindia/lawvistas.com search service . The robot runs weekly, and visits sites in a random order |
| Art-Online | [edit] | Yes | 8/02/2004 15:17:00 |
|
| Ask Jeeves/Teoma | [edit] | Yes | 8/02/2004 15:20:59 |
| Developed by Direct Hit Technologies which was aquired by Ask Jeeves in 2000. |
| ASpider (Associative Spider) | [edit] | | 10/02/2004 21:01:11 |
| ASpider is a CGI script that searches the web for keywords given by the user through a form. |
| Asterias | [edit] | Yes | 8/02/2004 19:38:37 |
| Singingfish uses an innovative, multimedia-oriented spider or robot to crawl the Web, looking for media files to index for the Singingfish Streambase. Our spider, called Asterias®, obeys spider Netiquette rules: It checks the Robot Exclusion File and it avoids hitting each site too frequently. Asterias® identifies multimedia (Real Media, Windows Media, Quicktime, MP3 and Flash) files and adds them to our index. Sometimes this process results in many sequential accesses to a Site. While this process results in a complete index, the behavior is atypical of other spiders found on the web. |
| ATN Worldwide | [edit] | | 10/02/2004 23:52:53 |
| The ATN robot is used to build the database for the AllThatNet search service operated by All That Net. The robot runs weekly, and visits sites in a random order. |
| Atomz.com Search Robot | [edit] | Yes | 10/02/2004 23:54:53 |
| Robot used for web site search service. |
| BBot | [edit] | Yes | 11/02/2004 0:02:44 |
| Mainly intended for site level search, sometimes set loose. |
| BebopBot | [edit] | | 1/12/2008 0:46:59 |
| BebopBot is an experimental web crawler under development since January 2007. BebopBot is owned and operated by A Passion for Jazz!™ as part of an effort to develop a niche based searchable Web index. We are not attempting to steal any copyrighted information from your site and will not be re-distributing your content. We will only be allowing users to find your website more easily. The information gathered will be indexed and made accessible via one or more publicly accessible web sites in the near future. |
| BecomeBot | [edit] | | 17/01/2005 10:17:49 |
| BecomeBot is the user-agent for Become's new web crawler. Become is crawling the web to build a next generation search engine. |
| boitho.com | [edit] | Yes | 5/01/2006 10:05:08 |
| The boitho.com-robot is a distributed crawler that downloads web pages to build the database used by boitho.com to search in. |
