|
|
|
|
|
User-Agents Database |
User Agent Details of Harvest
| Name: | Harvest |
| URL: | http://harvest.cs.colorado.edu/ |
| User Agent Key: | |
| User Agents: | |
| IP Ranges: | |
| Adheres robots.txt: | N/A |
| Adheres META tag: | N/A |
| Type(s): |
SearchEngine
|
| Verified: | | | Info: | Harvest's motivation is to index community- or topic- specific collections, rather than to locate and index all HTML objects that can be found. Also, Harvest allows users to control the enumeration several ways, including stop lists and depth and count limits. Therefore, Harvest provides a much more controlled way of indexing the Web than is typical of robots. Pauses 1 second between requests (by default). | | Date Added: | 13/02/2004 0:58:49 | | |  |
User Agents - Search
|
|
