|
|
|
|
|
User-Agents Database |
User Agent Details of URL Spider Pro
| Name: | URL Spider Pro |
| URL: | http://www.innerprise.net/usp-productinfo.aspx |
| User Agent Key: | URL_Spider_Pro |
| User Agents: | |
| IP Ranges: | |
| Adheres robots.txt: | N/A |
| Adheres META tag: | N/A |
| Type(s): |
IndexingTool
|
| Verified: | Yes | | Info: | URL Spider Pro is designed for creating a small Google-like search engine. You simply configure (through keyword and domain filters) the spider for the type of information it should index and it will crawl through the Web collecting only documents matching that configuration. You can also instruct it to crawl through one or more specific domains, making it easy to add site search functionality to your Web site. | | Date Added: | 7/02/2004 23:51:44 | | |  |
User Agents - Search
|
|
