|
|
|
|
|
User-Agents Database |
User Agent Details of special_archiver
| Name: | special_archiver |
| URL: | http://www.loc.gov/minerva/crawl.html |
| User Agent Key: | special_archiver/ |
| User Agents: | Mozilla/5.0 (compatible; special_archiver/1.5.0 +http://www.loc.gov/minerva/crawl.html)
Mozilla/5.0 (compatible; special_archiver/3.1.1 +http://www.archive.org/details/archive.org_bot)
|
| IP Ranges: | |
| Adheres robots.txt: | No |
| Adheres META tag: | N/A |
| Type(s): |
DataMiningPublic
OtherBad
|
| Verified: | | | Info: | bypasses robots.txt and we did not receive an e-mail to notice about the crawl as they say they do on their site!
Library of Congress Web Archives | | Date Added: | 7/09/2011 14:24:41 | | |  |
User Agents - Search
|
|
