|
|
|
|
|
User-Agents Database |
 |
User Agent Details of Scrapy
| Name: | Scrapy |
| URL: | http://scrapy.org |
| User Agent Key: | Scrapy/ |
| User Agents: | Scrapy/0.15 (+http://scrapy.org)
Scrapy/0.17.0-222-ge1f4144 (+http://scrapy.org)
|
| IP Ranges: | |
| Adheres robots.txt: | No |
| Adheres META tag: | N/A |
| Type(s): |
DataMiningPublic
OtherBad
|
| Verified: | | | Info: | Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
fell for bad bot trap!
| | Date Added: | 23/05/2012 14:41:28 | | |  |
User Agents - Search
|
|
