|
|
|
|
|
User-Agents Database |
User Agent Details of ShopWiki
| Name: | ShopWiki |
| URL: | http://www.shopwiki.com/wiki/Help:Bot |
| User Agent Key: | ShopWiki |
| User Agents: | ShopWiki/1.0 ( +http://www.shopwiki.com/wiki/Help:Bot)
|
| IP Ranges: | |
| Adheres robots.txt: | Yes |
| Adheres META tag: | N/A |
| Type(s): |
IndexingTool
|
| Verified: | | | Info: | ShopWiki finds products using Web crawlers similar to other search engines. This means we look into a Web site's domain for all robots.txt files, which tell our crawlers which files it may search. All Web sites have the ability to define what parts of their domain are off-limits to specific robot user agents. ShopWiki respects and obeys all robots.txt files. | | Date Added: | 23/02/2009 0:49:32 | | |  |
User Agents - Search
|
|
