| User Agent | | Date Added |
| NetTransmitterStudio |  | 2/05/2006 1:16:10 |
| Requests same url over and over again with a few days in between. |
| notifyninja.com | | 28/12/2016 11:05:07 |
| url not working (404) |
| PiltdownMan | | 25/07/2005 1:25:14 |
The PiltdownMan robot is used to get a list of links from the search engines in our database. These links are followed, and the page that they refer is downloaded to get some statistics from them. The robot runs once a month, more or less, and visits the first 10 pages listed in every search engine, for a group of keywords.
To maintain a database of search engines, we needed an automated tool. That's why we began the creation of this robot. |
| ping.blo.gs | | 17/03/2011 6:27:24 |
|
| ping.wordblog.de | | 30/01/2012 11:49:36 |
|
| Pingdom | | 5/11/2006 22:37:19 |
| Web site monitoring |
| PostFavorites | | 3/05/2006 1:26:28 |
Yahoo Search My Web
- Save what you like to build your own personal web
- "Re-find" pages instantly when you need them again
- Share your personal web
- Better than bookmarks |
| PRTG Network Monitor | | 25/09/2018 16:14:09 |
|
| Random User Agent | | 14/11/2005 23:55:13 |
In each request the user agent changes.
on 2005.11.10 first request was for WebKnight http://www.aqtronix.com/webknight/ Got redirected to PageID=99 and that was second and final request. |
| Right Web Monitor | | 6/02/2004 1:08:40 |
| Right Web Monitor is a handy tool that lets you automatically check your favorite Internet resources and detect changes in their content. |
| rwws.com | | 6/11/2005 0:10:04 |
mail.rwws.com
requests WebKnight FAQ each time with different user agent (in 1 day), not the same interval, so it is not really monitoring.
Connecting to http://mail.rwws.com/ shows coming soon page
connecting to http://mail.rwws.com/ shows windows 2000 under construction site
very bad thing, doesn't feel right! |
| ScooperBot | | 13/07/2014 10:51:49 |
| CustomScoop provides an all-in-one monitoring solution that covers traditional and social media. Every account includes on-demand analytics, distribution tools, and personalized service. |
| Site Valet | | 31/07/2005 23:45:09 |
| a deluxe site monitoring and analysis service |
| SiteCheck | | 25/09/2015 11:59:27 |
|
| sitedownchecker.com crawler | | 25/09/2015 13:24:48 |
|
| SiteMonitor | | 9/10/2015 20:54:13 |
|
| SMPU | | 3/05/2006 21:41:15 |
Referer: http://www.norhaus.com/smpu.html
SMPU is a HTTP/1.0 URI parser and spider. The purpose of SMPU is resource collection and web site analysis.
- SMPU does not request any page more than once on any crawl.
- We will send you any information we have collected by request.
What does it do?
More often than not SMPU is used as a download utility, as it can recursively download some (or all) resources on a website. If you are seeing many requests that are all different then your server's contents are being either wholly or partially mirrored by the user.
If you are seeing occasional requests the chances are SMPU is being used as a spider to traverse the internet looking for something, and found a reference to your site.
What can I make it do?
Plenty of things, as a download util it's pretty good but it is more powerful as an analysis tool. You should familiarise yourself with the arguments for an idea of what it can do. It is free to download, and if you are a regular command prompt user it's a pretty useful tool to have around. |
| Snappy | | 5/11/2006 22:54:45 |
|
| SnykeBot | | 6/02/2004 1:09:07 |
| Snyke est un "agent" de surveillance de page Web |
| Surfbot | | 9/09/2006 0:32:04 |
|
| SurveyBot | | 7/02/2004 22:34:38 |
Each week SurveyBot will query websites for statistics and other useful information. This information goes into the creation of the Whois Source domain search engine.
Uses referrer: http://www.whois.sc/ |
| The Informant | | 10/02/2004 1:40:03 |
| The Informant robot continually checks the Web pages that are relevant to user queries. Users are notified of any new or updated pages. The robot runs daily, but the number of hits per site per day should be quite small, and these hits should be randomly distributed over several hours. Since the robot does not actually follow links (aside from those returned from the major search engines such as Lycos), it does not fall victim to the common looping problems. The robot will support the Robot Exclusion Standard by early December, 1996. |
| UptimeBot | | 7/04/2004 17:30:24 |
|
| Uptimerobot | | 20/12/2012 17:13:24 |
|
| URLy Warning | | 9/09/2006 0:44:06 |
Get Notified Instantly When Any Web Page Changes.
And See Exactly What's Been Changed! |
| vb wininet | | 9/02/2004 23:24:21 |
vbcode example to fetch page from web server.
Proof of monitoring available (log files)
NOT VERIFIED: probably also used in Igrabber Internet content grabber (http://www.aldostools.com/igrabber.html). Website did not exist anymore, google cache was still available:
A powerful internet content grabber that lets you automatically download web server contents off the net, extract specific information and integrate data from multiple pages for your later review, reduce download size or for transfer to portable devices, by simply placing all the 'Content Filters' you want fetched into one directory and pressing a button, using the command line or directly from the Favorites Links using the integration with the new AWeb (Aldo's Web Server).
Includes AWeb 1.5 letting browse filtered sites using filter templates. To filter images enter from MSIE address bar an URL like: http://localhost/filter=filter.dat/http://www.anyweb.com/. |
| Webclipping.com | | 25/07/2005 1:43:10 |
| WebClipping provides clients with news, information, and rumors from every key online source that impacts their business. With critical information collected and delivered to them, decision-makers can spot threats and opportunities in time to act effectively while saving hours of manual research. |
| WebEnhancer | | 11/09/2006 23:02:20 |
The WebEnhancer product range started life as joint venture between Informix, Oninit and METhornton, a web marketing company. With Informix supplying the development tools, Oninit the wherewithal to write the application and METhornton providing the viral marketing skills.
Primarily, an Online Bookmarker, i.e. a means of storing your personal bookmarks on a central server enabling them to be accessed from any web enabled device, the WebEnhancer product range offers so much more. By harnessing the extensibility of the underlying Informix database WebEnhancer is able to deliver a system not only able to store web pages, but also ftp sites, email addresses and telephone numbers and these are delivered to any web enabled device from your PC to your mobile to your toaster. Individual links can be password protected allowing the system to store private user information such as username and password for a particular site. The additional modules extend the product to cater for full contact management, file backup, diary information and Web based email. |
| Web-Monitoring | | 7/11/2013 14:32:14 |
|
| WebRescuer Spider | | 13/02/2004 19:20:45 |
Some monitoring tool, nothing found in Google search or usenet (anno 2004.02.13)
Requests root folder about every 62 minutes to 65 minutes from a single IP address only (not real monitoring as they say they do).
Note: AQTRONIX blocked this at firewall level and requests stopped a little more than a week later (2004.02.24 12:14:25). At this time results showed up in Google, meaning that it's a new service and they added domain names themselves to lure us into visiting them or even signing up. |
| WWWC | | 27/05/2004 15:47:51 |
| Requests home page every hour (with a HTTP HEAD request). Does not request robots.txt |
| Zealbot | | 8/02/2004 19:26:02 |
| LookSmart software, called Zealbot, visits all Web sites listed with LookSmart each week to ensure that they are still active, responsive sites |
