ConveraCrawler is an experimental web crawler under development since April 2004. ConveraCrawler is owned and operated by Convera Corporation as part of an effort to develop a state-of-the-art searchable Web index. The information gathered by ConveraCrawler will be indexed and made accessible via one or more publicly-accessible web sites in the near future.
We intend for ConveraCrawler to be a polite and unobtrusive netizen. From January 2005 through March 2005 we will be rapidly increasing the size of our Web index towards an initial target of 4 billion pages. As a result, during this period we may occasionally make page requests at a rate exceeding commonly accepted standards for unobtrusive crawling. We apologize for this intrusion and will return to a more discreet crawl rate at the earliest possible date.
You may control ConveraCrawler's access to your site by updating your robots.txt file.