What is Googlebot Verifier?
AQTRONiX Googlebot Verifier is an application firewall for IIS with only one purpose. It filters fake and spoofed Googlebots while allowing Google to continue crawling the site. It is an extension to WebKnight but can run standalone also.
How it works
Googlebot Verifier uses a reverse and forward DNS lookup for verification when a Googlebot connects to your website. Even when Googlebot IP addresses change in the future, the lookup verifies the IP address belonging to a real Googlebot as described here.
Features
- Deny access to fake Googlebots
No more robots and SEO marketeers pretending to be Googlebot on your website.
- Spoofed User-Agents
When visitors are using plugins like the User Agent Switcher for Firefox to pretend to be Googlebot, they will be blocked as their IP address is not registered by Google as a Googlebot.
- Prevent hacking and DDoS attacks
A lot of websites whitelist Googlebots because they don't want to endanger their PageRank. Botnets and hackers are starting to identify themselves as Googlebot to avoid detection. With Googlebot Verifier you can combat these attacks.
- Performance
A lookup is done only once for each Googlebot IP. Caching the DNS responses for 72 hours improves perfomance. No delay in crawling for Google. The forward lookup is done only when the reverse lookup matches.
- Supports load balancers, CDN and reverse proxies
You can set the server variable to be used (HTTP_X_FORWARDED_FOR or HTTP_TRUE_CLIENT...) so Googlebot Verifier is using the correct IP address and is not trying to verify your load balancer or reverse proxy.
- IPv6 ready
Not only does it verify IPv4 addresses, but also IPv6 addresses are supported.
IIS Manager
Googlebot Verifier is built for IIS and integrates seamlessly with IIS Manager. IIS 7, 8, 10 and later are supported.
