404 Tech Support

Botopedia allows you to learn more about who is crawling your site

Incapsula is calling on the InfoSec community to band together to help solve some of the most common problems we all face on the Web. We believe that, as a community, we can crowd source a solution to fight back to defeat the worst bots and also recognize and reap the often-overlooked benefits of the good bots.

Their first tool to help in the fight is called Botopedia, a community-contributed directory to identify more information about bots, whether they be good or bad.

BotoPedia is a play on the stunning accomplishments of Wikipedia, a free worldwide directory of information on the widest variety of topics imaginable and maintained by a global community of volunteer experts. It’s a simple idea powerfully executed. We can do the same to protect our businesses and organizations on the Web. If we work together as a community, we can crowd source solutions to make the web safer for all of us and to also create a much better Internet user experience.

For example, I was able to look up the ‘mj12bot’ that I caught crawling 404TechSupport.com this morning and learn more about it from its own detailed entry. I have tried to look up various bots before, some with odd names and others seeming to impersonate Google’s bot, and a Google search didn’t always reveal the answer. It will be an improvement for the Internet community to have an authoritative source on some of this bot traffic that can crawl too aggressively or browse maliciously, trying SQL injections and posting spam comments.

Hear more about Incapsula’s goals with BotoPedia through the blog post announcing BotoPedia.