| | | |
|---|
| -- | Search Engine | |
AAfter | AAfter looks like a legit search engine. () | Search Engine | |
AboutUsBot | AboutUsBot is used by the About Us website to determine the contents, aspect, logo and owner of a website. It is legit. | Search Engine | |
aiHitBot | aiHitBot, seems to be a legit search engine. () | Search Engine | |
ia_archiver | Alexa ia_archiver () | Search Engine | |
almaden | almaden, Unstructured Information Analysis and Search @ IBM () | Search Engine | |
Scooter | AltavistaBot, Altavista () | Search Engine | |
aport | Aport () | Search Engine | |
appie | Appie-spider/Walhello () | Search Engine | |
ApptusBot | ApptusBot is the Apptus crawler bot, some business driven search engine. () | Search Engine | |
Ask Jeeves | Ask Jeeves, Teoma () | Search Engine | |
askpeter_bot | Ask Peter is a German based search engine () | Search Engine | |
ASPseek | ASPSeek () | Search Engine | |
Baiduspider | Baidu search engine web crawler. Not always respectful of robots.txt, sometimes a bit pushy as well. () | Search Engine | |
BecomeBot | BecomeBot shopping search () | Search Engine | |
Blazer | Blazer Browser, Sharp Zaurus () | Search Engine | |
CatchBot | CatchBot is a business page crawler. They claim to resell information for companies, academics and various professional fields. Bot behaves correctly. | Search Engine | |
abby/ | Ellerdale determines trends through the semantic web, usually through gathering recent Twits or Facebook entries. () | Search Engine | |
ExaBot | Exlead Exabot () | Search Engine | |
facebookexternalhit | Facebook External Hit () | Search Engine | |
fast | FAST-WebCrawler () | Search Engine | |
sitedossier.com | Featuring sitedossier.com as a referer, the IP 69.71.222.186 seems to check for websites recently crawled by one of their competitors, domaintools. Seems harmless. () | Search Engine | |
feedfetcher | Feedfetcher Google, gathers news feeds from websites () | Search Engine | |
Feedtrace-bot | Feedtrace-bot makes a list of the most popular twitter feeds (parses the most recent feeds all time round). () | Search Engine | |
ftxbrowser | ftxBrowser, Windows CE () | Search Engine | |
gais | Gais () | Search Engine | |
Gigabot | Gigablast's Gigabot () | Search Engine | |
Mediapartners | Google AdSense () | Search Engine | |
Google Desktop | Google Desktop is a desktop data manager/search. It should be harmless. () | Search Engine | |
googlebot | Googlebot () | Search Engine | |
ichiro | ichiro @ Goo Japan / Inktomi () | Search Engine | |
IconSurf | Icon Surf () | Search Engine | |
ICRA_Semantic_spider | ICRA semantic spider, Internet Content Rating association. () | Search Engine | |
infoseek | InfoSeek () | Search Engine | |
JS-Kit | JS-Kit is a blabla software for blogs. It usually connects here and there to promote their stuff through curiosity. For that reason no URL is provided here. | Search Engine | |
Linguee Bot | Linguee Bot is a legit search engine bot. However it WILL get banned from your Beamreactor enabled website for its extreme crawling speed with the argument 'flood'. () | Search Engine | |
LinkWalker | LinkWalker () | Search Engine | |
grub | Looksmart/Grub () | Search Engine | |
Mail.RU | mail.ru (Поиск@mail.ru) is tied to the mail.ru search engine () | Search Engine | |
MJ12bot | Majestic-12 distributed search engine bot () | Search Engine | |
MetaQuerier | MetaQuerier (University of Illinois in Urbana-Champaign) () | Search Engine | |
bingbot | Microsoft Bing () | Search Engine | |
Media Center PC 5.0 | Microsoft EnhanceIE enables Windows users to mod their useragents and emulate other user agents (!?) () | Search Engine | |
MLbot | MLBot is a mp3/video crawler. The true purpose of MLbot is undisclosed but might be related to piracy protection. This robot is fairly clean. () | Search Engine | |
Yahoo-MMCrawler | MM Crawler, seeks for images on the www. () | Search Engine | |
MOT-A768 | Motorola A768 browser client. Might be fairly harmless. | Search Engine | |
msnbot | MSN Search Crawler () | Search Engine | |
MSTV | MSTV WebTV () | Search Engine | |
MyIE2 | MyIE2 @ turkey? | Search Engine | |
netcraft | Netcraft () | Search Engine | |
Naverbot | NHN Corp bot/Naver.com () | Search Engine | |
Ocelli | Ocelli Engineering search () | Search Engine | |
OnetSzukaj | OnetSzukaj () | Search Engine | |
avantgo | PalmOs AvantGo () | Search Engine | |
PSbot | Picsearch web crawler () | Search Engine | |
plucker | Plucker Browser, Windows CE () | Search Engine | |
Plukkie | Plukkie: a search engine robot, fairly harmless. () | Search Engine | |
pompos | Pompos () | Search Engine | |
Moo | qsdfqs () | Search Engine | |
quepasacreep | QuePasaCreep () | Search Engine | |
StackRambler | Rambler search robot () | Search Engine | |
RSScache | RSS Cache website bandwith saver () | Search Engine | |
SapphireWebCrawler | SapphireWebCrawler crawls for a computer science project from Carnegie Mellon university. | Search Engine | |
scrubby | scrubby () | Search Engine | |
Shim | Shim Crawler (University of Tokyo) () | Search Engine | |
slurp | Slurp () | Search Engine | |
spbot | spbot; "we just want to find out to which web pages you link to" () | Search Engine | |
Speedy Spider | Speedy Spider is a part of the highly advanced search engine Entireweb.com, that was developed in Halmstad, Sweden during 1998-2000. () | Search Engine | |
sproose | Sproose Crawler () | Search Engine | |
Apple-PubSub | The PubSub client is checking your RSS for an Apple computer owner! Don't remove or block this client/IP. () | Search Engine | |
bnf.fr_bot | This robot comes from the National French Library. It makes a web archive of your website for various reasons and may, or may not respect robots.txt according to its settings. Harmless nonetheless. () | Search Engine | |
seznambot | Tied to the Seznam Czech search engine. () | Search Engine | |
turnitin | Turn It In () | Search Engine | |
Twiceler | Twiceler is the legit Cuil search engine crawler () | Search Engine | |
Twingly Recon | Twingly Recon is a RSS parser, focused towards blogs. Usually triggered with syndication tools, such as facebook / twitter post third party APIs. () | Search Engine | |
Twitturls | Twitter URL parser. Someone linked your content to twitter. () | Search Engine | |
Vagabondo | Vagabondo () | Search Engine | |
VideoSurf_bot | VideoSurf bot looks for videos. It uses social webs to parse URLs to visit, so its visit might be related to some of your website data being posted on Twitter or FB | Search Engine | |
VoilaBot | VoilaBot is from the Voila search engine, owned by The "France Telecom - Orange" group. Basically harmless. () | Search Engine | |
Jigsaw | W3C CSS validator - JFouffa () | Search Engine | |
W3C_Validator | W3C Validator () | Search Engine | |
WMP | Windows Media Player | Search Engine | |
Xenu Link Sleuth | Xenu Link Sleuth validates your website for dead links () | Search Engine | |
Yahoo! Mindset | Yahoo Mindset () | Search Engine | |
Yandex | Yandex. I at the end refers to search. H looks for mirror copies, P for images, F for favicons, D for Yandex declared websites, B for RSS () | Search Engine | |
YandexSomething | YandexSomething searches for news related RSS feeds for their news system. () | Search Engine | |
zyborg | Zyborg () | Search Engine | |