DDOS’ed by 80legs.com spider?

Just got some heavy traffic from a friggin’ spider with this useragent:

Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html) Gecko/2008032620

As changing robots.txt did not help immediately, more drastic measures were needed, so I added this tidbit to .htaccess to stop the fuckers:

SetEnvIfNoCase ^User-Agent$ .*(80legs) HTTP_SAFE_BADBOT
Deny from env=HTTP_SAFE_BADBOT

And now back to me just relaxing and having a bath! ;-)

8 thoughts on “DDOS’ed by 80legs.com spider?

  1. Markus

    Thanks for the hint!

    80legs DDOSed my server too. It took some time to find out the culprit, as the server was inaccessible due to high load for several hours.
    It’s Friday evening – if I didn’t discover the outage, the server (with lots of virtual hosts of paying customers) would have been down for the next 3 days!

    Wo pays the damage?

    80legs should go to hell with its botnet!

    Reply
  2. Cevius

    Ah, an effective and efficient solution. All my other searches for a fix to the 80legs problem had me adding entries to modsec2 or haproxy, and while those work, its a bit much for some users.

    No more bot problems now! :D

    Reply
  3. Frederique

    This 80leg bot is just an example of the many rogue bots around these days. These bots have the capability to take a whole server down by their agressive method of indexing. It should be treated as illegal almost!

    Reply
  4. Aaron

    Hi Guys,
    Just noticed this post on this bad bot.
    If ya on APACHE
    Put the below in your HTACCESS,

    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} ^.*80legs.*$ [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^.*008.*$ [NC]
    RewriteRule ^.*.* http://www.80legs.com/ [L,R=permanent]

    Rather than blocking and putting up a white flag eg 403 or whatever ya server responds with,
    Kick them back to there own site to DDOS THEMSELVES with there on bad bots.
    Trust me it works very well when there own servers fall over. :)

    Send bad Bots back to themselves!

    Double check my directive as it depends on how ya have HTAACCESS setup, but that works a treat on ourservers. We havnt had a prob with this BOT, but we use this to punish other BAD BOTS and it frees up our servers from throwing up error pages which still slow ya own server as its done before your server.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *