radhavallabh Posted December 21, 2015 Share Posted December 21, 2015 Hi, can you please help me block this MegaIndex.ru/2.0 crawler bot, It refers from static.11.156.9.5.clients.your-server.de Name: Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)IP address: 5.9.156.11User agent: Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler) I have used robots, htaccess and even spider.txt against it but it is still not blocking It keeps breaking down my store. Help would be deeply appreciated... Thanks in advance.... Link to comment Share on other sites More sharing options...
clustersolutions Posted December 21, 2015 Share Posted December 21, 2015 @@radhavallabh, how's it breaking down your store? if a bot is slowing your server to a crawl just imagine what happens when you have a lot of visitors... well...generally speaking I wouldn't reject an ip unless it is really necessary...but you can try typing this in a ssh shell... route add -net 5.9.156.11 netmask 255.255.255.255 reject It keeps breaking down my store. Link to comment Share on other sites More sharing options...
radhavallabh Posted December 21, 2015 Author Share Posted December 21, 2015 @@clustersolutions Thank you for providing a solution... Actually it is making huge amount of requests in matter of seconds which is giving high load to my server.... Link to comment Share on other sites More sharing options...
Jack_mcs Posted December 21, 2015 Share Posted December 21, 2015 @@radhavallabh It's called a data skimmer and they can bring a site/server to its knees if there are enough of them. They don't serve any purpose so you should block them as they are found. There's always more than one by the way. The best way to identify them is through the servers access log but you can see them with View Counter also. For this one, you should block their own range of IP's or they will just return with a different one. To do that, use this in your roots .htaccess file - deny from 5.9.0.0/16 - or just 5.9.0.0/16 from your control panel. Support Links: For Hire: Contact me for anything you need help with for your shop: upgrading, hosting, repairs, code written, etc. All of My Addons Get the latest versions of my addons Recommended SEO Addons Link to comment Share on other sites More sharing options...
radhavallabh Posted December 22, 2015 Author Share Posted December 22, 2015 Thank you@@Jack_mcs For your valuable reply I totally agree with you...... I am going ahead and adding this module as it seems to be very helpful I have added the above in my htaccess and now they are blocked permanently... :) Thank you once again..... Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.