Fredi Posted July 9, 2018 Share Posted July 9, 2018 There was such software. As described, this affects the osCommerce. Who knows the details and how to protect yourself from this? ICG-AutoExploiterBoT Support forum for osCommerce in russian language - from Ashkelon. Support since 2002.Best regards, Fredi Link to comment Share on other sites More sharing options...
♥JcMagpie Posted July 9, 2018 Share Posted July 9, 2018 No nothing new its been around for a while, you should be activly blocking these in your .htaccess. Just add it to your list. Here is a list of commen bots to block. //Block bad bots RewriteEngine On RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR] RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:[email protected] [OR] RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR] RewriteCond %{HTTP_USER_AGENT} ^Custo [OR] RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR] RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR] RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR] RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR] RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR] RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR] RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR] RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR] RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR] RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR] RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR] RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR] RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR] RewriteCond %{HTTP_USER_AGENT} ^HMView [OR] RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR] RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR] RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR] RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR] RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR] RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR] RewriteCond %{HTTP_USER_AGENT} ^larbin [OR] RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR] RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR] RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR] RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR] RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR] RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR] RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR] RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR] RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR] RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR] RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR] RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR] RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR] RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR] RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR] RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR] RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR] RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR] RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR] RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR] RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR] RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR] RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR] RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^Wget [OR] RewriteCond %{HTTP_USER_AGENT} ^Widow [OR] RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR] RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Zeus RewriteRule ^.* - [F,L] Link to comment Share on other sites More sharing options...
Fredi Posted July 9, 2018 Author Share Posted July 9, 2018 Thanks Zahid. Support forum for osCommerce in russian language - from Ashkelon. Support since 2002.Best regards, Fredi Link to comment Share on other sites More sharing options...
valquiria23 Posted July 16, 2018 Share Posted July 16, 2018 Dear all, When I put the rule above in my .htaccess it gives me the error: ERROR 500 - INTERNAL SERVER ERROR Any idea why that error happens? Best regards. Valqui Community Oscommerce fan You'll find the latest osC community version here. Link to comment Share on other sites More sharing options...
Hotclutch Posted July 17, 2018 Share Posted July 17, 2018 maybe you already have specified RewriteEngine On, and now have it twice. Link to comment Share on other sites More sharing options...
Stephan Gebbers Posted July 17, 2018 Share Posted July 17, 2018 10 hours ago, valquiria23 said: Dear all, When I put the rule above in my .htaccess it gives me the error: ERROR 500 - INTERNAL SERVER ERROR Any idea why that error happens? Best regards. Valqui try changing //Block bad bots to #Block bad bots Link to comment Share on other sites More sharing options...
♥JcMagpie Posted July 17, 2018 Share Posted July 17, 2018 //Block bad bots to #Block bad bots Both are fine in .htaccess should not cause error. Also "RewriteEngine On" more than once is not recommended but will not cause error! I have it 4 times in mine with no problems. Must be some other issue with your setup if simply adding another "RewriteEngine On" is causing this problem. Your best solution is to go to your error log and see what caused the error. Link to comment Share on other sites More sharing options...
Stephan Gebbers Posted July 17, 2018 Share Posted July 17, 2018 1 hour ago, JcMagpie said: //Block bad bots to #Block bad bots Both are fine in .htaccess should not cause error. adding //test to my htaccess = instant error 500 Link to comment Share on other sites More sharing options...
♥JcMagpie Posted July 17, 2018 Share Posted July 17, 2018 ? makes no difference to any of my sites, must have something to do with the way the server is setup! error 500 is useless as it is used for anything the server does not understand! you need to check your error log to see what is causing the .htaccess to do this by just adding //text as on its own its meaningless. But hey if removing it fixes it for you then all's good. Link to comment Share on other sites More sharing options...
Fredi Posted July 18, 2018 Author Share Posted July 18, 2018 I also had such a the error: ERROR 500 I change //Block bad botsto#Block bad bots Changes solved the problem. Support forum for osCommerce in russian language - from Ashkelon. Support since 2002.Best regards, Fredi Link to comment Share on other sites More sharing options...
fiodh Posted January 24, 2019 Share Posted January 24, 2019 This has been a real problem for us, about 30 accounts a day, using innocent people's email addresses, with fake details. Today I added googles recaptcha module to the create account page and it seems to have worked great. I added three pieces of text to the createaccount.php file, here they are, hopefully you can pick up where to insert them from the contextual hints immediately in front: Released under the GNU General Public License */ require('includes/application_top.php'); // ReCaptcha Start require(DIR_WS_FUNCTIONS . 'ReCaptcha/autoload.php'); // reCAPTCHA // ReCaptcha End :::::::::::::::::::::::::::::::::::::::; $process = false; if (isset($HTTP_POST_VARS['action']) && ($HTTP_POST_VARS['action'] == 'process') && isset($HTTP_POST_VARS['formid']) && ($HTTP_POST_VARS['formid'] == $sessiontoken)) { // reCAPTCHA - start $recaptcha = new \ReCaptcha\ReCaptcha(RECAPTCHA_PRIVATE_KEY); $resp = $recaptcha->verify($_POST['g-recaptcha-response'], $_SERVER['REMOTE_ADDR']); if ($resp->isSuccess()) { $error = false; $name = tep_db_prepare_input($HTTP_POST_VARS['name']); $email_address = tep_db_prepare_input($HTTP_POST_VARS['email']); $enquiry = tep_db_prepare_input($HTTP_POST_VARS['enquiry']); if (!tep_validate_email($email_address)) { $error = true; $messageStack->add('createaccount', ENTRY_EMAIL_ADDRESS_CHECK_ERROR); } } else { $error = true; $messageStack->add('createaccount', RECAPTCHA_ERROR); } // reCAPTCHA - end :::::::::::::::::::::::::::::: And then way down at the bottom of the page just above the button code: <div class="clearfix"></div> <!-- ReCaptcha Start --> <?php echo tep_draw_separator('pixel_trans.gif', '100%', '10'); ?> <div class="g-recaptcha" data-sitekey="<?php echo RECAPTCHA_PUBLIC_KEY; ?>"></div> <!-- ReCaptcha End --> <div class="buttonSet"> <div class="text-right"><?php echo tep_draw_button(IMAGE_BUTTON_CONTINUE, 'glyphicon glyphicon-user', null, 'primary', null, 'btn-primary'); ?></div> Hope this helps someone Link to comment Share on other sites More sharing options...
fiodh Posted January 24, 2019 Share Posted January 24, 2019 I take that back, it hasn't worked yet, not sure I got it installed properly. Will keep trying. Link to comment Share on other sites More sharing options...
♥JcMagpie Posted January 25, 2019 Share Posted January 25, 2019 16 hours ago, fiodh said: it hasn't worked yet Try using this https://apps.oscommerce.com/t3rEO&honey-pot As well as this, https://apps.oscommerce.com/f2UI4&recaptcha-2-form-validation-for-bs-edge You need to secure all form on your site to stop the buggers. They are not intrested in making an account, they are simply looking for any weekness in your site that will let them enter script into your database. So any method of input you have that saves to your db is open to abuse, even silly things like write a review! so check them all and make sure they are protected. In adition to this find out what ip's they are using and place them on block using your IP blocker. Link to comment Share on other sites More sharing options...
fiodh Posted January 25, 2019 Share Posted January 25, 2019 Thanks very much, I will have a go at that. Link to comment Share on other sites More sharing options...
♥John W Posted January 25, 2019 Share Posted January 25, 2019 Zahid, you have a couple lines like below where you don't have ^ before the bot name. Is this done on purpose, or accidental? RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR] I'm not really a dog. Link to comment Share on other sites More sharing options...
♥JcMagpie Posted January 25, 2019 Share Posted January 25, 2019 Sorry John I'm not abel to spot the one your refering to in the list? I take it you are refering to the bot list Link to comment Share on other sites More sharing options...
♥JcMagpie Posted January 25, 2019 Share Posted January 25, 2019 OK eyes working now 😂 yes looks like a typo to me I'll check my file to confirm Link to comment Share on other sites More sharing options...
♥JcMagpie Posted January 25, 2019 Share Posted January 25, 2019 No it's correct those are ment to be like that it's the same as the file I got from my host. My list is old so new ones are bound to be needed. You can find many of these posted by hosting companys. Here is another a # Ultimate htaccess Blacklist from Perishable Press # Deny domain access to spammers and other scumbags RewriteEngine on RewriteBase / RewriteCond %{HTTP_USER_AGENT} almaden [OR] RewriteCond %{HTTP_USER_AGENT} ^Anarchie [OR] RewriteCond %{HTTP_USER_AGENT} ^ASPSeek [OR] RewriteCond %{HTTP_USER_AGENT} ^attach [OR] RewriteCond %{HTTP_USER_AGENT} ^autoemailspider [OR] RewriteCond %{HTTP_USER_AGENT} ^BackWeb [OR] RewriteCond %{HTTP_USER_AGENT} ^Bandit [OR] RewriteCond %{HTTP_USER_AGENT} ^BatchFTP [OR] RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR] RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:[email protected] [OR] RewriteCond %{HTTP_USER_AGENT} ^Buddy [OR] RewriteCond %{HTTP_USER_AGENT} ^bumblebee [OR] RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR] RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR] RewriteCond %{HTTP_USER_AGENT} ^CICC [OR] RewriteCond %{HTTP_USER_AGENT} ^Collector [OR] RewriteCond %{HTTP_USER_AGENT} ^Copier [OR] RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR] RewriteCond %{HTTP_USER_AGENT} ^Custo [OR] RewriteCond %{HTTP_USER_AGENT} ^DA [OR] RewriteCond %{HTTP_USER_AGENT} ^DIIbot [OR] RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR] RewriteCond %{HTTP_USER_AGENT} ^DISCo\ Pump [OR] RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR] RewriteCond %{HTTP_USER_AGENT} ^Download\ Wonder [OR] RewriteCond %{HTTP_USER_AGENT} ^Downloader [OR] RewriteCond %{HTTP_USER_AGENT} ^Drip [OR] RewriteCond %{HTTP_USER_AGENT} ^DSurf15a [OR] RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR] RewriteCond %{HTTP_USER_AGENT} ^EasyDL/2.99 [OR] RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR] RewriteCond %{HTTP_USER_AGENT} email [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR] RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR] RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR] RewriteCond %{HTTP_USER_AGENT} ^FileHound [OR] RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR] RewriteCond %{HTTP_USER_AGENT} FrontPage [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR] RewriteCond %{HTTP_USER_AGENT} ^GetSmart [OR] RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR] RewriteCond %{HTTP_USER_AGENT} ^gigabaz [OR] RewriteCond %{HTTP_USER_AGENT} ^Go\!Zilla [OR] RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR] RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR] RewriteCond %{HTTP_USER_AGENT} ^gotit [OR] RewriteCond %{HTTP_USER_AGENT} ^Grabber [OR] RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR] RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR] RewriteCond %{HTTP_USER_AGENT} ^grub-client [OR] RewriteCond %{HTTP_USER_AGENT} ^HMView [OR] RewriteCond %{HTTP_USER_AGENT} ^HTTrack [OR] RewriteCond %{HTTP_USER_AGENT} ^httpdown [OR] RewriteCond %{HTTP_USER_AGENT} .*httrack.* [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR] RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR] RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^Indy*Library [OR] RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR] RewriteCond %{HTTP_USER_AGENT} ^InternetLinkagent [OR] RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR] RewriteCond %{HTTP_USER_AGENT} ^InternetSeer.com [OR] RewriteCond %{HTTP_USER_AGENT} ^Iria [OR] RewriteCond %{HTTP_USER_AGENT} ^JBH*agent [OR] RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR] RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR] RewriteCond %{HTTP_USER_AGENT} ^JustView [OR] RewriteCond %{HTTP_USER_AGENT} ^larbin [OR] RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR] RewriteCond %{HTTP_USER_AGENT} ^LexiBot [OR] RewriteCond %{HTTP_USER_AGENT} ^lftp [OR] RewriteCond %{HTTP_USER_AGENT} ^Link*Sleuth [OR] RewriteCond %{HTTP_USER_AGENT} ^likse [OR] RewriteCond %{HTTP_USER_AGENT} ^Link [OR] RewriteCond %{HTTP_USER_AGENT} ^LinkWalker [OR] RewriteCond %{HTTP_USER_AGENT} ^Mag-Net [OR] RewriteCond %{HTTP_USER_AGENT} ^Magnet [OR] RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR] RewriteCond %{HTTP_USER_AGENT} ^Memo [OR] RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR] RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR] RewriteCond %{HTTP_USER_AGENT} ^Mirror [OR] RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR] RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*Indy [OR] RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR] RewriteCond %{HTTP_USER_AGENT} ^Mozilla*MSIECrawler [OR] RewriteCond %{HTTP_USER_AGENT} ^MS\ FrontPage* [OR] RewriteCond %{HTTP_USER_AGENT} ^MSFrontPage [OR] RewriteCond %{HTTP_USER_AGENT} ^MSIECrawler [OR] RewriteCond %{HTTP_USER_AGENT} ^MSProxy [OR] RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR] RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR] RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR] RewriteCond %{HTTP_USER_AGENT} ^NetMechanic [OR] RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR] RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR] RewriteCond %{HTTP_USER_AGENT} ^Ninja [OR] RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR] RewriteCond %{HTTP_USER_AGENT} ^Openfind [OR] RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR] RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR] RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR] RewriteCond %{HTTP_USER_AGENT} ^Ping [OR] RewriteCond %{HTTP_USER_AGENT} ^PingALink [OR] RewriteCond %{HTTP_USER_AGENT} ^Pockey [OR] RewriteCond %{HTTP_USER_AGENT} ^psbot [OR] RewriteCond %{HTTP_USER_AGENT} ^Pump [OR] RewriteCond %{HTTP_USER_AGENT} ^QRVA [OR] RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^Reaper [OR] RewriteCond %{HTTP_USER_AGENT} ^Recorder [OR] RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR] RewriteCond %{HTTP_USER_AGENT} ^Scooter [OR] RewriteCond %{HTTP_USER_AGENT} ^Seeker [OR] RewriteCond %{HTTP_USER_AGENT} ^Siphon [OR] RewriteCond %{HTTP_USER_AGENT} ^sitecheck.internetseer.com [OR] RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR] RewriteCond %{HTTP_USER_AGENT} ^SlySearch [OR] RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^Snake [OR] RewriteCond %{HTTP_USER_AGENT} ^SpaceBison [OR] RewriteCond %{HTTP_USER_AGENT} ^sproose [OR] RewriteCond %{HTTP_USER_AGENT} ^Stripper [OR] RewriteCond %{HTTP_USER_AGENT} ^Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR] RewriteCond %{HTTP_USER_AGENT} ^Szukacz [OR] RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR] RewriteCond %{HTTP_USER_AGENT} ^URLSpiderPro [OR] RewriteCond %{HTTP_USER_AGENT} ^Vacuum [OR] RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR] RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR] RewriteCond %{HTTP_USER_AGENT} ^webcollage [OR] RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\ Downloader [OR] RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR] RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR] RewriteCond %{HTTP_USER_AGENT} ^WebHook [OR] RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR] RewriteCond %{HTTP_USER_AGENT} ^WebMiner [OR] RewriteCond %{HTTP_USER_AGENT} ^WebMirror [OR] RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR] RewriteCond %{HTTP_USER_AGENT} ^Website [OR] RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR] RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR] RewriteCond %{HTTP_USER_AGENT} ^Webster [OR] RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR] RewriteCond %{HTTP_USER_AGENT} WebWhacker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^Wget [OR] RewriteCond %{HTTP_USER_AGENT} ^Whacker [OR] RewriteCond %{HTTP_USER_AGENT} ^Widow [OR] RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR] RewriteCond %{HTTP_USER_AGENT} ^x-Tractor [OR] RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Xenu [OR] RewriteCond %{HTTP_USER_AGENT} ^Zeus.*Webster [OR] RewriteCond %{HTTP_USER_AGENT} ^Zeus RewriteRule ^.* - [F,L] Apache have a more upto date one which is applied using robot.txt https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker/blob/master/robots.txt/robots.txt Link to comment Share on other sites More sharing options...
♥raiwa Posted January 25, 2019 Share Posted January 25, 2019 robots.txt is useless to block bad bots. They will just ignore it. About Me: http://www.oscommerce.com/forums/user/249059-raiwa/ Need help? How To Get The Help You Need Is your version of osC up to date? You'll find the latest osC community version CE Phoenix here. Public Phoenix Change Log Cheat Set on Google Sheets Link to comment Share on other sites More sharing options...
♥JcMagpie Posted January 25, 2019 Share Posted January 25, 2019 10 minutes ago, raiwa said: robots.txt is useless to block bad bots. They will just ignore it. Oh ok I will let my host know this perhaps they don't know what they are doing? Link to comment Share on other sites More sharing options...
♥John W Posted January 25, 2019 Share Posted January 25, 2019 I did some googling and found the .htaccess references and found it both ways. Did some reading on this on the cpanel forums and they recommend using Mod Security to do it. I had a brain fart there for a moment because my mod_security does have rules for bad bots. Bad bots can ignore robots.txt. I'm not really a dog. Link to comment Share on other sites More sharing options...
♥JcMagpie Posted January 25, 2019 Share Posted January 25, 2019 1 minute ago, John W said: Bad bots can ignore robots.txt. 😊 Yes I know John just could not resist it! Sorry. Robot.txt is still used and kept upto date as it works for silly bots the ones that are not evil just a pain. Stop the crawling your site. The evil ones get round most block on way or another! The only real known way to block them is at the server level with a good firewall. Link to comment Share on other sites More sharing options...
René H4 Posted January 26, 2019 Share Posted January 26, 2019 I am running BS 2.3.4.1, but I cannot fins a robots.txt file in my setup. Should I have one? (I *do* have a spiders.txt file in the includes directroy) Neiter has the .htaccess file the above mentioned code in it. Should I act? Link to comment Share on other sites More sharing options...
♥JcMagpie Posted January 26, 2019 Share Posted January 26, 2019 18 minutes ago, honda4 said: Should I act? 😊 As you have seen opinions are diferent for robot.txt so you need to make your own mind up. See google advise for this, https://support.google.com/webmasters/answer/6062596?hl=en As for the .htaccess again depends if you are having problems with spam and other issues if yes then your should take all the steps you can to reduce this including the .htaccess file option. Link to comment Share on other sites More sharing options...
Jack_mcs Posted January 26, 2019 Share Posted January 26, 2019 3 hours ago, honda4 said: I am running BS 2.3.4.1, but I cannot fins a robots.txt file in my setup It never has been included that I can recall. You need to add it if you want to use it. Reasons I think a robots file should be used: It will block some bots that don't help the site. It should be used to list the sitemap file if it is not named sitemap.idx. Search engines look for a file by that name and will assume one is not present if it is named something else and there isn't a link to it. It is useful for scripts like View Counter and the IP Blocker to set up traps. Hackers see a directory or file marked as blocked and they simply can't help themselves to check them. Some sites don't want their images listed so the images directory can be blocked. Some pages should not be listed on the search engine pages, like advanced_search_result.php , or any page that may be linked to but requires a password. You can use the robots noindex module to do this. One thing that should never be in the robots file is the actual admin name. I see this over and over again and it is an open challenge to hackers to try it. Take a look at google.com/robots.txt and amazon.com/robots.txt. I don't think sites like those would use them if they were not useful. Support Links: For Hire: Contact me for anything you need help with for your shop: upgrading, hosting, repairs, code written, etc. Get the latest versions of my addons Recommended SEO Addons Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.