Bing creating sessions?!

[Guest]

Hey peeps :)

 

I am running V2.3.4

 

I have prevent spider sessions enable in the admin panel and have Bing in my spiders.txt list

 

It is still creating sessions, and indexing them with osCsid

 

I cannot figure out why, please can someone help me...

 

Not all bingbots create sessions... As you can see from my attached access log

 

So confused! :(

[acidvertigo]

You have probably to update your spider.txt file as it is very old.

 

If you have not changed this file try to change with this version: https://github.com/acidvertigo/oscommerce2/blob/patch-22/catalog/includes/spiders.txt

[Guest]

Updated my spiders.txt but I had updated it recently anyway...

Here is my spiders.txt... Website still is allowing bingbot to create sessions :-(

 

 

$Id$

ahrefsbot

almaden.ibm.com

abachobot

accoona-ai-agent

addsugarspiderbot

arachmo

baiduspider

becomebot

beslistbot

bimbot

bingbot

dataparksearch

dotbot

envolk[its]spider

exabot

fast enterprise crawler

fast-webcrawler

furlbot

fyberspider

galaxybot

gigabot

girafabot

googlebot

googlebot-image

holmes

ia_archiver

ichiro

igdespyder

irlbot

l.webis

larbin

ldspider

lexxebot

linguee bot

linkwalker

lmspider

lwp-trivial

mabontland

magpie-crawler

mediapartners-google

mj12bot

mnogosearch

mojeekbot

moreoverbot

morning paper

msnbot

mxbot

netresearchserver

netseer crawler

newsgator

ng-search

nicebot

noxtrumbot

nutchcvs

obot

oozbot

orangebot

polybot

psbot

pycurl

scoutjet

scrubby

seekbot

seochat::bot

seznambot

shim-crawler

shopwiki

shoula robot

silk

snappy

sogou spider

sosospider

speedy spider

stackrambler

suggybot

synoobot

teoma

thumbnail.cz robot

tineye

turnitinbot

twengabot

urlfilebot

vagabondo

voltron

websquash.com

yacy

yahoo! slurp

yahooseeker

yandexbot

yandeximages

yeti

yodaobot

youdaobot

zealbot

zyborg

 

 

Here is a snippet from my access log...

 

 

157.55.39.211 - - [14/Feb/2015:19:09:46 +0000] "GET /index.php/cPath/22/osCsid/feb909fa32799e2961308ac9d57be9bb HTTP/1.1" 200 32868 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

157.55.39.211 - - [14/Feb/2015:19:10:11 +0000] "GET /index.php/cPath/23/osCsid/05326afc0d0ab82cdb61fdc194eb659d HTTP/1.1" 200 33305 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

157.55.39.211 - - [14/Feb/2015:19:10:16 +0000] "GET /index.php/cPath/25/sort/2a/page/2/osCsid/2b05405d5bebd46aaf18745b802995bb HTTP/1.1" 200 34015 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

66.249.64.151 - - [14/Feb/2015:19:10:35 +0000] "GET /product_info.php/products_id/15183 HTTP/1.1" 200 18438 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

180.76.6.151 - - [14/Feb/2015:19:10:37 +0000] "GET /index.php/manufacturers_id/127 HTTP/1.1" 200 35392 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

207.46.13.76 - - [14/Feb/2015:19:10:58 +0000] "GET /index.php/cPath/26_162/osCsid/d4e296ef13fe1bd037615c2993107ce9 HTTP/1.1" 200 34747 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

157.55.39.211 - - [14/Feb/2015:19:11:22 +0000] "GET /index.php/cPath/50_59/osCsid/f67f1b3ec13dc0b100573832a629a367 HTTP/1.1" 200 25376 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

207.46.13.76 - - [14/Feb/2015:19:11:58 +0000] "GET /index.php/cPath/50_58/osCsid/3e66528868f1090a42257c8aa535bb46 HTTP/1.1" 200 35313 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

66.249.64.151 - - [14/Feb/2015:19:12:05 +0000] "GET /product_info.php/cPath/78_83/products_id/12811/osCsid/4773df3672ad5ee747d34a1a55519d0e HTTP/1.1" 200 22587 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

207.46.13.69 - - [14/Feb/2015:19:12:17 +0000] "GET /index.php/cPath/23/osCsid/f503939a329c208a3b500d1525344fbb HTTP/1.1" 200 33341 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

178.154.204.250 - - [14/Feb/2015:19:12:57 +0000] "GET /product_info.php/cPath/202_56/products_id/249 HTTP/1.1" 200 21436 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"

178.154.204.250 - - [14/Feb/2015:19:12:59 +0000] "GET /product_info.php/products_id/249 HTTP/1.1" 200 23284 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"

207.46.13.76 - - [14/Feb/2015:19:13:03 +0000] "GET /index.php/cPath/50_64/osCsid/dc2380fb8397c2e65f64cd03741a4301 HTTP/1.1" 200 36346 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

207.46.13.76 - - [14/Feb/2015:19:13:43 +0000] "GET /index.php/cPath/22/osCsid/827e9527d37e8d36d13e5bd4c2cad3e9 HTTP/1.1" 200 32889 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

207.46.13.76 - - [14/Feb/2015:19:13:58 +0000] "GET /index.php/cPath/22/osCsid/a4ad8caa5cfbcb43cb6aef41779ad394 HTTP/1.1" 200 32868 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

66.249.64.151 - - [14/Feb/2015:19:15:53 +0000] "GET /product_info.php/products_id/13500 HTTP/1.1" 200 22960 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

207.46.13.76 - - [14/Feb/2015:19:16:09 +0000] "GET /index.php/cPath/43_44/osCsid/9a4a4f3b691ee6fc44471aa02a9e1d3c HTTP/1.1" 200 35040 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

207.46.13.76 - - [14/Feb/2015:19:16:34 +0000] "GET /index.php/cPath/25/sort/2a/page/2/osCsid/003d9504c69137c291ef5f724c206b95 HTTP/1.1" 200 34068 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

157.55.39.211 - - [14/Feb/2015:19:17:18 +0000] "GET /gv_faq.php/osCsid/a1e330ad01fd8225e07f8555c96e36c8 HTTP/1.1" 200 16870 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

180.76.5.71 - - [14/Feb/2015:19:18:29 +0000] "GET /index.php/cPath/50_64/sort/2a/action/buy_now/products_id/13299 HTTP/1.1" 302 - "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

207.46.13.76 - - [14/Feb/2015:19:18:39 +0000] "GET /index.php/cPath/23/osCsid/5d288a7407609493c6b8539e3c3dc35f HTTP/1.1" 200 33305 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

157.55.39.212 - - [14/Feb/2015:19:19:23 +0000] "GET /index.php/cPath/25/osCsid/bc31b8e654d7ce3a2854413561da7870 HTTP/1.1" 200 33461 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

 

[MrPhil]

I note that the entries are for bingbot/2.0 -- perhaps that isn't covered by just bingbot? Try adding bingbot/2.0 to your list, and if that works, try more generic versions such as bingbot/* and see if it still works. Please let us know what you find.