Jump to content

Archived

This topic is now archived and is closed to further replies.

donny84

Bing creating sessions?!

Recommended Posts

Hey peeps :)

 

I am running V2.3.4

 

I have prevent spider sessions enable in the admin panel and have Bing in my spiders.txt list

 

It is still creating sessions, and indexing them with osCsid

 

I cannot figure out why, please can someone help me...

 

Not all bingbots create sessions... As you can see from my attached access log

 

So confused! :(

post-192893-0-75560400-1423733666_thumb.jpg

post-192893-0-86285700-1423733676_thumb.jpg

Share this post


Link to post
Share on other sites
Updated my spiders.txt but I had updated it recently anyway...
Here is my spiders.txt... Website still is allowing bingbot to create sessions :-(
 

 

$Id$

ahrefsbot

almaden.ibm.com
abachobot
accoona-ai-agent
addsugarspiderbot
arachmo
baiduspider
becomebot
beslistbot
bimbot
bingbot
dataparksearch
dotbot
envolk[its]spider
exabot
fast enterprise crawler
fast-webcrawler
furlbot
fyberspider
galaxybot
gigabot
girafabot
googlebot
googlebot-image
holmes
ia_archiver
ichiro
igdespyder
irlbot
l.webis
larbin
ldspider
lexxebot
linguee bot
linkwalker
lmspider
lwp-trivial
mabontland
magpie-crawler
mediapartners-google
mj12bot
mnogosearch
mojeekbot
moreoverbot
morning paper
msnbot
mxbot
netresearchserver
netseer crawler
newsgator
ng-search
nicebot
noxtrumbot
nutchcvs
obot
oozbot
orangebot
polybot
psbot
pycurl
scoutjet
scrubby
seekbot
seochat::bot
seznambot
shim-crawler
shopwiki
shoula robot
silk
snappy
sogou spider
sosospider
speedy spider
stackrambler
suggybot
synoobot
teoma
thumbnail.cz robot
tineye
turnitinbot
twengabot
urlfilebot
vagabondo
voltron
websquash.com
yacy
yahoo! slurp
yahooseeker
yandexbot
yandeximages
yeti
yodaobot
youdaobot
zealbot
zyborg

 

 
Here is a snippet from my access log...
 

 

157.55.39.211 - - [14/Feb/2015:19:09:46 +0000] "GET /index.php/cPath/22/osCsid/feb909fa32799e2961308ac9d57be9bb HTTP/1.1" 200 32868 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

157.55.39.211 - - [14/Feb/2015:19:10:11 +0000] "GET /index.php/cPath/23/osCsid/05326afc0d0ab82cdb61fdc194eb659d HTTP/1.1" 200 33305 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
157.55.39.211 - - [14/Feb/2015:19:10:16 +0000] "GET /index.php/cPath/25/sort/2a/page/2/osCsid/2b05405d5bebd46aaf18745b802995bb HTTP/1.1" 200 34015 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
66.249.64.151 - - [14/Feb/2015:19:10:35 +0000] "GET /product_info.php/products_id/15183 HTTP/1.1" 200 18438 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
180.76.6.151 - - [14/Feb/2015:19:10:37 +0000] "GET /index.php/manufacturers_id/127 HTTP/1.1" 200 35392 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
207.46.13.76 - - [14/Feb/2015:19:10:58 +0000] "GET /index.php/cPath/26_162/osCsid/d4e296ef13fe1bd037615c2993107ce9 HTTP/1.1" 200 34747 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
157.55.39.211 - - [14/Feb/2015:19:11:22 +0000] "GET /index.php/cPath/50_59/osCsid/f67f1b3ec13dc0b100573832a629a367 HTTP/1.1" 200 25376 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
207.46.13.76 - - [14/Feb/2015:19:11:58 +0000] "GET /index.php/cPath/50_58/osCsid/3e66528868f1090a42257c8aa535bb46 HTTP/1.1" 200 35313 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
66.249.64.151 - - [14/Feb/2015:19:12:05 +0000] "GET /product_info.php/cPath/78_83/products_id/12811/osCsid/4773df3672ad5ee747d34a1a55519d0e HTTP/1.1" 200 22587 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
207.46.13.69 - - [14/Feb/2015:19:12:17 +0000] "GET /index.php/cPath/23/osCsid/f503939a329c208a3b500d1525344fbb HTTP/1.1" 200 33341 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
178.154.204.250 - - [14/Feb/2015:19:12:57 +0000] "GET /product_info.php/cPath/202_56/products_id/249 HTTP/1.1" 200 21436 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
178.154.204.250 - - [14/Feb/2015:19:12:59 +0000] "GET /product_info.php/products_id/249 HTTP/1.1" 200 23284 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
207.46.13.76 - - [14/Feb/2015:19:13:03 +0000] "GET /index.php/cPath/50_64/osCsid/dc2380fb8397c2e65f64cd03741a4301 HTTP/1.1" 200 36346 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
207.46.13.76 - - [14/Feb/2015:19:13:43 +0000] "GET /index.php/cPath/22/osCsid/827e9527d37e8d36d13e5bd4c2cad3e9 HTTP/1.1" 200 32889 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
207.46.13.76 - - [14/Feb/2015:19:13:58 +0000] "GET /index.php/cPath/22/osCsid/a4ad8caa5cfbcb43cb6aef41779ad394 HTTP/1.1" 200 32868 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
66.249.64.151 - - [14/Feb/2015:19:15:53 +0000] "GET /product_info.php/products_id/13500 HTTP/1.1" 200 22960 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
207.46.13.76 - - [14/Feb/2015:19:16:09 +0000] "GET /index.php/cPath/43_44/osCsid/9a4a4f3b691ee6fc44471aa02a9e1d3c HTTP/1.1" 200 35040 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
207.46.13.76 - - [14/Feb/2015:19:16:34 +0000] "GET /index.php/cPath/25/sort/2a/page/2/osCsid/003d9504c69137c291ef5f724c206b95 HTTP/1.1" 200 34068 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
157.55.39.211 - - [14/Feb/2015:19:17:18 +0000] "GET /gv_faq.php/osCsid/a1e330ad01fd8225e07f8555c96e36c8 HTTP/1.1" 200 16870 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
180.76.5.71 - - [14/Feb/2015:19:18:29 +0000] "GET /index.php/cPath/50_64/sort/2a/action/buy_now/products_id/13299 HTTP/1.1" 302 - "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
207.46.13.76 - - [14/Feb/2015:19:18:39 +0000] "GET /index.php/cPath/23/osCsid/5d288a7407609493c6b8539e3c3dc35f HTTP/1.1" 200 33305 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
157.55.39.212 - - [14/Feb/2015:19:19:23 +0000] "GET /index.php/cPath/25/osCsid/bc31b8e654d7ce3a2854413561da7870 HTTP/1.1" 200 33461 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

 

Share this post


Link to post
Share on other sites

I note that the entries are for bingbot/2.0 -- perhaps that isn't covered by just bingbot? Try adding bingbot/2.0 to your list, and if that works, try more generic versions such as bingbot/* and see if it still works. Please let us know what you find.


If you are running the "official" osC 2.3.4 or 2.3.4.1 download, your installation is obsolete! Get (stable) Frozenpatches or (unstable) Edge. See also the naming convention and the latest community-supported responsive "Edge" release

Share this post


Link to post
Share on other sites

×