Jump to content
medvid

Who's Online Enhancement 1.4

Recommended Posts

Sorry, I don't very much understand in coding and making contributions. This is why I'm doing exactly as it saying, hoping this is a correct way. The 'Updated spiders.txt' (http://www.oscommerce.com/community/contributions,2455) I got installed this morning said: "A replacement for catalog/includes/spiders.txt - updated with newly seen spiders and optimized for quicker processing." This is exactly what I did, replaced spiders.txt. Misunderstanding.

 

What file should I use instead of my spiders.txt? Please, don't be so cruel!!! :blush:

Do you think this is the only reason why my bots are not being recognized?

 

Thanks a lot,

Irina.

 

Irina

Sorry, wasn't trying to be mean.

 

If you go back a page or two in this thread you will see I have posted a list that is in my spiders.txt file. Copy and paste this in your file and just check for duplications and remove any.

Personaly I do not understand why some of those contributors are saying it's a replacement file, maybe they don't know what they are talking about, I don't know.

 

At this point as far as if this is the only reason why your bots aren't being recognised I would say probably, yes. As google, yahoo and msn are the most common ones that really get around and my site is being crawed by them all the time. Essentially consistantly by at least one or two of them.

At least at this point, put the complete list in the spiders.txt file and see what happens. If that fixes the problem then we need go no further.

 

John

Thanks John. I'll try with a new spiders.txt. I'll let you know if the problem is resolved.

 

Regards,

Irina.

Share this post


Link to post
Share on other sites

Hi,

 

Sorry for being annoying but replacing spiders.txt wasn't very helpful for me. My bots still not being recognized. I just had msnbot visiting my web site, and it's defined as Real Customers. I have Prevent Spider Sessions set to true, if this tells you anything.

 

What else should I try to have it recognize those bots? :rolleyes:

Thanks a lot,

Irina.

Share this post


Link to post
Share on other sites

Here's a question for someone that might know...

 

WHY do we get DUPLICATE IPS listed in Whos Online?


Dan

Share this post


Link to post
Share on other sites
Here's a question for someone that might know...

 

WHY do we get DUPLICATE IPS listed in Whos Online?

Dan,

 

Entries are stored in the whos_online table based on session id. If the session id changes, there will be multiple entries with the same IP.

 

ed


Answers to osCommerce's most persistent questions! Tips & Tricks | Configuration | Common Problems.

Seek and ye shall find Contributions.

My Contributions

My Blog

Share this post


Link to post
Share on other sites
Patrice

You need to go back to your hosting provider and get these details checked.

The only question to ask I think is:

What is the correct Home path of your site?

Then you need to check your configure files in both catalog/includes/ and catalog/admin/includes/

and make sure all adds up.

Once that is done and confirmed and uploaded then you need to make sure your sessions and cache storage directories is set corrrectly in the admin section.

 

John

 

Hi John,

 

I've received answer from hosting provider...

/home/virtual/onveut.com/var/www/html/

and /home/virtual/site156/fst/var/www/html/

are equivalent path but I've to prefer the first one...

I've verified my configure.php in includes and admin/includes + I've verified settings in configuration part of the admin pages (sessions, log and cache) : the path are good...

 

And the problem stay the same...

 

Don't understand...

 

If you've some more ideas...

 

Thank you very much...

Edited by littleneo

Share this post


Link to post
Share on other sites

Patrice,

 

The 2 paths may be equivalent for your host's router/DNS/etc., but my guess is that the osCommerce code sees them as different. You need to pick one (you said you had a preference) and go through the site and make sure only one is being used.

 

ed


Answers to osCommerce's most persistent questions! Tips & Tricks | Configuration | Common Problems.

Seek and ye shall find Contributions.

My Contributions

My Blog

Share this post


Link to post
Share on other sites
Why do the Session IDs change?

Dan,

 

There are several ways...

 

1) In admin->configuration->sessions, you have Recreate Session turned on.

2) If your cookie paths or http vs. https paths aren't correct in configure.php AND the user hits the browser Back button AND then hits another link, they can get a new session.

3) They are REALLY two people sitting behind a shared router. The router IP is recorded by your server, not their individual workstations.

4) Search engines have your session ids stored in their indexes for your site. Someone clicks on a link to your site with a session id embedded in the link, goes back to the serach engine and clicks on another link to your site with a different session id embedded in the link.

 

I'm sure there's a few more but I have personbally seen/dealt with each of these.

 

ed


Answers to osCommerce's most persistent questions! Tips & Tricks | Configuration | Common Problems.

Seek and ye shall find Contributions.

My Contributions

My Blog

Share this post


Link to post
Share on other sites

My cookie paths are simply:

 

define('HTTP_COOKIE_PATH', '/');

define('HTTPS_COOKIE_PATH', '/');

 

Is that right?? I haven't had any issues, the shop has been working for a year like this but I do notice sometimes duplicate IPs and I have even had a customer on the phone and it wasn't adding stuff to his cart properly because it was assigning a new session ID with everything he did, which resulted in the dup IP entries in Whos Online (obviously).


Dan

Share this post


Link to post
Share on other sites
My cookie paths are simply:

 

  define('HTTP_COOKIE_PATH', '/');

  define('HTTPS_COOKIE_PATH', '/');

 

Is that right?? I haven't had any issues, the shop has been working for a year like this but I do notice sometimes duplicate IPs and I have even had a customer on the phone and it wasn't adding stuff to his cart properly because it was assigning a new session ID with everything he did, which resulted in the dup IP entries in Whos Online (obviously).

 

Dan

That address should be fine if your catalog is in the root directory.

 

Re the customer having thise problems you will probably find that the client had at least one possibly more firewalls installed and your cart/or site was not the only one they were having problems with.

I have personally experienced similar problems and all that was needed was to set the IE Security and Privacy settings back to Defaults.

 

John

Share this post


Link to post
Share on other sites

Dan,

 

I have the following settings. The shop is at the root and it uses a shared SSL cert. I think Chemo suggested in a thread to do the .domain.com for cookie domains and leave the cookie paths completely blank. After I changed to this, the number of duplicate IPS dropped.

 

define('HTTP_SERVER', 'http://www.shopdomain.com');

define('HTTPS_SERVER', 'https://shared.ssldomain.com/ssl/path');

define('HTTP_COOKIE_DOMAIN', '.shopdomain.com');

define('HTTPS_COOKIE_DOMAIN', '.shared.ssldomain.com');

define('HTTP_COOKIE_PATH', '');

define('HTTPS_COOKIE_PATH', '');

define('DIR_WS_HTTP_CATALOG', '/');

define('DIR_WS_HTTPS_CATALOG', '/');

 

 

John,

 

I wouldn't mind your thoughts on the above.

 

ed


Answers to osCommerce's most persistent questions! Tips & Tricks | Configuration | Common Problems.

Seek and ye shall find Contributions.

My Contributions

My Blog

Share this post


Link to post
Share on other sites

Hello,

 

I also have the same problem with bots not being recognized. Absolutely the same case scenario as with ashley. And I just updated my spiders.txt, so it looks like this:

What can be wrong? :blink:

Thanks a lot,

Irina.

 

You too have the same problem as "ashleylr" you do not have even the most basic bots in your file. eg: yahoo and msn.

As mentioned befoire, you are supposed to merge the files not replace the file.

 

 

 

Gob,

I used WinMerge to merge my files, I did not replace them. my spiders.txt file does include MSN, slurp, googlebot, etc.

 

However none are recognized.... I can't figure this out...

Ashley

Edited by ashleylr

Share this post


Link to post
Share on other sites
Dan,

 

I have the following settings.?  The shop is at the root and it uses a shared SSL cert.? I think Chemo suggested in a thread to do the .domain.com for cookie domains and leave the cookie paths completely blank.? After I changed to this, the number of duplicate IPS dropped.

 

? define('HTTP_SERVER', 'http://www.shopdomain.com');

? define('HTTPS_SERVER', 'https://shared.ssldomain.com/ssl/path');

? define('HTTP_COOKIE_DOMAIN', '.shopdomain.com');

? define('HTTPS_COOKIE_DOMAIN', '.shared.ssldomain.com');

? define('HTTP_COOKIE_PATH', '');

? define('HTTPS_COOKIE_PATH', '');

? define('DIR_WS_HTTP_CATALOG', '/');

? define('DIR_WS_HTTPS_CATALOG', '/');

John,

 

I wouldn't mind your thoughts on the above.

 

ed

 

This is mine from catalog/includes/

NOTE, my shop is not in the root directory.

 

 ?define('HTTP_SERVER', 'http://www.shopdomain.net.au');
?define('HTTPS_SERVER', 'https://secure.shopdomain.net.au');
?define('ENABLE_SSL', true);
?define('HTTP_COOKIE_DOMAIN', 'shopdomain.net.au');
?define('HTTPS_COOKIE_DOMAIN', 'secure.shopdomain.net.au');
?define('HTTP_COOKIE_PATH', '/shop/');
?define('HTTPS_COOKIE_PATH', '/shop/');
?define('DIR_WS_HTTP_CATALOG', '/shop/');
?define('DIR_WS_HTTPS_CATALOG', '/shop/');

Edited by Gob

Share this post


Link to post
Share on other sites
You too have the same problem as "ashleylr" you do not have even the most basic bots in your file. eg: yahoo and msn.

As mentioned befoire, you are supposed to merge the files not replace the file.

 

Gob,

I used WinMerge to merge my files, I did not replace them.  my spiders.txt file does include MSN, slurp, googlebot, etc. 

 

However none are recognized....  I can't figure this out...

Ashley

Ashley,

 

I'm sorry if you already answered this and it seems basic, but what is the exact path (remove the domain name if you wish) of your spiders.txt file? Also, if you are on a Unix/Linux server, please verify the exact spelling (including upper and lower case) of the file name.

 

Thanks,

Ed

Edited by medvid

Answers to osCommerce's most persistent questions! Tips & Tricks | Configuration | Common Problems.

Seek and ye shall find Contributions.

My Contributions

My Blog

Share this post


Link to post
Share on other sites
This is mine from catalog/includes/

NOTE, my shop is not in the root directory.

 

  define('HTTP_SERVER', 'http://www.shopdomain.net.au');
 define('HTTPS_SERVER', 'https://secure.shopdomain.net.au');
 define('ENABLE_SSL', true);
 define('HTTP_COOKIE_DOMAIN', 'shopdomain.net.au');
 define('HTTPS_COOKIE_DOMAIN', 'secure.shopdomain.net.au');
 define('HTTP_COOKIE_PATH', '/shop/');
 define('HTTPS_COOKIE_PATH', '/shop/');
 define('DIR_WS_HTTP_CATALOG', '/shop/');
 define('DIR_WS_HTTPS_CATALOG', '/shop/');

 

Hmmm,

Except for the leading dot on the cookie domain and the / vs. /shop setup, pretty much the same. And our WOE's work...

 

ed


Answers to osCommerce's most persistent questions! Tips & Tricks | Configuration | Common Problems.

Seek and ye shall find Contributions.

My Contributions

My Blog

Share this post


Link to post
Share on other sites
Hmmm,

Except for the leading dot on the cookie domain and the / vs. /shop setup, pretty much the same.  And our WOE's work...

 

ed

 

Ed

If I was looking at yours I would've said that your leading dot would have been wrong.

But it seems to work, so all the better.

 

One thing that might be playing around is as you mentioned if users are on a unix/linux server is the file name upper or lower case or a mix.

I always use lower case as a rule of thumb saves a lot of headaches, just like this.

 

The other is the upload style (program used, FTP or other) and or the upload format. ASCII or Binary, for the benefit of others most files are ASCII, pictures like jpg, gif etc, Binary.

(Some files have very specific requirements, so you must always check)

Share this post


Link to post
Share on other sites
Ashley,

 

I'm sorry if you already answered this and it seems basic, but what is the exact path (remove the domain name if you wish) of your spiders.txt file?  Also, if you are on a Unix/Linux server, please verify the exact spelling (including upper and lower case) of the file name.

 

Thanks,

Ed

 

 

I can't imagine that is the problem as I've had no problems with previous versions of whos_online. My path is: domain.com/catalog/includes/spiders.txt.

 

I will copy my file below in case you think that might be a problem too...

 

$Id: spiders.txt,v last update 2005/03/01 17:58:17 dgw_ Exp $
bot
abot
ebot
nbot
obot
rbot
sbot
crawl
slurp
spider
accoona
almaden.ibm.com
appie
appie 1.1
architext
aspseek
aspseek
aspseek
ask jeeves
asterias
asterias2.0
atomz
augurfind
baiduspider
bannana_bot
booch
diamondbot
bdcindexer
crawler
crawler@fast
docomo
fast-webcrawler
fluffy the spider
frooglebot
gazz
goforit
grub
geobot
GOOGLE
googlebot
gulliver
holmes
homer
henrythemiragorobot
ia_archiver
iconsurf
iltrovatore
indexer
inktomi
infoseek
ivia
jetbot
kit_fireball
knowledge
lachesis
larbin
linkwalker
lwp
lycos_spider
mantraagent
mediapartners
mercator
mj12
moget/
moget/1.0
muscatferret
myweb
MICROSOFT-GLOBAL-NET
msnbot
newmsbot
nameprotect
nationaldirectory-webspider
naverrobot
ncsa beta
netmechanic
netresearchserver
ng/
ng/1.0
npbot
nutch
osis-project
pear.
polybot
pompos
poppelsdorf
rambler
scooter
scrubby
searchbot
seeker
seventwentyfour
sidewinder
smartwit
sohu
spyder
sleek spider
slurp/si
slurp@inktomi.com
steeler/
steeler/1.3
szukacz
t-h-u-n-d-e-r-s-t-o-n-e
teoma
/teoma
tutorgig
turnitinbot
ultraseek
vagabondo
voyager/
voilabot
w3c_validator
websitepulse
worldlight
worm
yahoo
yahoobot
zao/
zao/0
xenu
zippp
zyborg
zyborg/1.0

Share this post


Link to post
Share on other sites

Okay, I just checked this out and noticed that I'm having the same problem that many others are. It shows bots as any other customer. If I have profile view open, the unidentified bot has no session id nor does it have a user agent. That hints that it's not a problem with spiders.txt because the bots are correctly not being given a session id.

 

I also have another problem. For some reason the last url clicked and the refferer sections are simply left blank. Does anyone else have this problem?

 

Hopefully we can get this all sorted.

Share this post


Link to post
Share on other sites

I believe the above posters are having problems that are actually unrelated to WOS contrib.

In the who's online instructions there is a entry that must be added to: catalog/admin/includes/functions/general.php

 

If you have not added this section you need to do one of 2 things.

 

Add it, and/or Checking 1st that the same code is not already there.

If you have double added this code the results will be undesirable.

 

Apart from this there is not much else to do.

 

This contrib works and works very well. It is one of the few contribs that works for 99% of people straight out of the box, so to speak.

 

Please check what other contribs you have got installed because clearly if you have done exactly as the instructions said to do, your having a conflict with one of them.

 

Please don't misunderstand my thoughts here, I am not saying we/I will not help anyone as much as possible, but you are going to have to look deeper at other changes that have been made to your cart.

 

regards

John

Share this post


Link to post
Share on other sites
I believe the above posters are having problems that are actually unrelated to WOS contrib.

In the who's online instructions there is a entry that must be added to: catalog/admin/includes/functions/general.php

 

If you have not added this section you need to do one of 2 things.

 

Add it, and/or Checking 1st that the same code is not already there.

If you have double added this code the results will be undesirable.

 

Apart from this there is not much else to do.

 

This contrib works and works very well.  It is one of the few contribs that works for 99% of people straight out of the box, so to speak.

 

Please check what other contribs you have got installed because clearly if you have done exactly as the instructions said to do, your having a conflict with one of them.

 

Please don't misunderstand my thoughts here, I am not saying we/I will not help anyone as much as possible, but you are going to have to look deeper at other changes that have been made to your cart.

 

regards

John

 

Okay, first, what do you mean it is unrelated with WOS? WOS is not working correctly so one way or another it's related to it.

 

Second, I don't think it's a problem with the bit of code you mentioned because generally the ip address shows up, what it's not doing is associating it with a bot. It just appears as a normal customer would.

 

Another thing I just noticed is that in the profile view it shows no session id, but on the bar below session it has a 'Y'. I'm going to go ahead and try reinstalling the contrib when I get a chance and see if that works. I'll post back with any updates.

 

Now you could be very right about it being a problem with something else installed. We should check into that. Everything was working fine with mine with version 1.6. It was when i upgraded to 1.7.1 that I started having problems.

 

I have a hefty list of contributions installed on my site. Check the sig for the list and anyone else tell us if you also have any of those installed that could be or isn't causing a problem.

Edited by Gidgidonihah

Share this post


Link to post
Share on other sites

Jason,

 

One of the differences between 1.6 and 1.7 is the location of the session files (no change if they're in mysql.) The change in 1.7 requires you to enter the FULL path in admin->Configuration->Sessions->Sessions Directory. Again, this only makes a difference if you store sessions in files.

 

The second big change is the logic on the catalog side. catalog/includes/functions/whos_online.php changed. Make sure you included that in the upgrade from 1.6.

 

Both of those changes could affect the way Bots are shown in WOE. WOE does NOT associate IPs with Bots. (Although an earlier version did.) It depends on Prevent Spider Sessions being turned on in admin->Configuration->Sessions. WOE (on the catalog side) checks a spider flag to determine if the visitor is a bot. The IP is just recorded for information sake. The Bot name is parsed from the User Agent on the admin side, again, just for information sake.

 

BTW, if the session id is blank and it's listed as a Guest, how do you know it's a Bot? Is the User Agent showing a Bot name? How about pasting the profile of one of these unidentified Bots. I'd be curious to see what is being reported.

 

ed


Answers to osCommerce's most persistent questions! Tips & Tricks | Configuration | Common Problems.

Seek and ye shall find Contributions.

My Contributions

My Blog

Share this post


Link to post
Share on other sites

Ed,

 

I checked out what you suggested and realized that i had failed to do both :blush: After making those changes the referer url is now being recognized, however both the user agent and last url remain blank. I haven't had a bot on yet, but i'll post back when I do.

 

The way that I know it is a bot is that I lookup the ip and it comes back to google inc or microsoft or inktomi.

 

As for the ip address, i worded it wrong. I know it doesn't associate a ip address with the bot, but what i was saying is that it wouldn't be the code that he had given me (which grabs the ip address) because the ips were showing up. if that makes any sense...

 

Anyway, i'll update if anything changes (or a bot shows up), thanks

Share this post


Link to post
Share on other sites
OK the last url works, but bots are still not showing up.  Here is a profile view while the googlebot was visiting my site like you asked.

Jason,

 

Do you have Force Cookie Use set to true?

 

ed


Answers to osCommerce's most persistent questions! Tips & Tricks | Configuration | Common Problems.

Seek and ye shall find Contributions.

My Contributions

My Blog

Share this post


Link to post
Share on other sites

I have V1.7.1 version of Who's Online, but I don't have user agents in my display. Did I miss something???

 

TIA

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×