SteveDallas Posted July 7, 2008 Share Posted July 7, 2008 Are you aware that Who's Online won't display bots when Force Cookie Use is enabled? It always displays Googlebot because of the line in includes/functions/whos_online.php: if (( $spider_flag ) or (strpos ($user_agent, "Googlebot") > 0 )) But when Force Cookie Use is enabled, in application_top.php the $spider_flag variable never gets set to true because of the elseif clause: elseif (SESSION_BLOCK_SPIDERS == 'True') I have fixed this by changing the above line to: if (!$session_started && SESSION_BLOCK_SPIDERS == 'True') Seems to be working fine now. Wouldn't mind an expert second opinion though ;) Jeff, That's a good catch of an osC bug. Is this in MS2 or is it still a bug in the latest release? The whos_online code shouldn't check for Googlebot, but it's a good diagnostic. I'll update the troubleshooting instructions to include this case and your fix. --Glen Quote Link to comment Share on other sites More sharing options...
jeffwdesign Posted July 7, 2008 Share Posted July 7, 2008 That's a good catch of an osC bug. Is this in MS2 or is it still a bug in the latest release? The whos_online code shouldn't check for Googlebot, but it's a good diagnostic. I'll update the troubleshooting instructions to include this case and your fix. It's in applications_top.php for v2.2 RC2a, and as far as I can see it's always been there. Quote Link to comment Share on other sites More sharing options...
Guest Posted July 14, 2008 Share Posted July 14, 2008 Is it possible to adjust the length of the last URL? Some of my URLs are longer than the set value, so when you click on the last url, the link is broken. I looked in the file but could only find $referrer_wordwrap_chars for the referer URL Quote Link to comment Share on other sites More sharing options...
daller Posted July 21, 2008 Share Posted July 21, 2008 Is there a way to make it pick up "crawl.yahoo.net" as a bot, and not a customer? Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted July 22, 2008 Share Posted July 22, 2008 Is there a way to make it pick up "crawl.yahoo.net" as a bot, and not a customer? Install the latest version of spiders.txt as mentioned in the readme file. If Force Cookie Use is enabled in Configuration->Sessions, see this post for a fix: http://www.oscommerce.com/forums/index.php?s=&...t&p=1273336 --Glen Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted July 27, 2008 Share Posted July 27, 2008 Is there a way to make it pick up "crawl.yahoo.net" as a bot, and not a customer? In addition to my previous response, I just saw a "crawl.yahoo.net" access that didn't identify itself as a bot. Here's the profile display: 00:00:00 Guest rz311409.crawl.yahoo.net 22:57:07 22:57:07 /catalog/reviews.php Yes Not Found Name: Guest ID: 0 IP Address: 67.195.54.90 User Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.4) Gecko/20080721 BonEcho/2.0.0.4 This may be Yahoo!'s implementation of a link quality agent, much as MSN Search has been doing for over a year. I get a lot of 65.55.x.x accesses from Microsoft with normal user agent strings. Apparently, MSN follows up a crawler access with a second that uses a normal user agent to verify that the server isn't serving different pages to spiders than actual users. It may be something that we have to live with, because you can't key on the user agent string to flag them as bots. --Glen Quote Link to comment Share on other sites More sharing options...
*luvnlight* Posted July 29, 2008 Share Posted July 29, 2008 (edited) I need help please, I am upgraded to the latest version from 3.21_1 & am now getting this error 1054 - Unknown column 'hostname' in 'field list' select customer_id, full_name, ip_address, hostname, time_entry, time_last_click, last_page_url, http_referer, user_agent, session_id from whos_online order by time_last_click DESC [TEP STOP] I can't work out how to fix it. If I take it back to the original oscommerce files it works fine but try & put the contribution in & the error shows. Thanks Edited July 29, 2008 by *luvnlight* Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted July 29, 2008 Share Posted July 29, 2008 (edited) I need help please, I am upgraded to the latest version f5rom 3.21_1 & am now getting this error 1054 - Unknown column 'hostname' in 'field list' select customer_id, full_name, ip_address, hostname, time_entry, time_last_click, last_page_url, http_referer, user_agent, session_id from whos_online order by time_last_click DESC [TEP STOP] I can't work out how to fix it. If I take it back to the original oscommerce files it works fine but try & put the contribution in & the error shows. Thanks You will need to add the 'hostname' field to the whos_online table in your database. This is part of step 3 of the installation instructions, which read: 3. Change the table Whos_online in your database to add the new fields. Use phpMyAdmin to run the SQL Query: ALTER TABLE whos_online ADD http_referer VARCHAR(255) NOT NULL; ALTER TABLE whos_online ADD user_agent VARCHAR(255) NOT NULL; ALTER TABLE `whos_online` ADD `hostname` VARCHAR( 255 ) NOT NULL AFTER `ip_address` ; (Since you are upgrading from v3.21, you only need to do the last one.) --Glen Edited July 29, 2008 by SteveDallas Quote Link to comment Share on other sites More sharing options...
*luvnlight* Posted July 29, 2008 Share Posted July 29, 2008 Thanks that worked. I had tried to run the SQL but kept getting errors, just using the last one worked. Thankyou :) Quote Link to comment Share on other sites More sharing options...
BBunter Posted July 31, 2008 Share Posted July 31, 2008 (edited) I have just installed 3.5.2 and it all looks very good. However in the default display (profile display none) no IP addresses are shown (tho if I logon Admin is shown) When I say no IP address there is nothing - it is NOT showing 0.0.0.0 or 127.0.0.1...there is nothing in the IP ADDRESS coloumn. The whois database has been populated and has IP addresses in ip_address field. If I click Profile display all the IP addresses are shown but no DNS lookup is done (and hence bots are not shown) Details are shown below : 00:29:51 Guest 11:29:56 11:59:47 /madras-mild-pr-4095.html Yes Not Found Name: Guest ID: 0 IP Address: 38.98.120.73 User Agent: osCsid: Both /admin/includes/configure.php and /includes/configure.php has define('STORE_SESSIONS', 'mysql') Latest spiders.txt installed and Configuration/Sessions/Prevent Spider Sessions is TRUE. Any ideas? It is such a great contribution..being able to tell when bots only are on would be great. Regards Mike Edited July 31, 2008 by BBunter Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted July 31, 2008 Share Posted July 31, 2008 (edited) I have just installed 3.5.2 and it all looks very good. However in the default display (profile display none) no IP addresses are shown (tho if I logon Admin is shown) When I say no IP address there is nothing - it is NOT showing 0.0.0.0 or 127.0.0.1...there is nothing in the IP ADDRESS coloumn. The whois database has been populated and has IP addresses in ip_address field. [snip] Any ideas? It is such a great contribution..being able to tell when bots only are on would be great. Regards Mike Mike, It sounds as though you didn't install the new collector code, which is /catalog/includes/functions/whos_online.php. The code that resolves host names moved to this function to reduce the server load on the display code. The whos_online table in the database should have a 'hostname' field right after the ip_address field. If it isn't there, perform the following SQL command to add it to your database: ALTER TABLE `whos_online` ADD `hostname` VARCHAR( 255 ) NOT NULL AFTER `ip_address`; Let me know if this fixes your problem. --Glen Edited July 31, 2008 by SteveDallas Quote Link to comment Share on other sites More sharing options...
BBunter Posted July 31, 2008 Share Posted July 31, 2008 Mike,It sounds as though you didn't install the new collector code, which is /catalog/includes/functions/whos_online.php. The code that resolves host names moved to this function to reduce the server load on the display code. The whos_online table in the database should have a 'hostname' field right after the ip_address field. If it isn't there, perform the following SQL command to add it to your database: ALTER TABLE `whos_online` ADD `hostname` VARCHAR( 255 ) NOT NULL AFTER `ip_address`; Let me know if this fixes your problem. --Glen Glen, Thanks for the prompt reply. I have double checked the installation and all looks ok. \includes\functions\whos_online.php (about 4k) \admin\whos_online.php (about 40k) \admin\includes\languages\english\whos_online.php (about 3k) \admin\images\ Various gifs.... Also the whos_online db has the hostname field. I have put 2 pics of the output online...may you could take a look and see what you think ? http://www.benstead.net/img1.jpg http://www.benstead.net/img2.jpg Regards Mike Quote Link to comment Share on other sites More sharing options...
BBunter Posted July 31, 2008 Share Posted July 31, 2008 Glen, Thanks for the prompt reply. I have double checked the installation and all looks ok. \includes\functions\whos_online.php (about 4k) \admin\whos_online.php (about 40k) \admin\includes\languages\english\whos_online.php (about 3k) \admin\images\ Various gifs.... Also the whos_online db has the hostname field. I have put 2 pics of the output online...may you could take a look and see what you think ? http://www.benstead.net/img1.jpg http://www.benstead.net/img2.jpg Regards Mike I have un-installed and re-installed and now everything is working perfect so I guess I installed something somewhere wrong...sorry for wasting your time :| Regards Mike Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted July 31, 2008 Share Posted July 31, 2008 Mike, Check your copy of /includes/functions/whos_online.php for this comment: 2008 Feb 06 v3.5 Glen Hoag aka SteveDallas Moved hostname resolution here to reduce admin tool load If it isn't there, replace the file with the one from the v3.5.2 distribution. If it is there, check that the hostname field of the whos_online table is being populated in your database. There may be an issue with DNS resolution on your server. --Glen Quote Link to comment Share on other sites More sharing options...
mathius Posted August 1, 2008 Share Posted August 1, 2008 Ive installed Who's online enhancement 3.52, for some reason when a customer is using cookies the Last URL only shows as / if I constantly refresh it will show the Last URL only while the page is loading on the clients browser. if the customer is idleand I refresh the admin>whos online page the last url goes back to / this only happens if the customer isnt using cookies, If they have oscid in their URL it functions fine.... any help would be appreciated Quote Link to comment Share on other sites More sharing options...
mathius Posted August 1, 2008 Share Posted August 1, 2008 Ive installed Who's online enhancement 3.52, for some reason when a customer is using cookies the Last URL only shows as / if I constantly refresh it will show the Last URL only while the page is loading on the clients browser. if the customer is idleand I refresh the admin>whos online page the last url goes back to /this only happens if the customer isnt using cookies, If they have oscid in their URL it functions fine.... any help would be appreciated EDIT *this only happens if the customer IS using cookies, If they have oscid in their URL it functions fine.... /EDIT Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted August 2, 2008 Share Posted August 2, 2008 EDIT *this only happens if the customer IS using cookies, If they have oscid in their URL it functions fine.... /EDIT This is very strange, as the customer populates the Last URL field in the database. Refreshing the admin viewer shouldn't change anything. Check the whos_online table in the database to verify that the field is being populated and matches what is displayed. --Glen Quote Link to comment Share on other sites More sharing options...
mathius Posted August 3, 2008 Share Posted August 3, 2008 This is very strange, as the customer populates the Last URL field in the database. Refreshing the admin viewer shouldn't change anything. Check the whos_online table in the database to verify that the field is being populated and matches what is displayed. --Glen Hi Steve thanks for yoru reply I checked with phpmyadmin and the whos_online table displays / also, If I refresh phpmyadmin while a customer is loading a page it dispays the correct url, once the page has loaded it goes back to / again this only happens if a customer uses cookies. If I disable cookies by deleting define('HTTP_COOKIE_DOMAIN') and define('HTTP_COOKIE_PATH') in my configure.php whos_online LAST URL works fine. Quote Link to comment Share on other sites More sharing options...
jeffwdesign Posted August 3, 2008 Share Posted August 3, 2008 This may be Yahoo!'s implementation of a link quality agent... I've been watching the activity of this BonEcho UA Yahoo bot for a while now. I think it is asking for far too many pages for it to be checking for cloaking. I think it is more likely that this bot is simply making screenshots. Adding "bonecho/2.0.0.4" to spiders.txt will prevent it from being given a session and will display it as a bot in Who's Online. (There can't be any real customers using this old beta of the Firefox 2.0 browser.) Quote Link to comment Share on other sites More sharing options...
scaistar Posted August 4, 2008 Share Posted August 4, 2008 have same problem of Daller Module not recognize this bot msnbot-65-55-209-28.search.msn.com lj511482.crawl.yahoo.net Have updated spider.txt but not solve for me MORE: - i prefer GEOIPTOOL to showmyip.com. If you don't want substitute it in next release.. it would be a great idea adding a GLOBE icon to the left of IP referred to Geoiptool - Italian language have a parse error at TEXT_SHOW_BOTS Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted August 4, 2008 Share Posted August 4, 2008 have same problem of DallerModule not recognize this bot Have updated spider.txt but not solve for me Spiders.txt contains fragments of User Agent strings, not host names. Therefore, adding "bonecho/2.0.0.4" should get rid of the new Yahoo! crawler, but the MSN search link quality bot uses user agent strings of actual browsers, so it's hard to identify. MORE:- i prefer GEOIPTOOL to showmyip.com. If you don't want substitute it in next release.. it would be a great idea adding a GLOBE icon to the left of IP referred to Geoiptool - Italian language have a parse error at TEXT_SHOW_BOTS Geo IP Tool http://www.geoiptool.com is a nice free IP locating tool. Thanks for bringing it to my attention. I will consider it for a future release. Sorry about the parse error in the Italian language file. The correct line is: define('TEXT_SHOW_BOTS', 'Visualizzi Spiders'); This will be corrected in the next release. --Glen Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted August 4, 2008 Share Posted August 4, 2008 Hi Steve thanks for yoru reply I checked with phpmyadmin and the whos_online table displays / also, If I refresh phpmyadmin while a customer is loading a page it dispays the correct url, once the page has loaded it goes back to / again this only happens if a customer uses cookies. If I disable cookies by deleting define('HTTP_COOKIE_DOMAIN') and define('HTTP_COOKIE_PATH') in my configure.php whos_online LAST URL works fine. This is very strange. Is there anything unusual about your installation, or your server configuration? Quote Link to comment Share on other sites More sharing options...
scaistar Posted August 5, 2008 Share Posted August 5, 2008 Added "bonecho... but still have Yahoo crawler :huh: PS: Visualizza spiders not Visualizzi Spiders :P Quote Link to comment Share on other sites More sharing options...
SteveDallas Posted August 5, 2008 Share Posted August 5, 2008 Added "bonecho... but still have Yahoo crawler :huh: PS: Visualizza spiders not Visualizzi Spiders :P I added "bonecho/2.0.0.4" to catalog/includes/spiders.txt and it is now being detected as a bot. Here's how it looks in context: boitho bonecho/2.0.0.4 booch Thanks for the spelling correction. :-) --Glen Quote Link to comment Share on other sites More sharing options...
mathius Posted August 7, 2008 Share Posted August 7, 2008 only pop up pages or pages Ive added myself such as www.store.com/dynamicsitemap.php or www.store.com/enquire.php are displayed correctly, Store urls are showing only as / ie index.php, product_info.php etc Ive attached a pic I can only see what bots are viewing, I have prevented sessions for bots, any customer using cookies shows only as / Any help would be appreciated, I have been searching for days for a solution? Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.