Guest Posted November 16, 2004 Share Posted November 16, 2004 Hi there! I've installed SEO sitemap and it looks like it's working properly...except that I'm not seeing the engines drilling down any deeper than sitemap_products.html. When I look at the page with spider simulators all the links have an osCsid session on the end IE: http://www.raw-pleasure.com/pleasurecart/product_30.html?osCsid=85e4043eb212afe2280e552cec236144 Could this be affecting the spiders ability to follow the links? Quote Link to comment Share on other sites More sharing options...
Guest Posted November 16, 2004 Share Posted November 16, 2004 PS. I'd got Prevent Spider Sessions in oSc set to true already. Quote Link to comment Share on other sites More sharing options...
Guest Posted November 17, 2004 Share Posted November 17, 2004 OK... now they seem to be picking up sitemap_products and sitemap_categories but still not following the links in them (which show up fine when I look at them with the .js redirect disabled). Any ideas on this would be greatly appreciated! Quote Link to comment Share on other sites More sharing options...
wizardsandwars Posted November 17, 2004 Share Posted November 17, 2004 If your browser does not accept cookies, then YOU should see session IDs, but spiders shoud NOT. Quote ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help. Link to comment Share on other sites More sharing options...
Guest Posted November 17, 2004 Share Posted November 17, 2004 I'm wondering if I might be seeing the session ID's because I'm just using a spider simulator and because I'm not the spider itself... Would oSc's Prevent Spider Sessions block them for simulators as well, or are they just for a specific set of bots? Quote Link to comment Share on other sites More sharing options...
wizardsandwars Posted November 17, 2004 Share Posted November 17, 2004 It depends on what USER AGENT the simulator is using. Run the simulator, and look in your raw logs to tell what user agent the simulator is using. Or, if you have cpanel, there is an option to view the 'latest visitors' which also gives the user agent. Once you've determined the USER AGENT, you can look in your spiders.txt file to determine if you are blocking that user agent from having session ids attached to the urls. I'm wondering if I might be seeing the session ID's because I'm just using a spider simulator and because I'm not the spider itself... Would oSc's Prevent Spider Sessions block them for simulators as well, or are they just for a specific set of bots? <{POST_SNAPBACK}> Quote ------------------------------------------------------------------------------------------------------------------------- NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit. If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help. Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.