I have my shop up and running but I would like to get more hits I have listed it with a local search engine and added mega-tags. But I have been reading about Spiders but I have no idea how they work can anyone please help??
Latest News: (loading..)
Spiders?
Started by Pixy, Jul 28 2005, 12:02
10 replies to this topic
#1
Posted 28 July 2005, 12:02
Pixy
One day at a time - we all have to learn
One day at a time - we all have to learn
#2
Posted 28 July 2005, 12:04
Sure, what would you like to know?
-------------------------------------------------------------------------------------------------------------------------
NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.
If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.
If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
#3
Posted 28 July 2005, 12:07
wizardsandwars, on Jul 28 2005, 12:04 PM, said:
Sure, what would you like to know?
Pixy
One day at a time - we all have to learn
One day at a time - we all have to learn
#4
Posted 28 July 2005, 12:08
Pixy, on Jul 28 2005, 08:02 PM, said:
I have my shop up and running but I would like to get more hits I have listed it with a local search engine and added mega-tags. But I have been reading about Spiders but I have no idea how they work can anyone please help??
spiders (robots) are like automated visitors which read your webpages, evaluate them according to their own criteria and list them on their search engines.
They read everything you give them but cannot (yet) understand javascript, flash, images and forms.
Treasurer MFC
#5
Posted 28 July 2005, 12:13
Do I need to add any scripting for the spiders to detect my site?
Pixy
One day at a time - we all have to learn
One day at a time - we all have to learn
#6
Posted 28 July 2005, 12:17
Nope.
But I would suggest in your admin setting 'Prevent Spider Sessions' to TRUE. This will prevent the spider from getting SIDs appended to the URLs they visit at your store.
But I would suggest in your admin setting 'Prevent Spider Sessions' to TRUE. This will prevent the spider from getting SIDs appended to the URLs they visit at your store.
-------------------------------------------------------------------------------------------------------------------------
NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.
If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.
If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
#7
Posted 28 July 2005, 12:29
SIDs? What is SID and what risk does it pose to the site?
Pixy
One day at a time - we all have to learn
One day at a time - we all have to learn
#8
Posted 28 July 2005, 12:37
An SID is a number assigned to a visitor to keep track of things like what you have put in your cart.
When a spider starts getting assigned an SID, a couple of bad things can happen. 1.) the spider can get stuck in an infinate loop on your webpage, sucking up bandwidth and system resources 2.) your listings in the seach engines index can have SIDs in them, possibly allowing session hijacking, or errors when visitors click on them, and 3.) URLs with session id are often dropped from seach engines.
By setting 'Prevent Spider Session' to TRUE the cart will not assign SIDs to spiders it recognizes.
When a spider starts getting assigned an SID, a couple of bad things can happen. 1.) the spider can get stuck in an infinate loop on your webpage, sucking up bandwidth and system resources 2.) your listings in the seach engines index can have SIDs in them, possibly allowing session hijacking, or errors when visitors click on them, and 3.) URLs with session id are often dropped from seach engines.
By setting 'Prevent Spider Session' to TRUE the cart will not assign SIDs to spiders it recognizes.
-------------------------------------------------------------------------------------------------------------------------
NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.
If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
NOTE: As of Oct 2006, I'm not as active in this forum as I used to be, but I still work with osC quite a bit.
If you have a question about any of my posts here, your best bet is to contact me though either Email or PM in my profile, and I'll be happy to help.
#9
Posted 28 July 2005, 12:41
Thanks for the info. I need help one other thing I want to add a downloadable document for clients to os-commerce but I have no-idea on how to do it. Can you help?
Pixy
One day at a time - we all have to learn
One day at a time - we all have to learn
#10
Posted 15 October 2005, 16:35
Pixy, on Jul 28 2005, 01:41 PM, said:
Thanks for the info. I need help one other thing I want to add a downloadable document for clients to os-commerce but I have no-idea on how to do it. Can you help?
Hi,
I have just set 'Prevent Spider Sessions' to TRUE. in my admin. One thing,can someone explain,when I look at my site and ask'show robot txt' it comes back no such text on your server.I see others do have robot text,is it something I need to add so robots find my site,if so how do I go about it?
Cheers
#11
Posted 18 October 2005, 14:00
robots.txt is to actually instruct the robots how/where to access pages on your site. Should be called robots.txt and placed in the root of your directory.
so for example the content of that file would look like:
In this case for all robots we block the private and images directories. To allow everything you leave the "Disallow:" field blank.
The problem is if the robots can d/l this file, anyone can and this offers hints of the structure of your site.
so for example the content of that file would look like:
# Robots.txt file User-agent: * Disallow: /private/ Disallow: /images/
In this case for all robots we block the private and images directories. To allow everything you leave the "Disallow:" field blank.
The problem is if the robots can d/l this file, anyone can and this offers hints of the structure of your site.














