Guest Posted April 29, 2003 Share Posted April 29, 2003 Hello all, I've just finished building an art gallery site, and I think I?ve done everything to please google, but my site still hasn?t been fully crawled. Its got as far as my root page, and indexed the links from there but nothing else. Can anyone give me any tips on what I might be doing wrong? Many thanks Caspar FyersArt.com Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 Sometimes it takes a couple of crawls before they fully crawl your site. A couple of things you can do though......You need a spider catcher/SID killer. I used this http://www.webconfs.com/search-engine-spid...r-simulator.php and it came out full of SIDs. Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 ah right, would you recommend any contribution in particular? Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 There is a thread in the Tips and Tricks that has one and there is another by Winterradio in the Contributions section. There is a third by Ian Wilson but, it has some conflicts at the moment. Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 i'm havinf trouble finding any other SID Contributions other than the one by Ian Wilson.... :? Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 http://www.oscommerce.com/community/contributions,1089 Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 ahhh cheers for that :wink: Do you think thats better than the one by Ian Wilson? Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 It works in all the situations I have used it in. Ian's has some problems and even he is recommending not using it. Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 through searching the forums, some people think the spider catcher is a form of redirection, which google doesnt like... whats your opinion on that? Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 In my opinion, that is complete bunk. I have the spider catcher installed on my site and the last Google crawl on my site resulted in more than 200 listings on Google. If they don't like it they have a funny way of showing it. :D The main purpose of the spider catcher is to identify the spider and make sure that it does not generate the session ids in the listings (could be a security problem) and to keep it from getting "stuck" on a site and using huge amounts of bandwidth. Link to comment Share on other sites More sharing options...
burt Posted April 29, 2003 Share Posted April 29, 2003 GoogleGuy has confirmed that stripping SIDs from the URLs (which is what the main SID killer script does), is not re-directing, cloaking etc. He has no problem with it whatsoever. FYI, GoogleGuy is a bit of a mysterious character, who appears to be high up in Google, maybe a senior engineer or somesuch. Link to comment Share on other sites More sharing options...
dreamscape Posted April 29, 2003 Share Posted April 29, 2003 hey burt maybe you could take googleguy out for drinks and get him to reveal the truth and secrets about Google's indexing and ranking algorithm :wink: The only thing necessary for evil to flourish is for good men to do nothing - Edmund Burke Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 Errm, excuse me for my ignorance, but is a SID killer exactly the same as a spider catcher? :oops: Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 Yes, the contribution performs BOTH duties. Link to comment Share on other sites More sharing options...
knipper Posted April 29, 2003 Share Posted April 29, 2003 Will the sid killer at http://www.oscommerce.com/community/contributions,1089 work in the CSV version 2.2? Steve K AKA - Knipper - Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 I haven't tried it on a recent snapshot but, I don't know why it wouldn't. I have it installed on an MS1 version. Link to comment Share on other sites More sharing options...
knipper Posted April 29, 2003 Share Posted April 29, 2003 HHhmmm.... When I went to the contribution he states this: If you are using cvs after 19.03.2003 you don't need it any more, cause a similar thing is implemented Is there a way to tell if my version has this implementation? Anyone have an idea what I'd look for? My csv is actually from a couple weeks ago. Steve K AKA - Knipper - Link to comment Share on other sites More sharing options...
Daemonj Posted April 29, 2003 Share Posted April 29, 2003 Admin ~ Configuration ~ Sessions ~ set Prevent Spider Sessions to true. If Prevent Spider Sessions is not listed then you do not have it. ;) "Great spirits have always found violent opposition from mediocre minds. The latter cannot understand it when a man does not thoughtlessly submit to hereditary prejudices but honestly and courageously uses his intelligence." - A. Einstein Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 Thanks Jim! I'm just in the process of installing yesterday's CVS. How's the built in spider catcher working for you? Link to comment Share on other sites More sharing options...
Daemonj Posted April 29, 2003 Share Posted April 29, 2003 No problem Kim :) It is working very well. Is this Burt's spider killer? "Great spirits have always found violent opposition from mediocre minds. The latter cannot understand it when a man does not thoughtlessly submit to hereditary prejudices but honestly and courageously uses his intelligence." - A. Einstein Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 I haven't looked at the source code yet but, I know both Burt's and Winterradio's work similarly. My connection is really slow today and I'm still uploading the latest snapshot...I'll look at the changes and let you know what I think. Link to comment Share on other sites More sharing options...
burt Posted April 29, 2003 Share Posted April 29, 2003 It's based on my logic (I even got a mention in the weekly newsletter thing) - however the code has been modified somewhat, and really bears little to no resemblance to my original script. For anyone using it, there is a typo in spider.txt, scroll down in that file and seperate kit_fireballlachesis to kit_fireball lachesis hth Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 Cool Burt! Congrats! I missed that in the news. It's gonna' be nice having it built in. Link to comment Share on other sites More sharing options...
knipper Posted April 29, 2003 Share Posted April 29, 2003 Well.... I am not lucky enough to have a newer version installed, so I am attempting to install the sid killer mentioned before. I followed the directions, and either missed something or did it incorrectly. I get a 404 error when I try to access the page spider_configure.php Anyone have a tip? Steve K AKA - Knipper - Link to comment Share on other sites More sharing options...
Guest Posted April 29, 2003 Share Posted April 29, 2003 Did you... - add at the end of your application_top.php before ?> // Spiderkiller require(DIR_WS_INCLUDES . 'spider_configure.php') Link to comment Share on other sites More sharing options...
New 
Recommended Posts
Archived
This topic is now archived and is closed to further replies.