Latest News: (loading..)

Archived

This topic is now archived and is closed to further replies.

makeholer

About Google bot on my pages

14 posts in this topic

Hello,

 

I noticed that google always search on non-existent pages on my oscommerce, like this:

 

/product_reviews.php?language=es&cPath=2_21&products_id=12

 

I just deleted the language ES like 2 months ago, but they KEEP searching on this, this page do not exist, i've only one language, and its PT-BR, and i see they are searching on products id 1-70, but these products have been online for something like 3 days (from the original template) then i deleted, and my real products started on 80 or 90, why google keep searching non-existent pages? i've no links on my webpage to them, and im using ultimate URL and ive uploaded sitemaps with the ultimate url's correctly too.

 

obs: they search the correctly urls too, but i dont want they looking these wrong urls.

 

 

Marco Castellari

Share this post


Link to post
Share on other sites

Do the pages for your old language/old products return a 404? If not, Google will keep them for a long time. If there are external links to those pages, Google will also keep trying them.

 

Regards

Jim

Share this post


Link to post
Share on other sites

Jack,

Thanks, but i did this, i think this bot is retarded, other bots dont visit those pages, only visit correctly pages (like microsoft bot)

 

Jim,

it dont return a 404, i think it only redirect to a other valid page i dont think ive external to those pages, i cant do nothing to help the bot to see the correctly pages? (already uploaded a sitemap)

 

Regards,

Marco Castellari

Share this post


Link to post
Share on other sites

Hi Marco.

 

I am not that technically minded to be honest with you, I have just joined up to get some much needed help myself lol!

 

Your question interested me a rang a bell too:

 

I am a Moderator on a Forum and I have noticed that the Google Bots tend to search obsolete threads (sometimes over 7 years old) a hell of a lot more than any other Bots do. They very rarely search the latest threads and only ever do the moment they are incepted. That is what I have noticed personally.

 

I can understand your frustration, in an ideal world they would continually pick up links to your new stock as opposed to the stock that you have had for longer.

 

A good trick is to just edit the item/link in question in order to incorporate the new product, thereby allowing the Bots to pick up on the link and make amendments. That is what the Forum Administrator does from time to time, particularly in the For Sale Section, but I am not sure if that's the way it works in regards to online shopping etc.

 

My advice is to let it get on with it, it's definitely not retarded....

 

Now, I guess it's time for me to post up some questions so see you around I hope!

Share this post


Link to post
Share on other sites

Thanks Elli,

I think its not the case for me, i don think i can edit the URLS one per one, but really thanks for the tip.

 

Thanks Luca,

I'll try this, hope this can help me.

Share this post


Link to post
Share on other sites

it dont return a 404, i think it only redirect to a other valid page

Well, if it doesn't get a 404, but does go to a valid page, why would a search bot stop searching/indexing the link? As far as it can tell, it's still a valid page, and therefore should be followed.

 

You could modify your .htaccess file to '404' any URI with "language=es" in it, if Luca's pointer doesn't work out for you.

RewriteEngine On
RewriteCond  %{REQUEST_URI}  language=es  [NC]
RewriteRule  .   /language_es_does_not_exist  [L]

That might work (or something close to it -- I don't have my references at hand).

Share this post


Link to post
Share on other sites

MrPhill,

 

I really dont know if this lot of "redirect pages" can help me in any way to rank my website on google, or if with this lot of pages google can think ive lot of equal pages and then give me a bad position, do u know about that?

Share this post


Link to post
Share on other sites

I don't understand your question. It's a single entry in your .htaccess file to grab anything coming in containing language=es and send it off to a non-existent file that should trigger a 404 error. Google should then remove that entry from its index (perhaps not immmediately, but at latest after a second attempt). I don't know what the rest of your system looks like, as far as .htaccess and exactly how you would have to do this, but if you don't have any SEO mods installed, it might work as given.

Share this post


Link to post
Share on other sites

I have noticed that Google shows 145 not found pages for my site: the pages it indexed before I installed the new SEO URLs and changed them - several months ago. I believe it was in April when I made the change and Google is still trying to find the old URLs. Googlebot indexes my site several times per month, so I don't see any evidence that bad links will be removed from Google's index, maybe ever.

 

In my case the bad links Google keeps trying to find are the product pages without the category name in the URL. The new URLs have the category name but the old did not. I really wish Google would stop looking for them and telling me there are problems.

 

Remember, Google ranks pages, not sites. I don't know that having indexed links that return a 404 error effect ranking. I haven't seen anything to make me think it would.

Share this post


Link to post
Share on other sites

Don't expect this to get fixed anytime soon. I converted a site to osCommerce from an old ASP template in 2003. Google is still trying to find some of those ASP pages today.

 

On the other hand, it doesn't seem to hurt the site's ranking.

 

Regards

Jim

Share this post


Link to post
Share on other sites

Interesting thread.

 

From what I understand from reading it, even if Google is trying to index pages connected with a language that has been removed, language=de for example, there is nothing that can be done, ie set up a 301 redirect back to the home page for all pages containing language=de

 

Also, I dont think it is connected to old pages. de has not been a language on my site for donkeys years yet you can still see in the logs Google trying to open newly added categories in german

 

I would prefer not to have thousands of 404s on the account so is there a 301 option?

Share this post


Link to post
Share on other sites

Does this do the job for you? It should send any request for a German page to the main entrance and tell Google to forget about the old page. An alternative would be to go to a "Sorry, we don't have any German pages anymore." page.

RewriteEngine On
RewriteCond  %{REQUEST_URI}  language=de  [NC]
RewriteRule  .   /index.php  [R=301,L]

Share this post


Link to post
Share on other sites