Google: Increase in soft 404 errors. How does one create a custom 404 page?

[WebDev22 0]

We received an email from Google about soft 404 errors. Step one was to set up a custom 404 error page, but the add-on we tried isn't working: http://addons.oscommerce.com/info/933.

 

What is the best way to set up custom 404 pages?

[♥kymation 631]

Here's a simple one that I wrote a while back. Change the text in the language file as you see fit.

 

Regards

Jim

[WebDev22 0]

Thanks, Jim. I uploaded it to our site, but must be doing something wrong. When I go to our site (http://bit.ly/12uC0Y1) then add an incorrect/non-existent page name to the URL string, I don't get the custom 404 page. Any idea what I might be doing wrong?

[Jack_mcs 1,020]

@@WebDev22 Having a custom 404 page won't solve the problem, although you certainly should have one. The problem is that the code in oscommerce doesn't return a 404 so the page would never be called. You need to install the Header Status Handler addon. For having a 404 pages, as well as many others, the Custom Server Error works well.

[MrPhil 648]

A "soft" 404 would be a missing page redirected to another (real) page. If you give a wrong product ID or whatever, it appears to be redirecting back to your home page (index.php). If you give a really missing page, such as http://domain.com/zappa, you get the server default 404 page. A custom 404 page would be something like /404.shtml, which allows you to make it look something like your site.

 

A soft 404 isn't going to either the default or custom 404 error page. I'm not sure what Google told you, unless they're saying that they'd prefer to see bad pages deliver a real 404 error and not get redirected to another page.

[Jack_mcs 1,020]

@@MrPhil If you delete a product and then try to visit the product page with the ID of the deleted product, the product info page still loads. It just says the product can't be found but it returns a 301 or 302. This is the usual cause of the soft 404's from google. They are expecting either a 404 or 410 status on such products.

[MrPhil 648]

Haven't I seen somewhere an add-on to take a no-longer existing product ID and return a 404 code, either at index.php home page or an error document-type page (e.g., "product not found" product page)? As it works now, it's apparently redirecting to index.php with a 301 or 302, or even just a 200. I could see that being a problem with duplicate content, from Google's POV. I don't see why you couldn't return a 404 or 410 code while ending up at index.php (home page), which should prevent its being duplicate indexed.

[Jack_mcs 1,020]

The Header Status Handler I mentioned does that. If you redirect to the home page that would be fine for the search engines but people may have old url's too. So if a former customer clicks on a url that is for a deleted product, they should be told why it didn't show up. Besides, all sites should have 404 pages. If a search engine or visitor visits an invalid url, even if it is by accident by the people, the server first checks for a private 404 page and if it can't be found, loads the default one. That can amount to a lot of extra bandwidth usage and server load over the course of a month on a busy site. Not really anything to be concerned with by itself but is just another one of those things that slows a site down.

[MrPhil 648]

Keep in mind that a discontinued product ID isn't going to trigger a normal 404 page. A 404 error document will only be triggered if a requested directory or page is not found. If you get to index.php or whatever with a bad product_id, it won't trigger a 404 page. osC needs to handle that itself, by taking whatever action the programmer thought appropriate. If Google is complaining that this is leading to duplicate pages or whatever, perhaps osC is going to some page with a 301, 302, or 200 code, when it should be giving a 404 code. I can't swear that's the proper way, as I wouldn't want index.php itself to be marked as "not found" and dropped from the SE index.

 

Certainly, there should be some message from osC telling the user why they ended up on a different page than expected. How that will interact with search engines is an interesting question.

[Jack_mcs 1,020]

The issue may be clouded due to they dynamic nature of the products page. But if you try to go to an invalid product, if the product isn't found, then a 404 should be issued since it was not found. The product_info file was found but not the page for that product.

 

The 404 isn't issued for the page being redirected to so if you redirect to index.php that page won't be hurt. Let's say you had a product with and id of 123 and then deleted it. Google is saying (paraphrasing here), "we have this link to product 123 but when we go there your code is telling us to go somewhere else. We expect it to be a 404 or 410, depending upon if you plan on bringing it back or not." They don't want to be redirected since it is more work for them.

[vampirehunter 16]

what is best option for this?

 

do you install both "header status handler" and this one http://addons.oscommerce.com/info/5863 ?

 

 

i would like to have a 404 page also if someone types in a wrong url or page which doesn't exist,

and would also like them to be redirected or informed if they enter a page who's product id is incorect or now deleted.

 

what other plugins should i need apart from those two?

 

thanks

[Jack_mcs 1,020]

For issuing the 404's for deleted or disabled products, categories and manufacturers, use the Header Status Handler. For the actual 404 page, you can install that addon, the code Jim mentioned or just upload a 404.shtml file, edited as you like. The addon I mentioned has the advantage of being part of the shop and handling other errors too.

[MrPhil 648]

if you try to go to an invalid product, if the product isn't found, then a 404 should be issued since it was not found. The product_info file was found but not the page for that product.

The point is, an invalid product does not trigger a server 404, since the file (product_info.php or whatever) was found. osC has to do some special operations to return a 404 or 410 while going to some other page. It can be done, but is not automatic. osC has to be explicitly coded to do this.

[Jack_mcs 1,020]

Exactly. That's why I wrote the Header Status Handler addon. :)

[♥bruyndoncx 260]

Just to chime in as an alternative,

 

For a product that no longer exists, I still show the product detail in a greyed out manner without pricing, and then list other products ie alternatives from the same category. (under the product_info.php?products_id=1234 page ...

I'd rather show them good alternatives, then a redirect to the homepage where they might get lost unable to find the product again..

As far as google is concerned, the old product is still a valid page.

We have a fair amount of service products for products that are no longer sold, so this works quite well.