Jump to content



Photo
- - - - -

Google Duplicate Content Manager version 1.0a


  • Please log in to reply
75 replies to this topic

#41   danta67

danta67
  • Members
  • 52 posts
  • Real Name:dan

Posted 30 September 2008 - 19:10

Hi there
do i put the code only in the index file or do i put it in all my root files
thanks danta67

#42   FWR Media

FWR Media
  • Members
  • 6,839 posts
  • Real Name:Robert Fisher
  • Gender:Male
  • Location:Stowmarket - Suffolk - UK

Posted 30 September 2008 - 19:35

Hi there
do i put the code only in the index file or do i put it in all my root files
thanks danta67


Depends on your site and its added contributions. Most sites are experiencing duplicates via index.php.

Are your duplicates index.php related?

#43   danta67

danta67
  • Members
  • 52 posts
  • Real Name:dan

Posted 30 September 2008 - 19:56

i have 5 sites and the one i want to install this i use headertag and seo-g
i looked at webmaster tools but its difficult for me to understand any tips were i have to look for the problems of my site for me

#44   nubbin

nubbin
  • Members
  • 52 posts
  • Real Name:Pete

Posted 03 November 2008 - 03:39

Robert,
Thanks for this very useful and clearly written contribution.

For a site that already has duplicated content in Google's index, does anyone know if Google would eventually drop the duplicated pages from its index if I use this setting?

var $IhaveDuplicateContent = false;

(I understand the purpose of the 'false' setting is to prevent duplicate content from occurring in the first place).

For example, Google has these duplicates:
product_info.php?cPath=1&products_id=15‎
product_info.php?currency=EUR&products_id=15‎
product_info.php?currency=GBP&products_id=15‎
product_info.php?currency=NZD&products_id=15‎

By setting the flag to "false" Google would no longer crawl the last 3 versions of the page. So would Google eventually drop them from its index?

For me this would seem to be the ideal situation as then Google would not hold pages with duplicate content in its index at all. It would avoid having pages in the index with essentially identical content and with title tags which are less than ideal.

Kind regards
Peter

#45   wi-fi

wi-fi
  • Members
  • 13 posts
  • Real Name:David

Posted 03 November 2008 - 22:42

I use template system and there are no problems.
It would depend on the version and contribution but most likely you will do modification in: /templates/templatename/main_page.tpl.php


I use STS version 2.01.... I have installed seo, SPPC, header tags controler....
When I install Google Duplicate Content Manager it dosen't make any difernce... it dosent work...
I have tryed adding code to sts files lihe I at google analytics and other contributions.... it dosent make any dieference..

Any idea?

#46   FWR Media

FWR Media
  • Members
  • 6,839 posts
  • Real Name:Robert Fisher
  • Gender:Male
  • Location:Stowmarket - Suffolk - UK

Posted 03 November 2008 - 22:54

I use STS version 2.01.... I have installed seo, SPPC, header tags controler....
When I install Google Duplicate Content Manager it dosen't make any difernce... it dosent work...
I have tryed adding code to sts files lihe I at google analytics and other contributions.... it dosent make any dieference..

Any idea?


It just has to be added around the code that actually prints to screen your tags. If installed correctly I've yet to find a situation where "it doesn't work".

#47   FWR Media

FWR Media
  • Members
  • 6,839 posts
  • Real Name:Robert Fisher
  • Gender:Male
  • Location:Stowmarket - Suffolk - UK

Posted 03 November 2008 - 23:00

Robert,
Thanks for this very useful and clearly written contribution.

For a site that already has duplicated content in Google's index, does anyone know if Google would eventually drop the duplicated pages from its index if I use this setting?

var $IhaveDuplicateContent = false;

(I understand the purpose of the 'false' setting is to prevent duplicate content from occurring in the first place).

For example, Google has these duplicates:
product_info.php?cPath=1&products_id=15‎
product_info.php?currency=EUR&products_id=15‎
product_info.php?currency=GBP&products_id=15‎
product_info.php?currency=NZD&products_id=15‎

By setting the flag to "false" Google would no longer crawl the last 3 versions of the page. So would Google eventually drop them from its index?

For me this would seem to be the ideal situation as then Google would not hold pages with duplicate content in its index at all. It would avoid having pages in the index with essentially identical content and with title tags which are less than ideal.

Kind regards
Peter


The jury is out on this one nubbin. I did a lot of research and testing but what G does and doesn't index, drop, keep in the index is something that needed testing.

The contribution had the two settings for good reasons but it is also possible .. as you suggest .. that false would have worked anyway. The reason for the true setting is that is "forces" G to see the pages as different which I viewed at the start as a pretty surefire way to remove the nasties.

As I haven't received any feedback I'm still at point one.

I suppose the good news for users is that nobody has returned to state that I have destroyed their rankings /biggrin.gif' class='bbc_emoticon' alt=':D' />

#48   nubbin

nubbin
  • Members
  • 52 posts
  • Real Name:Pete

Posted 03 November 2008 - 23:35

The jury is out on this one nubbin. I did a lot of research and testing but what G does and doesn't index, drop, keep in the index is something that needed testing.

The contribution had the two settings for good reasons but it is also possible .. as you suggest .. that false would have worked anyway. The reason for the true setting is that is "forces" G to see the pages as different which I viewed at the start as a pretty surefire way to remove the nasties.

As I haven't received any feedback I'm still at point one.

I suppose the good news for users is that nobody has returned to state that I have destroyed their rankings /biggrin.gif' class='bbc_emoticon' alt=':D' />


Robert, thanks for your reply. I will do an experiment to try and find the answer to this. For my most important site, which has been thoroughly indexed and so has much duplicate content, I will use the True setting as that seems to be a realtively risk free method. I have 2 other OSC sites which are relatively new, have far fewer pages and thus don't have much ranking. For those sites I will set the flag to False and I'll keep an eye on Google's site statistics over the next few months and see if the duplicate content gets dropped from the index. I'll let you know if it works as I hope.

Cheers
Peter

#49   FWR Media

FWR Media
  • Members
  • 6,839 posts
  • Real Name:Robert Fisher
  • Gender:Male
  • Location:Stowmarket - Suffolk - UK

Posted 03 November 2008 - 23:38

Robert, thanks for your reply. I will do an experiment to try and find the answer to this. For my most important site, which has been thoroughly indexed and so has much duplicate content, I will use the True setting as that seems to be a realtively risk free method. I have 2 other OSC sites which are relatively new, have far fewer pages and thus don't have much ranking. For those sites I will set the flag to False and I'll keep an eye on Google's site statistics over the next few months and see if the duplicate content gets dropped from the index. I'll let you know if it works as I hope.

Cheers
Peter


Thanks Peter that would be very much appreciated.

Rob

#50   nubbin

nubbin
  • Members
  • 52 posts
  • Real Name:Pete

Posted 05 November 2008 - 23:22

Thanks Peter that would be very much appreciated.

Rob


Rob,
OK my experiment seems to be working and remarkably quickly. In short, setting this contribution's mode to false on a site that is already indexed achieves the desired effect - the duplicates are removed from Google's index AND the pages left in Google's index have the title tags I want (not tags modified by the contribution).

Yesterday I implemented this contribution on a site which Google had indexed. Google was reporting 77 duplicate title tags for the site. I set the contribution's mode to false. This adds a "NOINDEX,FOLLOW" tag to the targetted pages.

Overnight Google crawled the site.

Today, 17 duplicated pages have now been removed from Google list of duplicate title tags - Google is now reporting only 60 duplicate title tags.

This is in line with what I expected based on Google's help page here : http://www.google.co...py?answer=93710

The major remaing problem I have is duplications like this:
/product_info.php?cPath=6&products_id=40
‎‎/product_info.php?products_id=40‎

If I was to add 'cpath' to Var $getValues, would that prevent these duplicates? Can you foresee any problems with that? I think it would work and am happy to try it if someone could give me a bit of reassurance I haven't overlooked a problem with doing this ! I am a bit wary of trying this for fear of accidentally screwing up my site's ranking!

Kind regards
Peter

#51   FWR Media

FWR Media
  • Members
  • 6,839 posts
  • Real Name:Robert Fisher
  • Gender:Male
  • Location:Stowmarket - Suffolk - UK

Posted 06 November 2008 - 13:22

Rob,
OK my experiment seems to be working and remarkably quickly. In short, setting this contribution's mode to false on a site that is already indexed achieves the desired effect - the duplicates are removed from Google's index AND the pages left in Google's index have the title tags I want (not tags modified by the contribution).

Yesterday I implemented this contribution on a site which Google had indexed. Google was reporting 77 duplicate title tags for the site. I set the contribution's mode to false. This adds a "NOINDEX,FOLLOW" tag to the targetted pages.

Overnight Google crawled the site.

Today, 17 duplicated pages have now been removed from Google list of duplicate title tags - Google is now reporting only 60 duplicate title tags.

This is in line with what I expected based on Google's help page here : http://www.google.co...py?answer=93710

The major remaing problem I have is duplications like this:
/product_info.php?cPath=6&products_id=40
‎‎/product_info.php?products_id=40‎

If I was to add 'cpath' to Var $getValues, would that prevent these duplicates? Can you foresee any problems with that? I think it would work and am happy to try it if someone could give me a bit of reassurance I haven't overlooked a problem with doing this ! I am a bit wary of trying this for fear of accidentally screwing up my site's ranking!

Kind regards
Peter


Thanks for feeding back Peter and glad it worked for you.

Regarding cPath this would not work as cPath is a valid link (category path) My suggestion for this would be ultimate seo urls as the reformatted urls do not suffer this problem.

#52   rochaesobrinho

rochaesobrinho
  • Members
  • 14 posts
  • Real Name:Paulo

Posted 07 November 2008 - 18:24

Hi Robert,

Thanks for the great contribution.
I have installed it and it is seems to work fine.
However, i have a question.
Today in my Google webmaster tools page appears only seven pages with duplicated meta description (my site has more than 100 products).
How should I proceed?
Should I set $IhaveDuplicateContent to TRUE and then when I perceive that the duplicated meta description no more exists on Google change it to FALSE?

#53   FWR Media

FWR Media
  • Members
  • 6,839 posts
  • Real Name:Robert Fisher
  • Gender:Male
  • Location:Stowmarket - Suffolk - UK

Posted 07 November 2008 - 18:55

Hi Robert,

Thanks for the great contribution.
I have installed it and it is seems to work fine.
However, i have a question.
Today in my Google webmaster tools page appears only seven pages with duplicated meta description (my site has more than 100 products).
How should I proceed?
Should I set $IhaveDuplicateContent to TRUE and then when I perceive that the duplicated meta description no more exists on Google change it to FALSE?


That is the method I initially invisaged when writing the contribution yes, so that is what I would recommend.

#54   tonyosc

tonyosc
  • Members
  • 2 posts
  • Real Name:tony

Posted 14 November 2008 - 14:29

THANKS. it's what i need .

Only one problem . i have installed HeaderTags_SEO , enable robots . so I got two name="ROBOTS" , as follow :

====
<meta name="googlebot" content="all" />
<meta http-equiv="Content-Language" content="en-US" />
<meta name="robots" content="index, follow" />
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<!-- EOF: Header Tags SEO Generated Meta Tags -->
<meta name="ROBOTS" content="NOINDEX, FOLLOW" />
==================


I want to know if it's ok for robots . hope robots dont get angry .

Edited by tonyosc, 14 November 2008 - 14:33.


#55   FWR Media

FWR Media
  • Members
  • 6,839 posts
  • Real Name:Robert Fisher
  • Gender:Male
  • Location:Stowmarket - Suffolk - UK

Posted 14 November 2008 - 14:39

THANKS. it's what i need .

Only one problem . i have installed HeaderTags_SEO , enable robots . so I got two name="ROBOTS" , as follow :

====
<meta name="googlebot" content="all" />
<meta http-equiv="Content-Language" content="en-US" />
<meta name="robots" content="index, follow" />
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
<!-- EOF: Header Tags SEO Generated Meta Tags -->
<meta name="ROBOTS" content="NOINDEX, FOLLOW" />
==================


I want to know if it's ok for robots . hope robots dont get angry .


I would guess that it is a bad idea to have the two versions.

#56   DimeNote

DimeNote
  • Members
  • 139 posts
  • Real Name:Unknown
  • Gender:Male

Posted 20 November 2008 - 15:08

Hello,

Just to verify, when I have this installed and turned on, I should see a "<meta name="ROBOTS" content="NOINDEX, FOLLOW" />" tag on the category pages and subcategory pages as well? Example:

Candy (this has noindex tag)
--Chocolate (this has noindex tag)
----Snickers (this does not have noindex tag)

Let me know if this makes sense and if this is proper behavior for this contribution. Thanks again.

#57   FWR Media

FWR Media
  • Members
  • 6,839 posts
  • Real Name:Robert Fisher
  • Gender:Male
  • Location:Stowmarket - Suffolk - UK

Posted 20 November 2008 - 20:26

Hello,

Just to verify, when I have this installed and turned on, I should see a "<meta name="ROBOTS" content="NOINDEX, FOLLOW" />" tag on the category pages and subcategory pages as well? Example:

Candy (this has noindex tag)
--Chocolate (this has noindex tag)
----Snickers (this does not have noindex tag)

Let me know if this makes sense and if this is proper behavior for this contribution. Thanks again.


Read the instructions.

#58   DimeNote

DimeNote
  • Members
  • 139 posts
  • Real Name:Unknown
  • Gender:Male

Posted 21 November 2008 - 17:27

Read the instructions.


Thanks for the tip. I'm using SEO URL's so the category tree path looks different to me and I momentarily forgot that they really are just a mirror of the original www.whatever.com/index.php?cPath=x. /wink.gif' class='bbc_emoticon' alt=';)' />

Like I said, just wanted to verify. Thanks.

Edited by DimeNote, 21 November 2008 - 17:29.


#59   clustersolutions

clustersolutions
  • Members
  • 75 posts
  • Real Name:Tim

Posted 13 January 2009 - 19:14

Just discovered our duplicate title/tag with Google yesterday and was glad to find this contribution. We make some mods to two methods to format the output. Nothing fancy just to get the job done quickly. BTW, we use Ultimate SEO and Header Tags SEO. This is for example if you have the need to mod the title/tag output. Test, test, test, and backup, backup, backup...good luck...Tim

function parseMeta(){

$addedMeta = '';
$count = count($this->caught);
for( $i=0; $i<$count; $i++ ){
$addedMeta .= $this->caught[$i] . '_' . tep_sanitize_string(tep_output_string_protected($_GET[$this->caught[$i]])) . '-';
}
$addedMeta = rtrim($addedMeta, '-'); //*** get rid of the last '-'
$addedMeta = preg_replace('/sort_2a/', 'Sort by Name Ascending', $addedMeta); //*** customize these as you may have more sort functions than I and they are not for name, style no., and price.
$addedMeta = preg_replace('/sort_2d/', 'Sort by Name Descending', $addedMeta);
$addedMeta = preg_replace('/sort_3a/', 'Sort by Style No. Ascending', $addedMeta);
$addedMeta = preg_replace('/sort_3d/', 'Sort by Style No. Descending', $addedMeta);
$addedMeta = preg_replace('/sort_4a/', 'Sort by Price Ascending', $addedMeta);
$addedMeta = preg_replace('/sort_4d/', 'Sort by Price Descending', $addedMeta);
$addedMeta = preg_replace('/page_/', 'Page ', $addedMeta); // *** Change page_ to Page
$this->addedMeta = $addedMeta;
$this->performPCRE();
}
/**
* PCRE replace operations to inject added meta info based on matched $_GET variables
*
* @param $this->finalMeta - Sets $this->finalMeta which contains the modified meat data
*/
function performPCRE() {

// $pattern[] = '@<title>\s*@i'; // *** original codes
// $pattern[] = '@<meta\s*name\s*=\s*"\s*description\s*"\s*content\s*=\s*"\s*@i';
// $pattern[] = '@<meta\s*name\s*=\s*"\s*keywords\s*"\s*content\s*=\s*"\s*@i'; // *** leaving keywords alone as Google complaints only duplicate title/tag
// $replace[] = '<title>' . str_replace('-', ' | ', $this->addedMeta);
// $replace[] = '<title>' . str_replace(array('-', '_'), ' ', ucfirst($this->addedMeta));
// $replace[] = '<meta name="description" content="' . str_replace('-', ' ', $this->addedMeta);
// $replace[] = '<meta name="keywords" content="' . str_replace('-', ', ', $this->addedMeta);

$pattern[] = '@\s*</title>@i';
$replace[] = ', ' . str_replace(array('-'), ', ', $this->addedMeta) . '</title>'; // *** the added meta is put at the end after title.

$this->meta = preg_replace('@\s*"\s*>@i', ', ' . str_replace('-', ', ', $this->addedMeta) . '">', $this->meta, 1); // *** if your first meta tag is not description then this won't work for you...it puts the added meta after description. A quick fix...

$this->finalMeta = preg_replace($pattern, $replace, $this->meta);
}

#60   multiman

multiman
  • Members
  • 2 posts
  • Real Name:Rodolphe

Posted 14 January 2009 - 05:46

Very nice job. Good contrib.

I've just a little problem.

I use STS template, SEO URL rewritting, SID Killer, Easy Meta tag.

so my url are like :

1) www.mysite.com/jeux-grande-taille-c-11.html
2) www.mysite.com/jeux-grande-taille-c-11.html?sort=3d&page=1
...

I've correctly install the contrib. She looks fine, I have the meta <meta name="ROBOTS" content="NOINDEX, FOLLOW" /> in the code
in the case 2) but also in the case 1)

After tracking, i've found that the ?sort is detected everytime (even if ?sort doesn't appear.) so I'll change your code like this

var $getValues = array( 'page', 'language', 'currency');

instead of

'sort', 'page', 'language', 'currency'); _linenums:0'>var $getValues = array([u]'sort',[/u] 'page', 'language', 'currency');

It's a good way to prevent duplicate content because of the meta. But have you a idea to prevent the url in case 1) ?

Thank you /rolleyes.gif' class='bbc_emoticon' alt=':rolleyes:' />