I have come across something that I have never had to work with before. I have a single category that I DON'T want Search Engines to crawl. Anyone have any ideas on how to stop indexing of just one category ?
Chris
Latest News: (loading..)
0
9 replies to this topic
#1
-
- Community Sponsor
-
- 10,464 posts
- Real Name:Chris Dunn
- Gender:Male
- Location:Tecumseh, Ontario, Canada N8N 1X8
Posted 15 August 2011, 18:14
:|: Was this post helpful ? Click the LIKE THIS button :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
#2
Posted 15 August 2011, 19:06
Since the category do have a link, I think you can do it using htaccess?
Sara
Sara
#3
-
- Community Sponsor
-
- 10,464 posts
- Real Name:Chris Dunn
- Gender:Male
- Location:Tecumseh, Ontario, Canada N8N 1X8
Posted 15 August 2011, 19:13
Hi Sara,
I have been researching this and don't think it can be done in .htaccess and although I can add a Disallow to the robots.txt, this will only work if the crawler follows the rules. Rogue crawlers will still index the category. I have to ensure they do not index it at all.
Any other suggestions ?
Chris
I have been researching this and don't think it can be done in .htaccess and although I can add a Disallow to the robots.txt, this will only work if the crawler follows the rules. Rogue crawlers will still index the category. I have to ensure they do not index it at all.
Any other suggestions ?
Chris
:|: Was this post helpful ? Click the LIKE THIS button :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
#4
Posted 16 August 2011, 21:50
If it appears on a page that a spider sees it will get indexed.
You can add the attribute rel="nofollow" to an anchor to keep the (good) bots from following the link but there is nothing to stop them from indexing the link.
So that leads us to the ultimate conclusion: Don't let any bot "see" the link and it won't get indexed.
And how do you do that?
Simple. Only display the link to logged in visitors.
Bots don't create accounts and don't log in.
Probably not what you wanted to hear, but it's the simplest solution.
You can add the attribute rel="nofollow" to an anchor to keep the (good) bots from following the link but there is nothing to stop them from indexing the link.
So that leads us to the ultimate conclusion: Don't let any bot "see" the link and it won't get indexed.
And how do you do that?
Simple. Only display the link to logged in visitors.
Bots don't create accounts and don't log in.
Probably not what you wanted to hear, but it's the simplest solution.
If I suggest you edit any file(s) make a backup first - I'm not perfect and neither are you.
"Headers already sent" - The definitive help
"Cannot redeclare ..." - How to find/fix it
SSL Implementation Help
Like this post? "Like" it again over there >
"Headers already sent" - The definitive help
"Cannot redeclare ..." - How to find/fix it
SSL Implementation Help
Like this post? "Like" it again over there >
#5
-
- Community Sponsor
-
- 10,464 posts
- Real Name:Chris Dunn
- Gender:Male
- Location:Tecumseh, Ontario, Canada N8N 1X8
Posted 17 August 2011, 00:15
Hi Jim,
No, that isn't what I wanted to hear. I appreciate the input though.
Chris
No, that isn't what I wanted to hear. I appreciate the input though.
Chris
:|: Was this post helpful ? Click the LIKE THIS button :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
#6
Posted 17 August 2011, 01:49
Not perfect, but working for google atleast...
[quote]<META NAME="ROBOTS" CONTENT="NOINDEX">[/quote]
or
[quote]<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">[/quote]
[quote]<META NAME="ROBOTS" CONTENT="NOINDEX">[/quote]
or
[quote]<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">[/quote]
Basics for osC 2.2 Design - Basics for Design V2.3+ - Seo & Sef Url's - Meta Tags for Your osC Shop - Steps to prevent Fraud... - MS3 and Team News... - SEO, Meta Tags, SEF Urls and osCommerce
Check out my profile [click here] for information on professional services, custom coding, templates, SEO optimization, modifications, commercial support and help.
Check out my profile [click here] for information on professional services, custom coding, templates, SEO optimization, modifications, commercial support and help.
#7
Posted 17 August 2011, 02:55
google is going to index everything regardless of what you tell them to do, they might not publicly show it though. Same goes for other engines.
If you want to 100% make sure googles eyes never see a particular page, your only option is to do some blackhat cloacking.
If you want to 100% make sure googles eyes never see a particular page, your only option is to do some blackhat cloacking.
#8
-
- Community Sponsor
-
- 10,464 posts
- Real Name:Chris Dunn
- Gender:Male
- Location:Tecumseh, Ontario, Canada N8N 1X8
Posted 17 August 2011, 02:57
toyicebear, on 17 August 2011, 01:49, said:
Not perfect, but working for google atleast...or
I want ALL but ONE category indexed. So the NO FOLLOW would not work.
Chris
:|: Was this post helpful ? Click the LIKE THIS button :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
#9
Posted 17 August 2011, 03:07
DunWeb, on 17 August 2011, 02:57, said:
I want ALL but ONE category indexed. So the NO FOLLOW would not work.
Chris
Chris
It will work if you write code making it show only for that category.
Edited by toyicebear, 17 August 2011, 03:08.
Basics for osC 2.2 Design - Basics for Design V2.3+ - Seo & Sef Url's - Meta Tags for Your osC Shop - Steps to prevent Fraud... - MS3 and Team News... - SEO, Meta Tags, SEF Urls and osCommerce
Check out my profile [click here] for information on professional services, custom coding, templates, SEO optimization, modifications, commercial support and help.
Check out my profile [click here] for information on professional services, custom coding, templates, SEO optimization, modifications, commercial support and help.
#10
-
- Community Sponsor
-
- 10,464 posts
- Real Name:Chris Dunn
- Gender:Male
- Location:Tecumseh, Ontario, Canada N8N 1X8
Posted 17 August 2011, 03:22
Thanks for the input Nick, that gave me an idea.
Chris
Chris
:|: Was this post helpful ? Click the LIKE THIS button :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
:|: Click Here to learn how I can help you with custom coding, add ons, security and templates :|:
:|: Need an Area Calculator, Pre-Paid Account, Virtual Pin, Auction or Layaway Add on ? Click Here :|:
