| |
|
| :: |
| Author |
Message |
triple
Joined: 18 Dec 2007
Posts: 11
Location: San Cristobal de las Casas - Mexico
|
 Posted: Mon Jan 21, 2008 7:10 am Post subject: remuve url tool for duplicate indexed page on google |
|
|
hallo everybody! 
i' ve installed rewrite mod and other good stuff from this wonderful site. Recently i've updated it from simple mod to mixed mod. Everything works fine but now my indexed pages on google presents duplicates that are linked to the same page, example:
*mysite.com/forum/viewtopic.php?p=49&sid=d6edb4ec3eed0885874a3cc2b6ba316e (old version with sid, link works)
*mysite.com/forum/viewtopic.php?p=49 (old version without sid, link works)
*mysite.com/forum/post49.html (old simple mod version, link works)
*mysite.com/forum/topic40.html#49 (actual ok version)
same thing for forums and subforums pages, ex.:
*mysite.com/forum/viewforum.php?f=43&sid=38bea68463f83640d37f50ec51a31b0e (old version with sid, link works)
*mysite.com/forum/viewforum.php?f=43 (old version without sid, link works)
*mysite.com/forum/forum43.html (old simple mod version, now obviously give 404)
*mysite.com/forum/catXX.html (old simple mod version, now obviously give 404)
*mysite.com/forum/name-of-forum-f43.html (actual ok version)
now i've eliminate sids from board and create a robots.txt that hide also
/forum/viewtopic.php
/forum/viewforum.php
/forum/post
/forum/cat
/forum/forum
i know that i can wait weeks for a total recrawls of my site but i'd like to eliminate old cached pages from google index using the 'official url eliminate tool'.
Before doing this, i want to be sure that eliminating the .php url (or one of the others), with this tool, i will not delete for the future also the newest .html url that link to the same page content!
What do you suggest? is better wait that google automatically update these urls in next months, if they can update it? or can i use this tool without any problem for my .html urls?
thanks! |
|
|
| Back to top |
|
 |
|
|
dcz
Administrateur - Site Admin
Joined: 28 Apr 2006
Posts: 13354
|
| Posted: Mon Jan 21, 2008 7:44 am Post subject: Re: remuve url tool for duplicate indexed page on google |
|
|
| triple wrote: | | Recently i've updated it from simple mod to mixed mod. |
Then you need to keep the old forum rewriterule up and running in your .htaccess (there is one or two depending on if you are using phpBB2 or phpBB3).
And yes, this means we'd allow another kind of duplicate (forumxx.html and forum-title-fxx.html), this in order to let the zero duplicate (looks like you are using phpBB2) do it's job.
The good thing is, it will as well take care of your other duplicates HTTP 301 redirecting them.
So install the zero duplicate and remove your robots.txt for a month or so, before you put it back on, the idea behind it is that it will be faster and better SEO wise (since HTTP 301 redirecting will transmit PR) to first let the bot acknowledge the redirection and then make sure they do not use the old links any more.
It's faster than to first wait all the old url are dropped thanks to the robots.txt and then wait until the new one are indexed. Because disallowed urls won't be crawled as much any more. |
_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________
Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche |
|
| Back to top |
|
|
triple
Joined: 18 Dec 2007
Posts: 11
Location: San Cristobal de las Casas - Mexico
|
| Posted: Mon Jan 21, 2008 8:13 am Post subject: Re: remuve url tool for duplicate indexed page on google |
|
|
| dcz wrote: |
Then you need to keep the old forum rewriterule up and running in your .htaccess (there is one or two depending on if you are using phpBB2 or phpBB3). |
It is an orion premodded phpbb2, so can tthe two rules works togheter...and i need to rewrite also the older one, but in what order?
| dcz wrote: | | And yes, this means we'd allow another kind of duplicate (forumxx.html and forum-title-fxx.html), this in order to let the zero duplicate (looks like you are using phpBB2) do it's job. |
zero duplicate is yet active from some days and analizing robots.txt results it work fine
| Quote: | | The good thing is, it will as well take care of your other duplicates HTTP 301 redirecting them. |
oldest viewtopic.php and viewforum.php urls works and automatically redirect on newst topicxxx.hmtl urls, but how can i redirect an html page that don't exist on my ftp browser window (catxx.html or forumxx.html)?
| Quote: | | So install the zero duplicate and remove your robots.txt for a month or so, before you put it back on, the idea behind it is that it will be faster and better SEO wise (since HTTP 301 redirecting will transmit PR) to first let the bot acknowledge the redirection and then make sure they do not use the old links any more. |
zero duplicate is up.
this thing scare me a little bit...i think you mean to remove from robots.txt all about phpbb folder...because there are in root other folders to protect, or not?
and after i delete phpbb folder info from robots.txt, all .php url will be recrawled and indexed, or not?
my PR is null, site is up from years but only now is time to promote it
| Quote: | | It's faster than to first wait all the old url are dropped thanks to the robots.txt and then wait until the new one are indexed. Because disallowed urls won't be crawled as much any more. |
i've read on google webmaster tools that it was for 6 months, but i believe if you says that is forever....also if i don't mind if google will not show pages like memberlist.php or similar. th question is: if i use the eliminate url tool for eliminate a wiewtopic.php url also new topicxx.html page will be deleted?
thanks! |
|
|
| Back to top |
|
|
SeO
Administrateur - Site Admin
Joined: 15 Mar 2006
Posts: 3103
|
| Posted: Tue Jan 22, 2008 8:13 am Post subject: Re: remuve url tool for duplicate indexed page on google |
|
|
| triple wrote: |
It is an orion premodded phpbb2, so can tthe two rules works togheter...and i need to rewrite also the older one, but in what order? |
what do you mean by the old one ?
| triple wrote: |
oldest viewtopic.php and viewforum.php urls works and automatically redirect on newst topicxxx.hmtl urls, but how can i redirect an html page that don't exist on my ftp browser window (catxx.html or forumxx.html)?
|
| Quote: | | So install the zero duplicate and remove your robots.txt for a month or so, before you put it back on, the idea behind it is that it will be faster and better SEO wise (since HTTP 301 redirecting will transmit PR) to first let the bot acknowledge the redirection and then make sure they do not use the old links any more. |
for forums :
| dcz wrote: | Then you need to keep the old forum rewriterule up and running in your .htaccess (there is one or two depending on if you are using phpBB2 or phpBB3).
And yes, this means we'd allow another kind of duplicate (forumxx.html and forum-title-fxx.html), this in order to let the zero duplicate (looks like you are using phpBB2) do it's job. |
For categories, since they do not exist in the same way in phpBB3 and assuming that this is your question, you can either set personalized redirection for each category (to the corresponding category forum) or redirect them all to index :
| Code: | # CATEGORIES
RewriteRule ^[a-z0-9_-]+-c[0-9]+\.html$ /index.php [QSA,L,NC,r=301] |
To ba added with proper slashes and paths after :
| Code: | | # HERE IS A GOOD PLACE TO ADD OTHER PHPBB RELATED REWRITERULES |
in phpBB3's .htaccess.
Obviously, we talk about the phpBB related part of the robots.txt, HTTP redirecting for a month or so should be enough for Google to have acknowledged most redirecting, but you could wait a bit more.
If you do not do like this, Google will first see that you suddenly disallow almost all your forum content, so it won't "believe" it at first. Crawling will almost stop for few time, while Google will make sure the robots.txt really is telling him to drop all. Then, it will eventually start dropping the old links while it will slowly, at list a first, index the new one again.
Besides, old topics will have less chances to be indexed again with the new url.
HTTP redirecting will make him jump directly form the old url to the new ones, without changing the crawling rate. It's a lot more efficient. |
_________________
|
|
| Back to top |
|
|
triple
Joined: 18 Dec 2007
Posts: 11
Location: San Cristobal de las Casas - Mexico
|
| Posted: Tue Jan 22, 2008 10:25 am Post subject: Re: remuve url tool for duplicate indexed page on google |
|
|
| SeO wrote: |
what do you mean by the old one ? |
I meant the old rewrite rule (simple) on .htaccess.
| dcz wrote: | Then you need to keep the old forum rewriterule up and running in your .htaccess (there is one or two depending on if you are using phpBB2 or phpBB3).
And yes, this means we'd allow another kind of duplicate (forumxx.html and forum-title-fxx.html), this in order to let the zero duplicate (looks like you are using phpBB2) do it's job. |
it is phpbb2  i will ripristinate simple rewrite mod combinate with mixed one, 'cause zero duplicate it will works for me . but some questions:
*how can i integrate the two rewrites rules on .htaccess? can i simply copy and paste the two one or i need to integrate them? and the rewrite rule for category?
*the .htaccess to modify is only the one in the root?
*zero duplicate will show mixed mode urls or the simple ones?
| dcz wrote: | | So install the zero duplicate and remove your robots.txt for a month or so, before you put it back on, the idea behind it is that it will be faster and better SEO wise (since HTTP 301 redirecting will transmit PR) to first let the bot acknowledge the redirection and then make sure they do not use the old links any more. |
Well, zero duplicate is yet installed.
Talking about redirect old urls: simple mod was active only for 2 weeks, then i update to mixed, so there is no problem. My Page Rank is null and i've not yet started a serious link exchange campaign. I tell you this to explain that i have no risk to lose PR or links.
*there is no seo problem if all old url will be redirected only to one? ex. all to portal.php
| SeO wrote: | For categories, since they do not exist in the same way in phpBB3 and assuming that this is your question, you can either set personalized redirection for each category (to the corresponding category forum) or redirect them all to index :
Code:
# CATEGORIES
RewriteRule ^[a-z0-9_-]+-c[0-9]+\.html$ /index.php [QSA,L,NC,r=301]
To ba added with proper slashes and paths after :
Code:
# HERE IS A GOOD PLACE TO ADD OTHER PHPBB RELATED REWRITERULES
in phpBB3's .htaccess. |
Sure i will also integrate categories rewrite rule, completing the .htaccess. but i need to know, like said before, exactly how to create a good .htaccess that consider simple rewrite mod, mixed and this last suggestion
| SeO wrote: | Obviously, we talk about the phpBB related part of the robots.txt, HTTP redirecting for a month or so should be enough for Google to have acknowledged most redirecting, but you could wait a bit more.
If you do not do like this, Google will first see that you suddenly disallow almost all your forum content, so it won't "believe" it at first. Crawling will almost stop for few time, while Google will make sure the robots.txt really is telling him to drop all. Then, it will eventually start dropping the old links while it will slowly, at list a first, index the new one again.
Besides, old topics will have less chances to be indexed again with the new url.
HTTP redirecting will make him jump directly form the old url to the new ones, without changing the crawling rate. It's a lot more efficient. |
Due to my terrible english i've don't understand well what you say about robots.txt. Help me resuming if is right or not. do i really remove all phpbb folder from robots.txt? The crawl will stop for a while if SEbot find the robots.txt? or i only wait that google finish his recrawl and update job?
majority of my indexed pages still looks like viewtopic.php, viewforum.php etc...before mixed and before simple rewrite mod...
for this reason the original question was about use or not google removal url tool to delete these viewtopic.php and viewforum.php indexed pages.
*If i remove viewtopic.php and viewforum.php indexed pages with the tool, the the new .html url will be crawled, recognized ad listed or will disappear with .php urls?
really thanks |
|
|
| Back to top |
|
|
dcz
Administrateur - Site Admin
Joined: 28 Apr 2006
Posts: 13354
|
| Posted: Thu Jan 24, 2008 1:00 pm Post subject: Re: remuve url tool for duplicate indexed page on google |
|
|
Just do not remove the old url, let the zero duplicate do it. This includes to get rid of the disallows related to phpBB in your robtos.txt (the ones with phpBB/ dir).
The simple mod rewrite thing is another matter, regular urls will be taken into account properly directly with the zero duplicate. With phpBB3 the zero duplicate always redirect to the correct url, simple advanced or mixed according to your settings.
For the previously rewritten urls, just add :
| Code: | # phpbb2 CATEGORIES
RewriteRule ^[a-z0-9_-]+-c[0-9]+\.html$ /index.php [QSA,L,NC,r=301]
# simple topic
RewriteRule ^topic([0-9]+)(-([0-9]+))?\.html$ /viewtopic.php?t=$1&start=$3 [QSA,L,NC] |
To ba added with proper slashes and paths after :
| Code: | | # HERE IS A GOOD PLACE TO ADD OTHER PHPBB RELATED REWRITERULES |
in your phpBB's .htaccess.
|
_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________
Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche |
|
| Back to top |
|
|
triple
Joined: 18 Dec 2007
Posts: 11
Location: San Cristobal de las Casas - Mexico
|
| Posted: Thu Jan 24, 2008 1:19 pm Post subject: Re: remuve url tool for duplicate indexed page on google |
|
|
| dcz wrote: | With phpBB3 the zero duplicate always redirect to the correct url, simple advanced or mixed according to your settings.
|
also phpbb2? cause my board is a orion phpbb 2.0.22...
so...i've change .htacce and i've delete from robots.txt all that disallow like viewtopic.php, viewforum.php, /post, /cat/, forum/ etc....all this to let zero duplicate do his job! well done?
now let's wait google start to update something!
thanks a lot for patience!
domenico |
|
|
| Back to top |
|
|
|
|
| Navigation |
Similar Topics |
|
|
|
|
|
|
|
