Problem with mod_rewrite mod on PHPBB3

Discussions and support about the different URL Rewriting techniques for phpBB.

Moderator: Moderators

Problem with mod_rewrite mod on PHPBB3

Postby rado84 » Sun Feb 24, 2008 5:26 pm

Hello. I installed the mod_rewrite mod. Everything works great, the urls are mod rewrited, but when google visits my site, gives me an error in google webmaster tools. URLs restricted by robots.txt (8) - -http://www.domain.com/post1.html, -http://www.domain.com/post2.html, -http://www.domain.com/post3.html .....

Actually my post urls are not in these format, they are in the format of topic name. Is it problem when google don't index urls like post1.html (blocked by robots.txt)?
rado84
 
Posts: 2
Joined: Tue Jan 23, 2007 8:39 pm

Advertisement

Postby dcz » Mon Feb 25, 2008 4:52 pm

This comes from the fact you most likely did not implement the advised robots.txt at first.

It's no big deal, here is how to fix this :
1) drop the post line in your robots.txt in order to allow post urls again (I know they are dupes, but we'll take care of them in step 2) ;))
2) Install activate the zero duplicate and set post redirection to at least guest (none by default).

This way is faster because Google will continue to crawl post url and will get http 301 redirected to topic urls, where if you keep the disallow, it will not acknowledge the redirecting since it will not (or only very occasionally) crawl these anymore.

After couple month, post url will have disappeared from Google index, then put back the disallow on post urls, this will save some useless redirecting for bots.

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21393
Joined: Fri Apr 28, 2006 9:03 pm

Postby rado84 » Mon Feb 25, 2008 7:14 pm

Hi thanks for the reply. Is it problem to don't touch anything in the code. Because I am not interested in indexing these postXXX.html pages. I want Google to index posts like -http://www.domain.com/some-test-topic-t1.html. If Google give me an error for postXXX.html, then does Google will indexing my mod_rewrited posts (http://www.domain.com/some-test-topic-t1.html) without any problems? Thanks

PS: Offtopic: Does anyone know how to prevent users to delete their own posts? Thanks again
rado84
 
Posts: 2
Joined: Tue Jan 23, 2007 8:39 pm

Postby dcz » Sat Mar 01, 2008 3:44 pm

rado84 wrote:Because I am not interested in indexing these postXXX.html pages.

So are we don't worry.

I was advising you to let Google crawl postxx.html again only because the zero duplicate will HTTP 301 redirect these to the corresponding topic url, example : http://phpbb3.phpbb-seo.net/post59.html

So, given that postxx.html urls where already crawled by Google in your case, it's faster to redirect these, than to wait for Google to first really "believe" the robots.txt disallow and then drop them. This way, there would be no crawling rescheduling, only acknowledgement of the new location upon the next cache update.

In all cases, Google will discover and use your topic urls now that you installed the zero duplicate.

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21393
Joined: Fri Apr 28, 2006 9:03 pm


Return to phpBB mod Rewrite

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: Google Adsense [Bot] and 9 guests