Problem with mod_rewrite mod on PHPBB3

rado84 · Joined: 23 Jan 2007 Posts: 2

Hello. I installed the mod_rewrite mod. Everything works great, the urls are mod rewrited, but when google visits my site, gives me an error in google webmaster tools. URLs restricted by robots.txt ( Cool

- -http://www.domain.com/post1.html, -http://www.domain.com/post2.html, -http://www.domain.com/post3.html .....

Actually my post urls are not in these format, they are in the format of topic name. Is it problem when google don't index urls like post1.html (blocked by robots.txt)?

dcz

This comes from the fact you most likely did not implement the advised robots.txt at first.

It's no big deal, here is how to fix this :
1) drop the post line in your robots.txt in order to allow post urls again (I know they are dupes, but we'll take care of them in step 2) Wink

)
2) Install activate the zero duplicate and set post redirection to at least guest (none by default).

This way is faster because Google will continue to crawl post url and will get http 301 redirected to topic urls, where if you keep the disallow, it will not acknowledge the redirecting since it will not (or only very occasionally) crawl these anymore.

After couple month, post url will have disappeared from Google index, then put back the disallow on post urls, this will save some useless redirecting for bots.

++

rado84 · Joined: 23 Jan 2007 Posts: 2

Hi thanks for the reply. Is it problem to don't touch anything in the code. Because I am not interested in indexing these postXXX.html pages. I want Google to index posts like -http://www.domain.com/some-test-topic-t1.html. If Google give me an error for postXXX.html, then does Google will indexing my mod_rewrited posts (http://www.domain.com/some-test-topic-t1.html) without any problems? Thanks

PS: Offtopic: Does anyone know how to prevent users to delete their own posts? Thanks again

dcz