Using robots.txt to block parts of the forum

Discussions about the phpBB3 Forum. How to get the best from this powerful script.
Note that there is no official support for phpBB3 here, a place to share more than to ask for.

Moderator: Moderators

Using robots.txt to block parts of the forum

Postby MM84 » Wed Sep 05, 2007 9:07 am

Hi people, I hope you can help me!

I've recently upgraded to phpbb3 and so far only installed one mod- google sitemap from phpbbhacks.com.

It's working brilliantly, except that google keeps crawling the hidden moderators forum and as a result, you can read private threads in the google index.

How can I stop google from indexing it?

The forum is:

viewforum.php?f=9

Please help! It's a matter of urgency!!

Thanks,

MM
MM84
 
Posts: 2
Joined: Wed Sep 05, 2007 9:02 am

Advertisement

Postby dcz » Wed Sep 05, 2007 11:23 am

Are the private forums really cached in Google's index, or do you only see the login form in there ?

phpBB3 can allow Bots to crawl private forums, but you need to set the authorisation to do so. It's not the default behaviour, so the first thing to make sure is : are the private forums and topic inside really cached.

And, make sure as well that the sitemap you are using does not provide bots with unnecessary links to pages they cannot load.

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21391
Joined: Fri Apr 28, 2006 9:03 pm

Postby mhmdkhamis » Wed Sep 05, 2007 5:50 pm

i noticed google dont into in any forums in my forum but my topics get cash

what do you think dcz
mhmdkhamis
PR4
PR4
 
Posts: 437
Joined: Sun Nov 26, 2006 11:15 pm
Location: egypt

Postby dcz » Thu Sep 06, 2007 10:45 am

Do you mean that you do not see the Google bot but it's still caching public content ?

Well, you must have missed the bot ;)

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21391
Joined: Fri Apr 28, 2006 9:03 pm

Postby emielvenlo » Thu Sep 06, 2007 10:45 am

I have had this problem to after upgrading. This is because the permissions of the group "bots" have to be set. You have to go to group permissions of the bots and alter them so they can only see the public forums.
User avatar
emielvenlo
PR0
PR0
 
Posts: 82
Joined: Sun Jan 21, 2007 10:01 am

Postby mhmdkhamis » Thu Sep 06, 2007 10:56 am

dcz wrote:Do you mean that you do not see the Google bot but it's still caching public content ?

Well, you must have missed the bot ;)

++


yes dcz i see google bot in my forum index page only but my topics get cash good :roll:
mhmdkhamis
PR4
PR4
 
Posts: 437
Joined: Sun Nov 26, 2006 11:15 pm
Location: egypt

Postby dcz » Thu Sep 06, 2007 11:03 am

No forum cached ?
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21391
Joined: Fri Apr 28, 2006 9:03 pm

Postby mhmdkhamis » Fri Sep 07, 2007 4:42 pm

dcz wrote:No forum cached ?


and my forums get cash good :) but no google bot can see into my forums :lol:
mhmdkhamis
PR4
PR4
 
Posts: 437
Joined: Sun Nov 26, 2006 11:15 pm
Location: egypt

Postby dcz » Fri Sep 07, 2007 5:23 pm

Then I would not really bother.
The chances are actually few to see a bot on a given page, you could need to wait for like two or three days to see the Google bot on a given forum.
And it would not be bad to have Google checking the forums every couple days, that's actually a nice rate.

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21391
Joined: Fri Apr 28, 2006 9:03 pm

Postby MM84 » Sat Sep 08, 2007 1:24 pm

emielvenlo wrote:I have had this problem to after upgrading. This is because the permissions of the group "bots" have to be set. You have to go to group permissions of the bots and alter them so they can only see the public forums.



Thanks!
MM84
 
Posts: 2
Joined: Wed Sep 05, 2007 9:02 am


Return to phpBB Forum

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 7 guests