Question on Robots.txt

Discussions and support about the different URL Rewriting techniques for phpBB.

Moderator: Moderators

Question on Robots.txt

Postby 5942marine » Thu Jun 11, 2009 4:37 am

Hey guys, so I have pretty much every mod that you guys have installed. But!!! one thing I noticed is that I don't have a robots.txt file.

Should I be using one?
5942marine
PR0
PR0
 
Posts: 69
Joined: Thu Mar 12, 2009 11:04 pm

Advertisement

Re: Question on Robots.txt

Postby remoteinsider » Thu Jun 11, 2009 9:27 am

The robots.txt file should only be used if there are pages on your website/forum which you do not want the search engines to index, for example you may have a locked forum for members who make a donation in which case you wouldn't want the search engines to index the content here.
remoteinsider
PR0
PR0
 
Posts: 72
Joined: Tue Apr 21, 2009 12:09 pm

Re: Question on Robots.txt

Postby dcz » Fri Jun 12, 2009 7:51 am

Read the robots.txt part of the install file, the only thing to really car about with it is not to disallow anything that should first be redirected.

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21379
Joined: Fri Apr 28, 2006 9:03 pm

Re: Question on Robots.txt

Postby remoteinsider » Fri Jun 12, 2009 10:15 am

What dcz said also is correct.
remoteinsider
PR0
PR0
 
Posts: 72
Joined: Tue Apr 21, 2009 12:09 pm


Return to phpBB mod Rewrite

 


  • Related topics
    Replies
    Views
    Last post
  • robots
    by quizzical » Mon Feb 02, 2009 5:09 pm
    4 Replies
    1562 Views
    Last post by SeO View the latest post
    Sat Mar 07, 2009 11:32 am

Who is online

Users browsing this forum: No registered users and 4 guests