| |
| |
|
|
|
|
| |
|
| |
|
| :: |
| Author |
Message |
sceltic
PR0
Joined: 04 May 2006
Posts: 55
|
 Posted: Fri May 05, 2006 6:04 pm Post subject: Robots Txt for PHPBB mod rewrite |
|
|
What can be improved or needs to be over the old able2know robots txt suggested standard.
Here is that robots txt
| Code: | OPEN ]------------------------------------------
robots.txt
Disallow: /phpbb/post-*.html$
Disallow: /phpbb/updates-topic.html*$
Disallow: /phpbb/stop-updates-topic.html*$
Disallow: /phpbb/ptopic*.html$
Disallow: /phpbb/ntopic*.html$
|
I believe some of the above were done prior to wild cards the * part being realised as no use.
For instance now that would be more correct
as
Disallow: /phpbb/post- |
|
|
| Back to top |
|
 |
|
|
dcz
Administrateur - Site Admin
Joined: 28 Apr 2006
Posts: 15242
|
| Posted: Fri May 05, 2006 7:30 pm Post subject: Re: Robots Txt for PHPBB mod rewrite |
|
|
I'd say you go for :
| Code: |
Disallow: /phpbb/post-
Disallow: /phpbb/updates-topic.html
Disallow: /phpbb/stop-updates-topic.html
Disallow: /phpbb/ptopic
Disallow: /phpbb/ntopic
Disallow: /phpbb/index.php?
Disallow: /phpbb/posting.php
Disallow: /phpbb/modcp.php
Disallow: /phpbb/login.php
Disallow: /phpbb/privmsg.php
Disallow: /phpbb/viewonline.php
|
And you could even add those too :
| Code: |
Disallow: /phpbb/faq.php
Disallow: /phpbb/profile.php
Disallow: /phpbb/groupcp.php
|
But it's as you want, it's just in case you don't want spammer to love your memberlist too much, groucp and faq are mostly interesting to members, not really a search target.
I'll soon start posting article on how to analyse a site indexing and such, to find out better what should be done.
++ |
_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________
Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche |
|
| Back to top |
|
|
sceltic
PR0
Joined: 04 May 2006
Posts: 55
|
| Posted: Sat May 06, 2006 10:35 am Post subject: Re: Robots Txt for PHPBB mod rewrite |
|
|
| Good information. |
|
|
| Back to top |
|
|
|
| Navigation |
Similar Topics |
|
|
|
|
|
|
|
| |
|
|
|
|
| |
|
|
|
|
| |
