| |
| |
|
|
|
|
| |
|
| |
|
| :: |
| Author |
Message |
emielvenlo
PR0
Joined: 21 Jan 2007
Posts: 79
|
 Posted: Mon Feb 26, 2007 7:33 pm Post subject: Do i still need to make an robots.txt |
|
|
On my forum: [MOD]link removed[/MOD] I have made the following mods:
Advanced mod_rewrite
zero dupe
optimal titles
Do i still need to make an robots.txt for instance to not let google index posts(dupe's from topic) and members?? |
|
|
| Back to top |
|
 |
|
|
HB
phpBB SEO Team
Joined: 16 Oct 2006
Posts: 812
|
| Posted: Tue Feb 27, 2007 2:13 am Post subject: Re: Do i still need to make an robots.txt |
|
|
The zero dupe will redirect viewtopic.php / viewforum.php to the new URLs; you don't want to block search engine access. Below are robots.txt rules I recommend:
| Code: | User-agent: *
Disallow: /forums/groupcp.php
Disallow: /forums/memberlist.php
Disallow: /forums/member
Disallow: /forums/faq.php
Disallow: /forums/login.php
Disallow: /forums/modcp.php
Disallow: /forums/posting.php
Disallow: /forums/search.php
Disallow: /forums/profile.php
Disallow: /forums/privmsg.php
Disallow: /forums/viewonline.php |
dcz's list is slightly different:
http://www.phpbb-seo.com/robots.txt
They're essentially the same except for index.php, viewtopic.php, and viewforum.php. I omitted them to allow the search engines a chance to be redirected. |
_________________
Dan Kehn |
|
| Back to top |
|
|
dcz
Administrateur - Site Admin
Joined: 28 Apr 2006
Posts: 15127
|
| Posted: Tue Feb 27, 2007 1:45 pm Post subject: Re: Do i still need to make an robots.txt |
|
|
Yes, you should not disallow :
| Code: | Disallow: /boards/viewtopic.php
Disallow: /boards/viewforum.php
Disallow: /boards/index.php |
when migrating urls, but this can help out once it's done. As these will prevent all the advanced pagination sorting options to be cached as well.
So while migrating urls, I'd say, get rid of these for couple month, and put them back after, when bots already stopped using them.
++ |
_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________
Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche |
|
| Back to top |
|
|
emielvenlo
PR0
Joined: 21 Jan 2007
Posts: 79
|
| Posted: Tue Feb 27, 2007 2:13 pm Post subject: Re: Do i still need to make an robots.txt |
|
|
| dcz wrote: | Yes, you should not disallow :
| Code: | Disallow: /boards/viewtopic.php
Disallow: /boards/viewforum.php
Disallow: /boards/index.php |
when migrating urls, but this can help out once it's done. As these will prevent all the advanced pagination sorting options to be cached as well.
So while migrating urls, I'd say, get rid of these for couple month, and put them back after, when bots already stopped using them.
++ |
In this case its a new forum (not a month old yet), so i dont think google has much indexed yet. Is it better to keep the rules if its a new forum because its not really migrating?
(this is not brandpreventieforum, but the link was removed) |
|
|
| Back to top |
|
|
dcz
Administrateur - Site Admin
Joined: 28 Apr 2006
Posts: 15127
|
|
| Back to top |
|
|
|
|
| Navigation |
Similar Topics |
|
|
|
|
|
|
|
| |
|
|
|
|
| |
|
|
|
|
| |
