| |
| |
|
|
|
|
| |
|
| |
|
| :: |
| Author |
Message |
apeee
PR0
Joined: 06 Nov 2007
Posts: 60
|
 Posted: Tue Jan 01, 2008 9:39 am Post subject: Why to Disallow: /member ? |
|
|
Hi!
I was just experimenting with robots.txt and wanted to know whether I should disallow /member and /messages in phpbb3 seo premod. I want search engines to view users profiles and their messages. Is there anything wrong with seo if I do this? I don't want to do anything bad with search engines and seo. What will be the best from seo point of view? |
|
|
| Back to top |
|
 |
|
|
SeO
Administrateur - Site Admin
Joined: 15 Mar 2006
Posts: 3103
|
| Posted: Tue Jan 01, 2008 3:02 pm Post subject: Re: Why to Disallow: /member ? |
|
|
It's mostly as you wish, but profile pages do not usually have enough good content to be worth showing up on serps.
This of course depends on how important profiles are on your board, but the first target is the topic for SEO. |
_________________
|
|
| Back to top |
|
|
apeee
PR0
Joined: 06 Nov 2007
Posts: 60
|
| Posted: Wed Jan 02, 2008 8:20 pm Post subject: Re: Why to Disallow: /member ? |
|
|
| Code: | User-agent: *
Disallow: /adm/
Disallow: /cgi-bin/
Disallow: /cache/
Disallow: /docs/
Disallow: /download/
Disallow: /files/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /phpbb_seo/
Disallow: /store/
Disallow: /styles/
Disallow: /common.php
Disallow: /common.php?
Disallow: /config.php
Disallow: /config.php?
Disallow: /cron.php
Disallow: /cron.php?
Disallow: /download.php
Disallow: /download.php?
Disallow: /faq.php
Disallow: /faq.php?
Disallow: /fs.php
Disallow: /fs.php?
Disallow: /index.php
Disallow: /index.php?
Disallow: /index.html?
Disallow: /mcp.php
Disallow: /mcp.php?
Disallow: /memberlist.php
Disallow: /memberlist.php?
Disallow: /posting.php
Disallow: /posting.php?
Disallow: /ratetopic.php
Disallow: /ratetopic.php?
Disallow: /report.php
Disallow: /report.php?
Disallow: /search.php
Disallow: /search.php?
Disallow: /style.php
Disallow: /style.php?
Disallow: /ucp.php
Disallow: /ucp.php?
Disallow: /viewforum.php
Disallow: /viewforum.php?
Disallow: /viewonline.php
Disallow: /viewonline.php?
Disallow: /viewtopic.php
Disallow: /viewtopic.php?
Disallow: /messages
Disallow: /post
Disallow: /? |
Well! It's kind of wired but I really have disallowed each and everything from my forum. I've allowed bots and guests to crawl /members/seo (example) area. I've redirected index.php to index.html in .htaccess. Please let me know if I'm doing something wrong. I only want search engine to find .html files (topics and members) nothing else. Do I need to do any changes in above robots.txt file? |
|
|
| Back to top |
|
|
SeO
Administrateur - Site Admin
Joined: 15 Mar 2006
Posts: 3103
|
| Posted: Thu Jan 03, 2008 10:40 am Post subject: Re: Why to Disallow: /member ? |
|
|
To do what you want, the suggested robots.txt is enough.
The robots.txt is just a suggestion, you are not actually protecting your files doing so, and actually, disallowing file that are never accessed is a way to have bots trying them (otherwise they would not know about them).
Only Disallow: /images/ could have some interest if you want to make sure bots won't list images directly from this dir. |
_________________
|
|
| Back to top |
|
|
|
| Navigation |
Similar Topics |
|
|
|
|
|
|
|
| |
|
|
|
|
| |
|
|
|
|
| |
