My robots.txt

phpBB3 SEO Mixed mod Rewrite support forum. This mods performs URL rewriting for phpBB3, injecting forums in their URLs and keeping a static rewriting for topics.

Moderator: Moderators

My robots.txt

Postby Cobra » Fri Feb 27, 2009 6:51 pm

Hello, here's my robots.txt. Any suggestion?

Code: Select all
User-agent: *
Disallow: /adm/
Disallow: /cache/
Disallow: /download/
Disallow: /files/
Disallow: /images/
Disallow: /includes/
Disallow: /language/
Disallow: /store/
Disallow: /styles/
Disallow: /chat.php
Disallow: /common.php
Disallow: /config.php
Disallow: /contact.php
Disallow: /cron.php
Disallow: /faq.php
Disallow: /functions.php
Disallow: /mcp.php
Disallow: /memberlist.php
Disallow: /posting.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /shout.php
Disallow: /ucp.php
Disallow: /viewonline.php
Disallow: /viewtopic.php
Disallow: /viewforum.php
Disallow: /index.php?
Disallow: /member
Cobra
PR0
PR0
 
Posts: 50
Joined: Fri Aug 22, 2008 5:57 pm

Advertisement

Re: My robots.txt

Postby SeO » Tue Mar 03, 2009 11:26 am

If none of the concerned url are cached right now, you should as well add :
Code: Select all
Disallow: /post


Generally speaking, it's not very useful to disallow directories such as files/ or download/, because they are already locked by .htaccess. So all you do is actually tell bots that such a directory exists (they could not even know it if not).
SeO
Admin
Admin
 
Posts: 6333
Joined: Wed Mar 15, 2006 9:41 pm


Return to Mixed SEO URL

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 6 guests