Hi i wan to know wich are the first things i have to edit in my robots.txt file in order to make my phpbb forum firendly?

I have no clue for that...

I will really appreciate some suggestions.

Thanks

The robots.txt content depends on you site and your URLs.

As a starting principle you should disallow any page being a duplicate of what you consider as the best URL to offer for each page, obviously if your script outputs more than one kind.

As well, files only meant for example to post messages should be disallowed in order to help bots focusing on your real content, the result of those posts if we follow this example.

++

Here you have the complete robots.txt file at my site

Code: Select all

User-agent: Mediapartners-Google*
Disallow:

User-agent: *
Disallow: /admin/
Disallow: /images/
Disallow: /includes/
Disallow: /plugins/
Disallow: /skins/
Disallow: /foro/admin/
Disallow: /foro/db/
Disallow: /foro/files/
Disallow: /foro/images/
Disallow: /foro/includes/
Disallow: /foro/languages/
Disallow: /foro/templates/
Disallow: /foro/posting.php
Disallow: /foro/login.php
Disallow: /foro/memberlist.php
Disallow: /foro/statistics.php
Disallow: /foro/profile.php
Disallow: /foro/viewtopic.php
Disallow: /foro/viewforum.php
Disallow: /foro/index.php?
Disallow: /foro/groupcp.php
Disallow: /foro/search.php
Disallow: /foro/login.php
Disallow: /foro/member
Disallow: /foro/profile.php
Disallow: /foro/faq.php

Any suggestions in order to adsense runs better or other SEO thechniques?

If you had this robots.txt before you found out about this site, then it's very normal you where not spidered, you where disallowing all topics and posts

So keep this robots.txt and add the line suggested in the release topic of the mod rewrite you are using

++

No friend. This is my new robots file after i install mod rewrite mixed.

Because of that I disallow all the php extensions in order to not have duplicates.

In this case is ok? or i have to change something?

Your robots txt is fine. also you may add any other files you do not want search engines to crawl - like any MODs you might have. shop.php, album.php, rss.php ect.

I already told you to get rid of this part :

Code: Select all

User-agent: Mediapartners-Google*
Disallow:

and to look at the release topic of the mod rewrite you're using, you miss some of the robots.txt suggested there, such as disallow /post /member ...

++