Hi All,
I am new to phpbb board and with great difficulty I am trying to learn it.I understand that SEO is very important.Could anyone of you tell me how do I create robot.txt and where should I store it in my forum.Do these robots really help in SEO?Help appreciated.

the robots.txt is just a plain text file to be uploaded in your domain's root (not in the sub directories).

It is used to tell bots what url should not be used for indexing. The disallow are only instructions given to bots, but it's up to them to follow them. Google msn and yahoo do follow them with more or less ease, but in the end they do.

So it's useful to make sure only the urls you target will be crawled by the major Search engines.

Basic structure is :

Code: Select all

User-agent: *
Disallow: /script.php
Disallow: /dir/script.php
Disallow: /dir2/post


The wild-card (*) means this will apply to all bots.
Then, as you see, you need to specify the proper path to the files.

As well, Disallow: /dir2/post show will disallow everything starting with "post" in the dir2/ directory, eg postxx.html etc ...

robots.txt will help out SEO wise to fight against duplicates and to prevent some url to be indexed.

You should take a look at the phpBB SEO category to find mod that will positively SEO your board, robots.txt is just a small part of the whole process.

++

Thank You very much.Help Appreciated

In robots.txt file User-name and Disallow are case sensitive. So careful to use.