News of roBots

I don't have much idea about robot.txt

someone gave me this,

Code: Select all

# User-agent: *
# Crawl-delay: 10

User-agent: *
Host: fjunction.org
Disallow: forum/engine/go.php
Disallow: forum/user/
Disallow: forum/newposts/
Disallow: forum/statistics.html
Disallow: forum/*subaction=userinfo
Disallow: forum/*subaction=newposts
Disallow: forum/*do=lastcomments
Disallow: forum/*do=feedback
Disallow: forum/*do=register
Disallow: forum/*do=lostpassword
Disallow: forum/*do=addnews
Disallow: forum/*do=stats
Disallow: forum/*do=pm
Sitemap: http://fjunction.org/sitemap.xml
Sitemap: http://fjunction.org/rss.xml

Please help me friends, I have no idea :roll:

My phpBB is installed in /forum/ folder, when the site map is generated from /forum/ url.

Hi

I have a forum www.forumquiz.com . Problem i am facing is that pages are not index by google. I think there is a problem with my robots.txt file. Following is the robots.txt contents.

User-agent: *
Disallow: /phpbb/viewtopic.php
Disallow: /phpbb/viewforum.php
Disallow: /phpbb/index.php?
Disallow: /phpbb/posting.php
Disallow: /phpbb/search.php?
Disallow: /phpbb/ucp.php
Disallow: /phpbb/mcp.php
Disallow: /phpbb/post
Disallow: /phpbb/member
Disallow: /phpbb/memberlist.php
Disallow: /phpbb/faq.php

Please tell me where is the mistake and what should i do.

Thanks

Pradip

I have installed advanced url, but google indexing post.html instead of advanced url format links how to prevent it.

Hi, I have my root folder as public_html where all my phpbb files are lying, and I would like to know how do I make a good robot.txt and what to disallow etc.

I have this on my robot.txt

Code: Select all

User-agent: *
Allow: /

I installed my forum on -www.site.com/forum/ . So it gets to root's robots (-www.site.com/robots.txt)
How I can change?