robots

Discussions and support about the different URL Rewriting techniques for phpBB.

Moderator: Moderators

robots

Postby quizzical » Mon Feb 02, 2009 5:09 pm

I have installe mod rewrite and afterwards zero duplicate. i just don't want the postxx to show up on google, only the topic titles. So in the Zero duplicate options I have it this way:
Activate Zeroduplicate yes
Strict Mode yes
Posts redirections All

My questions is what I have to put on my robots.txt?
Is this right? Because I don't see topics show in google and I did install them both 1 month ago

User-agent: *
Disallow: /forum/viewtopic.php
Disallow: /forum/viewforum.php
Disallow: /forum/index.php?
Disallow: /forum/posting.php
Disallow: /forum/search.php?
Disallow: /forum/ucp.php
Disallow: /forum/mcp.php
Disallow: /forum/post
Disallow: /forum/member
Disallow: /forum/memberlist.php
Disallow: /forum/faq.php

Do I have to disallow
Disallow: /forum/post
or
Disallow: post

Why if I use a program like GoogleSiteCrwaler a lot of postXX and memberXX show?

This is the forum, give it a look if it seems to you all right
http://quizzical.altervista.org/forum.html
Thanks, same old questions I know, but I'm not very sure on this kind of stuff.
quizzical
 
Posts: 7
Joined: Sun Dec 14, 2008 4:05 pm

Advertisement

Postby quizzical » Tue Feb 03, 2009 9:01 pm

nobody? :(
quizzical
 
Posts: 7
Joined: Sun Dec 14, 2008 4:05 pm

Postby IPB_Refugee » Wed Feb 04, 2009 8:19 am

Hi quizzical,

your robots.txt doesn't really reflect the URL structure of your board.

There is no

/forum/memberlist.php
/forum/viewtopic.php
/forum/viewforum.php

and so on - at least not for visitors (I visited your board with Google's user-agent.)

I would try to disallow

/memberlist.php
/viewtopic.php
/viewforum.php
/member
/group

and so on.

Also try to install No Dupe MOD and the Sitemap MOD from phpbb-seo.com (There should be a way to exclude certain URLs from the generated sitemap when they are listed at all.)

But Google knows your topics' URLs already. They are just not listed at the beginning. I tried e.g. the search results 351-360 (random choice):

http://www.google.de/search?hl=de&safe= ... t=350&sa=N

Regards
Wolfgang
User avatar
IPB_Refugee
PR0
PR0
 
Posts: 82
Joined: Thu Jul 24, 2008 2:18 pm

Re: robots

Postby quizzical » Tue Mar 03, 2009 4:02 pm

thanks for oyur answer.

/forum/memberlist.php
/forum/viewtopic.php
/forum/viewforum.php

these files do exist.

I don't seem to get any google page rank as well. I wonder why?
quizzical
 
Posts: 7
Joined: Sun Dec 14, 2008 4:05 pm

Re: robots

Postby SeO » Sat Mar 07, 2009 11:32 am

quizzical wrote:/forum/memberlist.php
/forum/viewtopic.php
/forum/viewforum.php


They actually do not exist, there is nothing much to say about it unless they actually did before a conversion for example.

PR takes time to update, so be patient ;)
SeO
Admin
Admin
 
Posts: 6334
Joined: Wed Mar 15, 2006 9:41 pm


Return to phpBB mod Rewrite

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: Baidu [Spider] and 5 guests