| |
| |
|
|
|
|
| |
|
| |
|
| :: |
| Author |
Message |
asiaexpat
PR1
Joined: 04 Aug 2006
Posts: 121
|
 Posted: Mon Nov 20, 2006 1:32 pm Post subject: question about robots text |
|
|
my robots look like this
User-agent: *
Disallow: /phpBB2/viewtopic.php
Disallow: /phpBB2/viewforum.php
Disallow: /phpBB2/index.php?
Disallow: /phpBB2/posting.php
Disallow: /phpBB2/groupcp.php
Disallow: /phpBB2/profile.php
Disallow: /phpBB2/memberlist.php
Disallow: /phpBB2/search.php
Disallow: /phpBB2/login.php
Disallow: /phpBB2/faq.php
Disallow: /phpBB2/about
Disallow: /phpBB2/member
Disallow: /phpBB2/post
Disallow: /phpBB2/mark
Disallow: /phpBB2/updates-topic.html
Disallow: /phpBB2/stop-updates-topic.html
Disallow: /phpBB2/ptopic
Disallow: /phpBB2/ntopic
Disallow: /phpBB2/album_rate.php
Disallow: /phpBB2/privmsg.php
I have since added some pages like
http://www.asia-expatsforum.com/phpBB2/vietnam-links.php
are these new pages being spidered ? because I got a message from one of those auto link checkers that vietnam-links.php was blocked by robots .text |
|
|
| Back to top |
|
 |
|
|
dcz
Administrateur - Site Admin
Joined: 28 Apr 2006
Posts: 14779
|
|
| Back to top |
|
|
asiaexpat
PR1
Joined: 04 Aug 2006
Posts: 121
|
| Posted: Mon Nov 20, 2006 2:57 pm Post subject: Re: question about robots text |
|
|
ok thanks
itsd not my checker-its another checking for alink that i put there |
|
|
| Back to top |
|
|
|
| Navigation |
Similar Topics |
|
|
|
|
|
|
|
| |
|
|
|
|
| |
|
|
|
|
| |
