question about robots text

Discussions about SEO Techniques implemented in your sites. Tests, Studies and results analysis.

Moderator: Moderators

question about robots text

Postby asiaexpat » Mon Nov 20, 2006 1:32 pm

my robots look like this

User-agent: *
Disallow: /phpBB2/viewtopic.php
Disallow: /phpBB2/viewforum.php
Disallow: /phpBB2/index.php?
Disallow: /phpBB2/posting.php
Disallow: /phpBB2/groupcp.php
Disallow: /phpBB2/profile.php
Disallow: /phpBB2/memberlist.php
Disallow: /phpBB2/search.php
Disallow: /phpBB2/login.php
Disallow: /phpBB2/faq.php
Disallow: /phpBB2/about
Disallow: /phpBB2/member
Disallow: /phpBB2/post
Disallow: /phpBB2/mark
Disallow: /phpBB2/updates-topic.html
Disallow: /phpBB2/stop-updates-topic.html
Disallow: /phpBB2/ptopic
Disallow: /phpBB2/ntopic
Disallow: /phpBB2/album_rate.php
Disallow: /phpBB2/privmsg.php


I have since added some pages like
http://www.asia-expatsforum.com/phpBB2/ ... -links.php

are these new pages being spidered ? because I got a message from one of those auto link checkers that vietnam-links.php was blocked by robots .text
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Advertisement

Postby dcz » Mon Nov 20, 2006 2:54 pm

Well, this robots.txt in not blocking -http://www.asia-expatsforum.com/phpBB2/vietnam-links.php, and the URL is listed in Google's cache.

It must be the checker you're using to mistake ;)

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21238
Joined: Fri Apr 28, 2006 9:03 pm

Postby asiaexpat » Mon Nov 20, 2006 2:57 pm

ok thanks

itsd not my checker-its another checking for alink that i put there
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am


Return to SEO Techniques

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 5 guests


 
cron