phpBB SEO
Boards
Directory  
SEO  
Downloads
  phpBB SEO : Search Engine Optimization, Directory, Forums  
Index
Forums
Annuaire
Référencement
Télécharger
 
  Search Rechercher
    Register
Username :  Password :  Log me on automatically each visit  
S'enregistrer  
 
   
robots.txt unreachable (google sitemap)

 
Post new topic   Reply to topic    phpBB SEO » SEO Forum  » Google Forums
::  
Author Message
AmirAbbas
phpBB SEO Team
phpBB SEO Team


Joined: 11 May 2006
Posts: 529
Location: IRAN

robots.txt unreachable (google sitemap)Posted: Sat Sep 16, 2006 9:12 am    Post subject: robots.txt unreachable (google sitemap)

hi

i have seen this error from six days ago
see this pictures








but im sure that robots.txt is reachable

-http://forum.persia-cms.com/robots.txt

the premission of this file is 644 and i can open it in browser
i check it in http header view
it shows 200 OK

its the help in google help center for this problem



whats the matter
Quote:
URL unreachable /robots.txt unreachable

Before we crawled the pages of your site, we tried to check your robots.txt file to ensure we didn't crawl any pages that you had roboted out. However, your robots.txt file was unreachable. To make sure we didn't crawl any pages listed in that file, we postponed our crawl. When this happens, we return to your site later and crawl it once we can reach your robots.txt file. Note that this is different from a 404 response when looking for a robots.txt file. If we receive a 404, we assume that a robots.txt file does not exist and we continue the crawl.

_________________
چهار گوش - طراحی وب - مجله طراحی وب
Back to top
Visit poster's website
dcz
Administrateur - Site Admin
Administrateur - Site Admin


Joined: 28 Apr 2006
Posts: 14854

robots.txt unreachable (google sitemap)Posted: Sat Sep 16, 2006 4:54 pm    Post subject: Re: robots.txt unreachable (google sitemap)

Well I confirm your robots.txt is very reachable and correct at http://forum.persia-cms.com/robots.txt which is the correct location for the sub domain.

So I guess it's either a server or network temporary issue, Google visited the robots.txt while all the web site was unreachable for example, or another example of Google experiencing troubles.

By the way, it's no big deal and I am pretty sure it will be able again to stats the obvious, this is the correct URL.

You could try resubmitting your web site and sitemap to make it faster, and delete the http://forum.persia-cms.com/index.php entry, just the first one is needed.

++

_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche
Back to top
Visit poster's website
AmirAbbas
phpBB SEO Team
phpBB SEO Team


Joined: 11 May 2006
Posts: 529
Location: IRAN

robots.txt unreachable (google sitemap)Posted: Sun Sep 17, 2006 7:17 am    Post subject: Re: robots.txt unreachable (google sitemap)

dcz wrote:
and delete the http://forum.persia-cms.com/index.php entry, just the first one is needed.


its not clear to me Rolling Eyes

i remved my site and my site map and resubmit it again
i receive that error message again

i have this message in google sitemap page

Quote:
We can't currently access your home page because of a timeout. [?]

_________________
چهار گوش - طراحی وب - مجله طراحی وب
Back to top
Visit poster's website
dcz
Administrateur - Site Admin
Administrateur - Site Admin


Joined: 28 Apr 2006
Posts: 14854

robots.txt unreachable (google sitemap)Posted: Sun Sep 17, 2006 11:16 am    Post subject: Re: robots.txt unreachable (google sitemap)

amir abbas wrote:
dcz wrote:
and delete the http://forum.persia-cms.com/index.php entry, just the first one is needed.


its not clear to me Rolling Eyes


Well, in the screenshot you sent, there are two urls for your web site, only the first one is required.
amir abbas wrote:

i remved my site and my site map and resubmit it again
i receive that error message again

i have this message in google sitemap page

Quote:
We can't currently access your home page because of a timeout. [?]


Actually, I just timed out while trying to load http://forum.persia-cms.com/index.php so it seems you are experiencing network (rooting) or server load issues. I could load http://www.persia-cms.com/ though.


++

_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche
Back to top
Visit poster's website
HB.
PR0
PR0


Joined: 26 Oct 2006
Posts: 62

robots.txt unreachable (google sitemap)Posted: Sun Nov 26, 2006 4:53 pm    Post subject: Re: robots.txt unreachable (google sitemap)

I'm having a somewhat similar problem. Before I submit my sitemap, I wanted to verify my site. www.example.com or www.example.com/phpbb Google said URL unreachable /robots.txt. The difference is I don't have a robots.txt in my root, but I have a robots.txt in my phpbb folder reading
Code:
User-agent: *
Disallow: /viewtopic.php
Disallow: /viewforum.php
Disallow: /index.php?
Disallow: /posting.php
Disallow: /groupcp.php
Disallow: /search.php
Disallow: /login.php
Disallow: /post
Disallow: /member
Disallow: /profile.php
Disallow: /memberlist.php
Disallow: /faq.php


Do I need to create a robots.txt at the root, and is there anything I should insert? I'm not understanding what the problem is. Sad

edit: I just noticed that the robots.txt I had in my forum had 755 permissions, does that have any effect? I just changed it to 644, and added a blank robots.txt to the root. Do any changes I make today, will I have to wait a day or so before I should check with Google again?
Back to top
dcz
Administrateur - Site Admin
Administrateur - Site Admin


Joined: 28 Apr 2006
Posts: 14854

robots.txt unreachable (google sitemap)Posted: Sun Nov 26, 2006 5:15 pm    Post subject: Re: robots.txt unreachable (google sitemap)

The only robots.txt that matters is the root one.

It should contain the path to the phpBB folder eg :

Code:
User-agent: *
Disallow: /phpBB/viewtopic.php

...


++

_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche
Back to top
Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    phpBB SEO » SEO Forum  » Google Forums
Page 1 of 1

Navigation Similar Topics

Jump to: