| |
| |
|
|
|
|
| |
|
| |
|
| :: |
| Author |
Message |
AmirAbbas phpBB SEO Team


Joined: 11 May 2006 Posts: 529 Location: IRAN
|
Posted: Sat Sep 16, 2006 9:12 am Post subject: robots.txt unreachable (google sitemap) |
|
|
hi
i have seen this error from six days ago
see this pictures
but im sure that robots.txt is reachable
-http://forum.persia-cms.com/robots.txt
the premission of this file is 644 and i can open it in browser
i check it in http header view
it shows 200 OK
its the help in google help center for this problem
whats the matter
| Quote: | URL unreachable /robots.txt unreachable
Before we crawled the pages of your site, we tried to check your robots.txt file to ensure we didn't crawl any pages that you had roboted out. However, your robots.txt file was unreachable. To make sure we didn't crawl any pages listed in that file, we postponed our crawl. When this happens, we return to your site later and crawl it once we can reach your robots.txt file. Note that this is different from a 404 response when looking for a robots.txt file. If we receive a 404, we assume that a robots.txt file does not exist and we continue the crawl.
|
|
_________________ چهار گوش - طراحی وب - مجله طراحی وب |
|
| Back to top |
|
 |
|
 |
dcz Administrateur - Site Admin

Joined: 28 Apr 2006 Posts: 14854
|
|
| Back to top |
|
 |
AmirAbbas phpBB SEO Team


Joined: 11 May 2006 Posts: 529 Location: IRAN
|
Posted: Sun Sep 17, 2006 7:17 am Post subject: Re: robots.txt unreachable (google sitemap) |
|
|
its not clear to me
i remved my site and my site map and resubmit it again
i receive that error message again
i have this message in google sitemap page
| Quote: | | We can't currently access your home page because of a timeout. [?] |
|
_________________ چهار گوش - طراحی وب - مجله طراحی وب |
|
| Back to top |
|
 |
dcz Administrateur - Site Admin

Joined: 28 Apr 2006 Posts: 14854
|
|
| Back to top |
|
 |
HB. PR0

Joined: 26 Oct 2006 Posts: 62
|
Posted: Sun Nov 26, 2006 4:53 pm Post subject: Re: robots.txt unreachable (google sitemap) |
|
|
I'm having a somewhat similar problem. Before I submit my sitemap, I wanted to verify my site. www.example.com or www.example.com/phpbb Google said URL unreachable /robots.txt. The difference is I don't have a robots.txt in my root, but I have a robots.txt in my phpbb folder reading
| Code: | User-agent: *
Disallow: /viewtopic.php
Disallow: /viewforum.php
Disallow: /index.php?
Disallow: /posting.php
Disallow: /groupcp.php
Disallow: /search.php
Disallow: /login.php
Disallow: /post
Disallow: /member
Disallow: /profile.php
Disallow: /memberlist.php
Disallow: /faq.php |
Do I need to create a robots.txt at the root, and is there anything I should insert? I'm not understanding what the problem is.
edit: I just noticed that the robots.txt I had in my forum had 755 permissions, does that have any effect? I just changed it to 644, and added a blank robots.txt to the root. Do any changes I make today, will I have to wait a day or so before I should check with Google again? |
|
|
| Back to top |
|
 |
dcz Administrateur - Site Admin

Joined: 28 Apr 2006 Posts: 14854
|
|
| Back to top |
|
 |
|
|
| Navigation |
Similar Topics |
|
|
|
|
|
|
|
| |
|
|
|
|
| |
|
|
|
|
| |