my mx Google sitemaps htaccess is ok?

asiaexpat · by **asiaexpat** » Wed Aug 09, 2006 1:36 pm

Ok how is this??
we have tried already to use a different redirect method but did not work

Code: Select all: Options +FollowSymlinks RewriteEngine on RewriteRule ^$ http://www.asia-expatsforum.com/phpBB2/portal.php [QSA,L,R=301] RewriteCond %{HTTP_HOST} ^asia-expatsforum\.com$ [NC] RewriteRule ^(.*) http://www.asia-expatsforum.com/$1 [QSA,L,R=301] RewriteBase / # phpBB # FORUM PROTECTION RULE RewriteRule ^phpBB2/.+/([^/]+\.html)$ /phpBB2/index.php [R=301,L] # CATEGORIES RewriteRule ^phpBB2/.+-vc([0-9]+)\.html$ /phpBB2/index.php?c=$1 [QSA,L] # PAGINATED FORUM RewriteRule ^phpBB2/.+-vf([0-9]+)-([0-9]+)\.html$ /phpBB2/viewforum.php?f=$1&start=$2 [QSA,L] # FORUM RewriteRule ^phpBB2/.+-vf([0-9]+)\.html$ /phpBB2/viewforum.php?f=$1 [QSA,L] # PAGINATED TOPIC RewriteRule ^phpBB2/.+-vt([0-9]+)-([0-9]+)\.html$ /phpBB2/viewtopic.php?t=$1&start=$2 [QSA,L] # TOPIC RewriteRule ^phpBB2/.+-vt([0-9]+)\.html$ /phpBB2/viewtopic.php?t=$1 [QSA,L] # POST RewriteRule ^phpBB2/post([0-9]+)\.html$ /phpBB2/viewtopic.php?p=$1 [QSA,L] #PROFILES RewriteRule ^phpBB2/member([0-9]+)\.html$ /phpBB2/profile.php?mode=viewprofile&u=$1 [QSA,L] RewriteRule ^phpBB2/sitemaps.xml$ /phpBB2/sitemap.php [L] RewriteRule ^phpBB2/forum-sitemap-([0-9]+).xml$ /phpBB2/sitemap.php?fid=$1 [L] RewriteRule ^phpBB2/sitemap-forum.xml$ /phpBB2/sitemap.php?forum [L] # END PHPBB PAGES

Code: Select all: <Limit GET POST PUT> Order Allow,Deny Deny from All </Limit>

Is that correct??

dcz · by **dcz** » Wed Aug 09, 2006 3:02 pm

There is just no way Google spidered all those this past hour, these are your old stats

From : http://www.smart-it-consulting.com/inte ... /index.htm

Schema validating with XSV 2.10-1 of 2005/04/22 13:10:49

* Target: http://www.asia-expatsforum.com/phpBB2/ ... map-10.xml
(Real name: http://www.asia-expatsforum.com/phpBB2/ ... map-10.xml
Server: Apache/1.3.31 (Unix) PHP/4.3.11 mod_ssl/2.8.18 OpenSSL/0.9.6b FrontPage/5.0.2.2635 mod_throttle/3.1.2)
* docElt: {http://www.google.com/schemas/sitemap/0.84}urlset
* Validation was strict, starting with type [Anonymous]
* schemaLocs: http://www.google.com/schemas/sitemap/0.84 -> http://www.google.com/schemas/sitemap/0.84/sitemap.xsd
* The schema(s) used for schema-validation had
no errors
* No schema-validity problems were found in the target

Schema resources involved

Attempt to load a schema document from
http://www.google.com/schemas/sitemap/0.84/sitemap.xsd
(source: schemaLoc) for
http://www.google.com/schemas/sitemap/0.84,
succeeded

And :

Schema validating with XSV 2.10-1 of 2005/04/22 13:10:49

* Target: http://www.asia-expatsforum.com/phpBB2/sitemaps.xml
(Real name: http://www.asia-expatsforum.com/phpBB2/sitemaps.xml
Server: Apache/1.3.31 (Unix) PHP/4.3.11 mod_ssl/2.8.18 OpenSSL/0.9.6b FrontPage/5.0.2.2635 mod_throttle/3.1.2)
* docElt: {http://www.google.com/schemas/sitemap/0.84}sitemapindex
* Validation was strict, starting with type [Anonymous]
* schemaLocs: http://www.google.com/schemas/sitemap/0.84 -> http://www.google.com/schemas/sitemap/0 ... eindex.xsd
* The schema(s) used for schema-validation had
no errors
* No schema-validity problems were found in the target

Schema resources involved

Attempt to load a schema document from
http://www.google.com/schemas/sitemap/0 ... eindex.xsd
(source: schemaLoc) for
http://www.google.com/schemas/sitemap/0.84,
succeeded

100% valid output, that's what this is

++

asiaexpat · by **asiaexpat** » Fri Aug 11, 2006 1:34 pm

hello

I am still getting error messages and have posted in the google users forum to see why there are no error descriptions--you can check that thread

http://groups.google.com/group/Google_W ... ba655157c3

the poster tells me to check the robots file

I thought we already verified that it was ok?

dcz · by **dcz** » Fri Aug 11, 2006 1:39 pm

Code: Select all: Disallow: /phpBB2/forum-

does not help, delete this line and it should be ok.

++

asiaexpat · by **asiaexpat** » Fri Aug 11, 2006 1:44 pm

Done
lets see how it reacts--thanks

asiaexpat · by **asiaexpat** » Sun Aug 13, 2006 1:52 am

Still errors but only 19 of 34 files have errors now

another reply here about www prefix
http://groups.google.com/group/Google_W ... ba655157c3

why would this one have errors
http://asia-expatsforum.com/phpBB2/forum-sitemap-8.xml
and this one no errors
http://asia-expatsforum.com/phpBB2/forum-sitemap-5.xml

also --sample from robots analysis

over 3000 urls restricted

for example

http://www.asia-expatsforum.com/phpBB2/ ... 8e15b424... URL restricted by robots.txt [?] Aug 8, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 81585ac4... URL restricted by robots.txt [?] Aug 9, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 0802d026... URL restricted by robots.txt [?] Aug 6, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 09df100c... URL restricted by robots.txt [?] Aug 9, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 59ffe904... URL restricted by robots.txt [?] Aug 7, 2006
http://www.asia-expatsforum.com/phpBB2/ ... f6fb3d13... URL restricted by robots.txt [?] Aug 6, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 693a63b9... URL restricted by robots.txt [?] Aug 5, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 92b98e6b... URL restricted by robots.txt [?] Aug 5, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 94aacb5d... URL restricted by robots.txt [?] Aug 9, 2006

asiaexpat · by **asiaexpat** » Sun Aug 13, 2006 12:26 pm

well

it wentr from 19 errors down to 4 and then to 2 and now

0 errors

strange or not??

dcz · by **dcz** » Sun Aug 13, 2006 1:24 pm

Not at all actually, Google had to revisit every sitemap before it updated it's status, as I told it will

Then, it is normal that post url appears disallowed in Google stats, they are, and it is an SEO requirement to disallow those since they are the number one source of Duplicate in a phpBB forum.
You'll see, there is no need to output them in Search engine result, since all your content is already listed in Topic URLs, you'll just Optimize PageRanking in your site by concentrating everything on topic URLs this way.

And actually, the mx Google sitemaps does not output any post links, these stats are concerning your site in general, not the sitemap.

++

asiaexpat · by **asiaexpat** » Sun Aug 13, 2006 1:50 pm

ok good

the number of indexed pages has been jumping around from 219 -180 I expect in a few weeks it will balloon into the thousands

dcz · by **dcz** » Sun Aug 13, 2006 2:40 pm

Actually, yes.

And since you had many pages indexed with SIDs in Google, and those where using different URLs than the one you are now using you could do one thing to help google.

first, update your robots.txt and add :

Code: Select all: Disallow: /phpBB2/forum-

Yes, add it, we'll take it out after.

Then go to this page http://www.google.com/support/webmasters/bin/answer.py?answer=35303
And use the automatic URL removal system link.

Once done, take this out of your robots.txt :

Code: Select all: Disallow: /phpBB2/forum-

And within few days, all those useless and disallowed URLs will have disappeared of Google's Cache, allowing him to concentrate more on mx Google sitemaps and your new URLs.

And as a last, but not least, effort, you could consider installing mx Sitemaps, not that hard in the end, and really worth it.

I just had time to tweak it for the phpBB SEO multi-forums and domains set up and installed it here on this site : http://www.phpbb-seo.com/sitemaps.html
Check the footer links

This mod really helps out for PageRank.

++

asiaexpat · by **asiaexpat** » Sun Aug 13, 2006 2:53 pm

I am a little confused about this automatic url remove system.

I should follow this?
Note:If you believe your request is urgent and cannot wait until the next time Google crawls your site, use our automatic URL removal system.In order for this automated process to work, the webmaster must first insert the appropriate meta tags into the page's HTML code. Doing this and submitting via the automatic URL removal system will cause a temporary, 180-day removal of these pages from the Google index, regardless of whether you remove the robots.txt file or meta tags after processing your request.

my mx Google sitemaps htaccess is ok?

my mx Google sitemaps htaccess is ok?

Advertisement

Who is online

Advertisement