my mx Google sitemaps htaccess is ok?

Support for the phpBB2 SEO mods released in the phpBB2 SEO Toolikt forum.

Moderator: Moderators

my mx Google sitemaps htaccess is ok?

Postby asiaexpat » Wed Aug 09, 2006 1:36 pm

Ok how is this??
we have tried already to use a different redirect method but did not work


Code: Select all
Options +FollowSymlinks
RewriteEngine on
RewriteRule ^$ http://www.asia-expatsforum.com/phpBB2/portal.php [QSA,L,R=301]
RewriteCond %{HTTP_HOST} ^asia-expatsforum\.com$ [NC]
RewriteRule ^(.*) http://www.asia-expatsforum.com/$1 [QSA,L,R=301]
RewriteBase /
# phpBB

# FORUM PROTECTION RULE
RewriteRule ^phpBB2/.+/([^/]+\.html)$ /phpBB2/index.php [R=301,L]
# CATEGORIES
RewriteRule ^phpBB2/.+-vc([0-9]+)\.html$ /phpBB2/index.php?c=$1 [QSA,L]
# PAGINATED FORUM
RewriteRule ^phpBB2/.+-vf([0-9]+)-([0-9]+)\.html$ /phpBB2/viewforum.php?f=$1&start=$2 [QSA,L]
# FORUM
RewriteRule ^phpBB2/.+-vf([0-9]+)\.html$ /phpBB2/viewforum.php?f=$1 [QSA,L]
# PAGINATED TOPIC
RewriteRule ^phpBB2/.+-vt([0-9]+)-([0-9]+)\.html$ /phpBB2/viewtopic.php?t=$1&start=$2 [QSA,L]
# TOPIC
RewriteRule ^phpBB2/.+-vt([0-9]+)\.html$ /phpBB2/viewtopic.php?t=$1 [QSA,L]
# POST
RewriteRule ^phpBB2/post([0-9]+)\.html$ /phpBB2/viewtopic.php?p=$1 [QSA,L]
#PROFILES
RewriteRule ^phpBB2/member([0-9]+)\.html$ /phpBB2/profile.php?mode=viewprofile&u=$1 [QSA,L]
RewriteRule ^phpBB2/sitemaps.xml$ /phpBB2/sitemap.php [L]
RewriteRule ^phpBB2/forum-sitemap-([0-9]+).xml$ /phpBB2/sitemap.php?fid=$1 [L]
RewriteRule ^phpBB2/sitemap-forum.xml$ /phpBB2/sitemap.php?forum [L]
# END PHPBB PAGES
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Advertisement

Postby dcz » Wed Aug 09, 2006 1:41 pm

ho please calm down :lol:

Why starting a new thread when this one http://boards.phpbb-seo.com/phpbb-mod-r ... vt175.html is already here ?

Besides it is working nicely now : http://www.asia-expatsforum.com/phpBB2/ ... map-17.xml (I just found out the first link you posted was missing an "l" at the end (.xm instead of .xml ;) )

And output the correct advanced mod rewrite URLs ;)

So you can safely go ahead and submit your sitematIndex ( http://www.asia-expatsforum.com/phpBB2/sitemaps.xml ) :D

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21379
Joined: Fri Apr 28, 2006 9:03 pm

Postby asiaexpat » Wed Aug 09, 2006 1:49 pm

Lets see if it returns errors again

Anyway--thank you so much for your help.

the first link was missing an "l" because i did not copy it--it was there
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Postby dcz » Wed Aug 09, 2006 1:57 pm

yes I know, and maybe the mod was already working before as I tested the first one only the second time :lol:

Don't worry, the output is 100% valid and the mod is very efficient ;)

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21379
Joined: Fri Apr 28, 2006 9:03 pm

Postby asiaexpat » Wed Aug 09, 2006 2:30 pm

I still get google errors

View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-1.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-10.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-11.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-12.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-13.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-14.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-15.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-16.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-17.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-18.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-19.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-2.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-20.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-21.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-22.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-23.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-24.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-25.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-26.xml Errors
View errors for: http://asia-expatsforum.com/phpBB2/forum-sitemap-27.xml Errors
View errors for: http://

all the way to 34


But you say this is normal?? I should just ignore it??

There is a second htacess file in my /asia-expatsforum.com/phpBB2/mx_ggsitemaps/includes/ folder

that file contains
Code: Select all
<Limit GET POST PUT>
Order Allow,Deny
Deny from All
</Limit>


Is that correct??
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Postby dcz » Wed Aug 09, 2006 3:02 pm

There is just no way Google spidered all those this past hour, these are your old stats ;)


From : http://www.smart-it-consulting.com/inte ... /index.htm

Schema validating with XSV 2.10-1 of 2005/04/22 13:10:49

* Target: http://www.asia-expatsforum.com/phpBB2/ ... map-10.xml
(Real name: http://www.asia-expatsforum.com/phpBB2/ ... map-10.xml
Server: Apache/1.3.31 (Unix) PHP/4.3.11 mod_ssl/2.8.18 OpenSSL/0.9.6b FrontPage/5.0.2.2635 mod_throttle/3.1.2)
* docElt: {http://www.google.com/schemas/sitemap/0.84}urlset
* Validation was strict, starting with type [Anonymous]
* schemaLocs: http://www.google.com/schemas/sitemap/0.84 -> http://www.google.com/schemas/sitemap/0.84/sitemap.xsd
* The schema(s) used for schema-validation had
no errors
* No schema-validity problems were found in the target


Schema resources involved

Attempt to load a schema document from
http://www.google.com/schemas/sitemap/0.84/sitemap.xsd
(source: schemaLoc) for
http://www.google.com/schemas/sitemap/0.84,
succeeded



And :
Schema validating with XSV 2.10-1 of 2005/04/22 13:10:49

* Target: http://www.asia-expatsforum.com/phpBB2/sitemaps.xml
(Real name: http://www.asia-expatsforum.com/phpBB2/sitemaps.xml
Server: Apache/1.3.31 (Unix) PHP/4.3.11 mod_ssl/2.8.18 OpenSSL/0.9.6b FrontPage/5.0.2.2635 mod_throttle/3.1.2)
* docElt: {http://www.google.com/schemas/sitemap/0.84}sitemapindex
* Validation was strict, starting with type [Anonymous]
* schemaLocs: http://www.google.com/schemas/sitemap/0.84 -> http://www.google.com/schemas/sitemap/0 ... eindex.xsd
* The schema(s) used for schema-validation had
no errors
* No schema-validity problems were found in the target


Schema resources involved

Attempt to load a schema document from
http://www.google.com/schemas/sitemap/0 ... eindex.xsd
(source: schemaLoc) for
http://www.google.com/schemas/sitemap/0.84,
succeeded


100% valid output, that's what this is ;)

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21379
Joined: Fri Apr 28, 2006 9:03 pm

Postby asiaexpat » Fri Aug 11, 2006 1:34 pm

hello

I am still getting error messages and have posted in the google users forum to see why there are no error descriptions--you can check that thread

http://groups.google.com/group/Google_W ... ba655157c3

the poster tells me to check the robots file

I thought we already verified that it was ok?
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Postby dcz » Fri Aug 11, 2006 1:39 pm

Code: Select all
Disallow: /phpBB2/forum-
does not help, delete this line and it should be ok.


++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21379
Joined: Fri Apr 28, 2006 9:03 pm

Postby asiaexpat » Fri Aug 11, 2006 1:44 pm

Done
lets see how it reacts--thanks
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Postby asiaexpat » Sun Aug 13, 2006 1:52 am

Still errors but only 19 of 34 files have errors now

another reply here about www prefix
http://groups.google.com/group/Google_W ... ba655157c3

why would this one have errors
http://asia-expatsforum.com/phpBB2/forum-sitemap-8.xml
and this one no errors
http://asia-expatsforum.com/phpBB2/forum-sitemap-5.xml

also --sample from robots analysis

over 3000 urls restricted

for example

http://www.asia-expatsforum.com/phpBB2/ ... 8e15b424... URL restricted by robots.txt [?] Aug 8, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 81585ac4... URL restricted by robots.txt [?] Aug 9, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 0802d026... URL restricted by robots.txt [?] Aug 6, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 09df100c... URL restricted by robots.txt [?] Aug 9, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 59ffe904... URL restricted by robots.txt [?] Aug 7, 2006
http://www.asia-expatsforum.com/phpBB2/ ... f6fb3d13... URL restricted by robots.txt [?] Aug 6, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 693a63b9... URL restricted by robots.txt [?] Aug 5, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 92b98e6b... URL restricted by robots.txt [?] Aug 5, 2006
http://www.asia-expatsforum.com/phpBB2/ ... 94aacb5d... URL restricted by robots.txt [?] Aug 9, 2006
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Postby asiaexpat » Sun Aug 13, 2006 12:26 pm

well

it wentr from 19 errors down to 4 and then to 2 and now

0 errors

strange or not??
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Postby dcz » Sun Aug 13, 2006 1:24 pm

Not at all actually, Google had to revisit every sitemap before it updated it's status, as I told it will ;)

Then, it is normal that post url appears disallowed in Google stats, they are, and it is an SEO requirement to disallow those since they are the number one source of Duplicate in a phpBB forum.
You'll see, there is no need to output them in Search engine result, since all your content is already listed in Topic URLs, you'll just Optimize PageRanking in your site by concentrating everything on topic URLs this way.

And actually, the mx Google sitemaps does not output any post links, these stats are concerning your site in general, not the sitemap.

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21379
Joined: Fri Apr 28, 2006 9:03 pm

Postby asiaexpat » Sun Aug 13, 2006 1:50 pm

ok good

the number of indexed pages has been jumping around from 219 -180 I expect in a few weeks it will balloon into the thousands :D :D :D
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Postby dcz » Sun Aug 13, 2006 2:40 pm

Actually, yes.

And since you had many pages indexed with SIDs in Google, and those where using different URLs than the one you are now using you could do one thing to help google.

first, update your robots.txt and add :
Code: Select all
Disallow: /phpBB2/forum-


Yes, add it, we'll take it out after.

Then go to this page http://www.google.com/support/webmasters/bin/answer.py?answer=35303
And use the automatic URL removal system link.

Once done, take this out of your robots.txt :

Code: Select all
Disallow: /phpBB2/forum-


And within few days, all those useless and disallowed URLs will have disappeared of Google's Cache, allowing him to concentrate more on mx Google sitemaps and your new URLs.

And as a last, but not least, effort, you could consider installing mx Sitemaps, not that hard in the end, and really worth it.

I just had time to tweak it for the phpBB SEO multi-forums and domains set up and installed it here on this site : http://www.phpbb-seo.com/sitemaps.html
Check the footer links ;)

This mod really helps out for PageRank.

++
Useful links :
SEO Forum || SEO Directory || SEO phpBB || Search
____________________

Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Recherche
dcz
Admin
Admin
 
Posts: 21379
Joined: Fri Apr 28, 2006 9:03 pm

Postby asiaexpat » Sun Aug 13, 2006 2:53 pm

I am a little confused about this automatic url remove system.

I should follow this?
Note:If you believe your request is urgent and cannot wait until the next time Google crawls your site, use our automatic URL removal system.In order for this automated process to work, the webmaster must first insert the appropriate meta tags into the page's HTML code. Doing this and submitting via the automatic URL removal system will cause a temporary, 180-day removal of these pages from the Google index, regardless of whether you remove the robots.txt file or meta tags after processing your request.
asiaexpat
PR1
PR1
 
Posts: 121
Joined: Fri Aug 04, 2006 4:29 am

Next

Return to phpBB2 SEO MODS

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 2 guests