Actually, you don't care about the meta here, you'll just end up submitting your robots.txt, and Google will do the rest with the datas in the submitted robots.txt.

That's why putting it back to what it was first like will do the trick for mx sitemaps, Google won't fell again on a disallow for "forum" after you submitted.
The sitemaps pages won't be thrown out of index, actually, they are not indexed, just crawled.

Doing this will just delete all those from the Google listing, which is rather an enhancement as it will prevent Google from loosing time spidering again and again useless pages.
So far the robots.txt updates only moved those to the omitted results, and they'll end up being throw out as Google follows robots.txt instructions, but after a lot more time than if you do it at once.

And you don't care losing all those entries, because they are really bad, SIDs in most, meaning no PageRank and duplicate of the good ones.
With your actual PageRank of 4, I think Google will re-index you fully quite fast.

Actually, as said in the release post, a site like yours just went from less than 10 pages in cache to 37 800 one month after thanks to mx Google sitemaps and the phpBB SEO mod Rewrites. This is a fact.

++

ok

so I add the entry to robots.txt

go right to url remomal tool and remove all urls
then go straight back to and remove the entry from robots?

all in like 10 minutes time?

and then after that I can aplly the mx sitemap mod?

Exactly
But you can as well let him find out those are disallowed now, as it already did, it's just going to be a bit longer.

Then, for sure you must now experience less user coming from Google while your pages goes to trash like this, but trust me, there is no way those old links with SID in it would show up in a better position than the new ones you just added.
As soon as they'll get re indexed, they'll show up at least in the same position and soon in a better one.
And this is mostly because those add SIDs in them, and where recently indexed, meaning no PageRank or anything like it, so you will not lose nothing in the end, just dupes and useless entries we have to get rid of anyway.

And this is as well going to be quite fast because of your PageRank, Google will just love to crawl your site and this is going to be quite fast too.


For mx Sitemaps, it's a regular, but Search Engine Optimized, html site map. You can install it when you want, the faster the better here, especially when having to re index like this.

++

ok
I have used the remove url tool and that now is set to status pending.
so i should not put back robot.txt as was until the status changes to completed or something else??

Nope, put it back to what it was right now, the robots.txt content was submitted and will be taken into account as it was for this task.

It should be couple days before they'll vanish from Google index

I bet you'll soon have to upgrade ISP

++

ok removed

now i will install the mx-sitemaps mod

so it is ok to overwrite my existing sitemaps.php file with the one in the installation package of mx-sitemaps v.0.01


now i see the file to add is sitemaps.php when i already have an sitemap.php file

they are different--intended??

OK mx sitemaps installed

now what??

Perfect, now you need to activate mod rewrite for it and to install the auto link add on and rewriterules.
Everything is explained in the release thread (page 4)

++

ok

have done that

now what??

how should I configure the new acp panels??

You need to implement the suggested code changes in sessions.php as well, rewriterules seems ok. Auto link add on too.

Then, you'll need this patch and this one too (don't worry it is simple, and I'll fix all those in next release )

Once done, go acp and add as many categories and link as you like in the site map index. As well, you can play with the last topic listing settings to output nice last topic lists in those. As well, you can play with the pagination parameters, allowing you for example to output 50 topic per page in the forum map view while you are using 25 in your true forums.

Take a look at the way I set it up here : http://www.phpbb-seo.com/sitemaps.html

++

ok

have applied patches

in the forum map when drill down to post level it reports error

for example

http://www.asia-expatsforum.com/phpBB2/ ... -fmp6.html

then click on a link

some problems

when try to open post from recent topic box on portal--cannot open

so I think my portal.php needs sopme changes again

and cannot open links from index--but can from last post icon

my htfile is as follows

Code: Select all

Options +FollowSymlinks
RewriteEngine on
RewriteRule ^$ http://www.asia-expatsforum.com/phpBB2/portal.php [QSA,L,R=301]
RewriteCond %{HTTP_HOST} ^asia-expatsforum\.com$ [NC]
RewriteRule ^(.*) http://www.asia-expatsforum.com/$1 [QSA,L,R=301]
RewriteBase /
# phpBB

# FORUM PROTECTION RULE
RewriteRule ^phpBB2/.+/([^/]+\.html)$ /phpBB2/index.php [R=301,L]
# CATEGORIES
RewriteRule ^phpBB2/.+-vc([0-9]+)\.html$ /phpBB2/index.php?c=$1 [QSA,L]
# PAGINATED FORUM
RewriteRule ^phpBB2/.+-vf([0-9]+)-([0-9]+)\.html$ /phpBB2/viewforum.php?f=$1&start=$2 [QSA,L]
# FORUM
RewriteRule ^phpBB2/.+-vf([0-9]+)\.html$ /phpBB2/viewforum.php?f=$1 [QSA,L]
# PAGINATED TOPIC
RewriteRule ^phpBB2/.+-vt([0-9]+)-([0-9]+)\.html$ /phpBB2/viewtopic.php?t=$1&start=$2 [QSA,L]
# TOPIC
RewriteRule ^phpBB2/.+-vt([0-9]+)\.html$ /phpBB2/viewtopic.php?t=$1 [QSA,L]
# POST
RewriteRule ^phpBB2/post([0-9]+)\.html$ /phpBB2/viewtopic.php?p=$1 [QSA,L]
#PROFILES
RewriteRule ^phpBB2/member([0-9]+)\.html$ /phpBB2/profile.php?mode=viewprofile&u=$1 [QSA,L]
RewriteRule ^phpBB2/sitemaps.xml$ /phpBB2/sitemap.php [L]
RewriteRule ^phpBB2/forum-sitemap-([0-9]+).xml$ /phpBB2/sitemap.php?fid=$1 [L]
RewriteRule ^phpBB2/sitemap-forum.xml$ /phpBB2/sitemap.php?forum [L]
# END PHPBB PAGES
RewriteRule ^phpBB2/sitemaps([0-9]+)\.html$ /phpbb/sitemaps.php?c=$1 [QSA,L]
RewriteRule ^phpBB2/sitemaps\.html$ /phpBB2/sitemaps.php [QSA,L]
RewriteRule ^phpBB2/forum-m\ap\.html$ /phpBB2/sitemaps.php?fim [QSA,L]
RewriteRule ^phpBB2/.+-fmp([0-9]+)-([0-9]+)\.html$ /phpBB2/sitemaps.php?fmp=$1&start=$2 [QSA,L]
RewriteRule ^phpBB2/.+-fmp([0-9]+)\.html$ /phpBB2/sitemaps.php?fmp=$1 [QSA,L]
RewriteRule ^phpBB2/.+-sc([0-9]+)\.html$ /phpBB2/sitemaps.php?c=$1 [QSA,L]

actually you messed up with all topic urls.

I am sorry it seems I told you to use the patch for the mixed mod.

So here is what should be used for your happend_sid function in sessions.php

Code: Select all

function append_sid($url, $non_html_amp = false)
{
   global $SID;   
   // www.phpBB-SEO.com SEO TOOLKIT BEGIN
   global $phpEx, $seo_cat_name, $seo_forum_name, $seo_topic_name;
   $amp = ($non_html_amp) ? '&' : '&';
   $find = array(   $amp.'start=0',
         $amp.'postdays=0',
         $amp.'topicdays=0',
         $amp.'postorder=asc',
      );
   $url = str_replace ($find, "", $url);
   if ( $seo_cat_name ) {
      $urlin = array(   "'(?)index." . $phpEx . "\?c=([0-9]+)(". $amp ."){0,1}'",
            // mx Sitemaps Module (cat URL injection)
            "'(?)sitemaps([0-9]+)\.html(". $amp ."){0,1}'",
         );
      $urlout = array( "". format_url($seo_cat_name)."-vc\\1.html" . if_query('\\2')."",
             // mx Sitemaps Module (cat URL injection)
             "". format_url($seo_cat_name)."-sc\\1.html". if_query('\\2')."",
         );
      $url = preg_replace($urlin, $urlout, $url);
   }
   if ( $seo_forum_name ) {
      $urlin = array( "'(?)viewforum.".$phpEx."\?f=([0-9]+)". $amp ."start=([0-9]+)(". $amp ."){0,1}'",
            "'(?)viewforum.".$phpEx."\?f=([0-9]+)(". $amp ."){0,1}'",
            // mx Sitemaps Module (forum URL injection + pagination)
            "'(?)forum-map([0-9]+)\.html". $amp ."start=([0-9]+)(". $amp ."){0,1}'",
            "'(?)forum-map([0-9]+)\.html(". $amp ."){0,1}'",
         );
      $urlout = array( "". format_url($seo_forum_name)."-vf\\1-\\2.html". if_query('\\3')."",
             "". format_url($seo_forum_name)."-vf\\1.html". if_query('\\2')."",
             // mx Sitemaps Module (forum URL injection + pagination)
             "". format_url($seo_forum_name)."-fmp\\1-\\2.html". if_query('\\3')."",
             "". format_url($seo_forum_name)."-fmp\\1.html". if_query('\\2')."",
         );
      $url = preg_replace($urlin, $urlout, $url);
   }
   if ($seo_topic_name) {
      $urlin = array(   "'(?)viewtopic.". $phpEx ."\?t=([0-9]+)". $amp ."start=([0-9]+)(". $amp ."){0,1}'",
            "'(?)viewtopic.".$phpEx."\?t=([0-9]+)(".$amp."){0,1}'",
         );
      $urlout = array("". format_url($seo_topic_name)."-vt\\1-\\2.html". if_query('\\3')."",
            "". format_url($seo_topic_name)."-vt\\1.html". if_query('\\2')."",
         );
      $url = preg_replace($urlin, $urlout, $url);
   }
   $urlin = array(   "'(?)viewtopic.". $phpEx ."\?p=([0-9]+)(". $amp ."){0,1}'",
         "'(?)profile.".$phpEx."\?mode\=viewprofile".$amp."u=([0-9]+)(".$amp."){0,1}'",
         // mx Sitemaps Module (mxBB map URL + pagination)
         "'(?)mx-map\.html". $amp ."start=([0-9]+)(". $amp ."){0,1}'",
      );
   $urlout = array( "post\\1.html". if_query('\\2')."",
          "member\\1.html". if_query('\\2')."",
      );
   $url = preg_replace($urlin, $urlout, $url);
   $url = (substr($url, strlen($url) - 1, 1) == '?') ? substr($url, 0, strlen($url) - 1) : $url;
   // www.phpBB-SEO.com SEO TOOLKIT END

   if ( !empty($SID) && !preg_match('#sid=#', $url) )
   {
      $url .= ( ( strpos($url, '?') !== false ) ?  ( ( $non_html_amp ) ? '&' : '&' ) : '?' ) . $SID;
   }

   return $url;
}

Sorry.

++

no problem

i made the changes and it seems ok now
thanks

Magnificent

Everything is working nicely, mod rewrite, mx Google sitemaps and mx sitemaps.

You can just play with adding links in your site map index, will be outputted sorted in categories with images, like here.

++