Google does not come further than the index

emielvenlo · PR0 Joined: 21 Jan 2007 Posts: 79

Since i have moved from able2know mod_rewrite to advanced mod rewrite, zero dupe and optimal titles, my forum is not crawled by google anymore.

Sometimes googlebot visits, but it visits only the index pages of the portal and the forum. None of the topics made after the change (1 week ago) are indexed. And all the urls in google are still the ones from able2know. Before, google indexed new topics within one or two day's.
My first thought was that robots.txt was not allowing google to visit topics, but i cant find anything weird.

emielvenlo · PR0 Joined: 21 Jan 2007 Posts: 79

I will post my .htaccess, in case it is needed:

dcz

Well, first, the able2know rewriterules should be installed at the very end of the .htaccess.

Will save some process as they should with time, be less and less used. The [L] flag at the end of the rewriterules make so no other rewriterule is tested after the first one that matched.
But this is another matter.

Now 47 indexed pages for the forum (44 cached is not exactly like if Google was only visiting your index.

And if I recall well, you first where issuing 404 on all the old able2know urls, depending on how long this lasted, it can take some time for Google to re-crawl these.

But, it really looks like indexing is on its way, and remember, you're running a duplicate free forum now, meaning you'll have quite some less url listed, but a lot better ranked.

Your index is showing about 350 topics, you should not end up with many more URL listed, total is the number of topic, plus the number of forums, plus the number of pages for forums and topics, pretty much what Google sitemaps would show as the total URL amount, if you are using GYM sitemaps.

++

mhmdkhamis · PR4 Joined: 26 Nov 2006 Posts: 436 Location: egypt

use gym it the best Wink

i was have same problem from 2 month ago but i wasnt use able2know mod

but after gym google now crawl all my site *it the best* Idea

emielvenlo · PR0 Joined: 21 Jan 2007 Posts: 79

dcz

Well, you can run more than one sitemap if you like, and it's not harmful at all to only provide sitemaps for the forum.

As you can see on phpBB SEO, we do not run sitemaps for our directories so far, we do not include our html sitemap in it either and these pages are still very well indexed.

A sitemap will just help out Google to crawl you forum in a more efficient way, more cached pages with fewer visits, that's good to take for all, us and Google.
In all case, it won't cause your portal to be less indexed, actually, it's the contrary since Google will spend less resources to index your forum, it will have more time to spend on your domain.

++