problem with site crawl test

crokseth · Joined: 11 Feb 2008 Posts: 3

Hi

I have installed phpBB3 SEO Premod 3.0.0 for the purpose of a side by side comparison with the phpBB3 standard version.

Site1: (phpBB3 SEO Premod 3.0.0 )
http://www.visiwork.com

Site2: (phpBB3 standard )
http://www.pixavi.com/community

I am getting better crawl test results from http://www.seomoz.org/crawl-test/ with the standard phpBB install than with the phpBB3 SEO premod.

The phpBB3 SEO premod site returns 301 redirect messages and will only crawl one page. I have also tested other crawl test tools like WebCEO with the same result.

Ironically it looks like I need to stick with phpBB3 standard version to have any chance of getting the site indexed.

I have experimented a lot with the settings without any luck. My settings currently look like:

Activate URL rewriting: YES
URL rewriting type:ADVANCED
Profiles and groups injection:NO
Profiles ID removing:NO
Search User Page rewriting:YES
SID Removing:NO
Highlights Removing:YES
Remove small words:YES
Virtual Folder:YES
Virtual Root:NO
Forum URL caching:YES
Forum ID Removing:NO
No duplictate: Yes
Zero duplictate: Yes
Posts Redirections: POST
Strict Mode: YES

(I always update the .htaccess after changing settings requiring this)

I want to make this work using your mod. Please advice how!

SeO · Posted: Tue Feb 12, 2008 8:41 am Post subject: Re: problem with site crawl test

Performing crawling test on a 1 message board is pretty much meaningless Rolling Eyes

Besides, the result actually obtained are the exact opposite of what you stand :
Without URL rewriting :
http://www.google.com/search?q=site:www.pixavi.com/community/ => 0 pages
With url rewriting :
http://www.google.com/search?q=site:www.visiwork.com => 1 page and the image dir.

Then, all that we can conclude about the SEOmoz crawl test is that their test is not a good one, the HTTP 301 only means that their tool does not accept cookies, which is not the real SE bots behaviour.

Of course, the HTTP 301 for user that do not accept cookie should and will be better handled in next versions, but, it's not an SEO issue at all.
And we need SID for user without cookies to be able to log in so ...

And please note, for your future experiments, that switching URL rewriting while the board is online and indexed is not something that will help bots to properly crawl your site, even if you properly HTTP 301 redirect url from previous settings (which is not the case by default), it's disturbing for SE to find new URL every other day.

The idea is to define a fine and personalized URL standard before you go online and then stick with it.

Then, Search Engine Optimization only is Optimization, it will under no circumstances replace content and the eventual interest it could have for people reading it.

crokseth · Joined: 11 Feb 2008 Posts: 3

Hi. I have not published the first site, so I am not suprised that Google has not listed it. Performing the Seomoz crawl test "offline" before google bots got to my site was done with the exact purpose on experimenting before going public, as you recommend. Is there any other (better) way of crawl-checking the site before serving it to the bots?

I noted that google had managed to get hold of the second site Google still indexes www.visiwork.com without titles, description etc.

Could you also recommend settings for the last 4 points, as I find the documentation a but confusing on these:

No duplictate: Yes
Zero duplictate: Yes
Posts Redirections: POST
Strict Mode: YES

SeO · Posted: Fri Feb 15, 2008 3:31 pm Post subject: Re: problem with site crawl test

Yes, your settings are correct.

You could need to deactivate strict mod in case you install a mod and have trouble with url redirected that shouldn't.

About crawling, you can just trust us, we will never release a script that does not allow crawling, it's pretty much the total opposite Wink