Something very wrong with my google results

[GoBieN]

Hey,

I have phpBB 3.0.2 with latest Seo-mod (advanced) and GYM sitemaps/RSS.
I have set virtual root, virtual folder. Zero dupe and no dupe are on.

The problem is that when i go see my results in google for the important keywords, my site is no longer on 1st or 2nd but way lower.

This happens because everytime google tries to index my page they get an error saying they are not allowed to use the search function ?

look here:
http://64.233.183.104/search?q=cache:3L ... cd=1&gl=be

Informatie
Sorry maar u wordt niet toegelaten om het zoekssysteem te gebruiken.

english translation:

Information
Sorry you are not allowed to use the search function.

When i check all the results from my site, they are all like that:
http://www.google.be/search?aq=f&hl=en& ... eken&meta=

Please advise ?

[GoBieN]

what i have done now is changed the permisions for bots so that they are allowed to use search function. But this can't be standard behaviour, something must have gone wrong, somewhere.

[dcz]

By default search pages and profiles are not public on phpBB3.
If you make them available for bots, I advise you to as well make these available for guests to be sure you won't be cloaking the Search engines.

But, the worst thing is that, as a matter of fact, http://www.google.com/search?q=cache:ww ... forum.html
bots do not have access to your forum index

You should fix the bots auth ASAP, and provide them with the same auth as guest (it's the safest).

++

[GoBieN]

But WHY is the forum telling the bot, it is not allowed to use the search function ?

i changed nothing to the settings that could cause this ?

[SeO]

For the search function, it's by default on phpBB3 that bots cannot access it, same for user profiles.

[GoBieN]

For the search function, it's by default on phpBB3 that bots cannot access it, same for user profiles.

But why would they get an error about the search function on the forum index and every other page ?

This can't be normal behaviour !

[SeO]

Well what occurs is that people trying to view the search results pages (including user posts pages) when they don't have enough right to will fell on the login box with a HTTP 302 header.

For some reason, while Google will in such case keep the original url in mind (since 302 means that the uri exists) it will still apply the disallow set in the robots.txt for the login page :

Code: Select all

Disallow: /phpBB3/ucp.php

The good thing about this when everything is set fine is that bots won't repeatedly cache the login page under zillions of different urls.

[GoBieN]

Let me make myself more clear.

Why would google get a error about not allow to search on these pages:

www.camino-tuning.be/forum.html <-- forum index (no logn box, no search page no profile ... just the plain index) but google got error anyway !

www.camino-tuning.be/koop-f5/ <-- subforum nr5 index -> error

www.camino-tuning.be/filmpjes-galerij-f27 <-- subforum nr 27 index -> error


I don't see any reason why google bot would get an error about not being allowed to search, when the bot is NOWHERE NEAR the search page.

[dcz]

Well yes, but the result is the same if these pages are not available for bots, bots see the login page or an info page with the same HTTP 302 header when trying to browse private forums :

http://www.google.com/search?q=cache:ww ... e/koop-f5/

[GoBieN]

But my question is how come this happend ?

Your reaction seem to indicate this is normal behaviour, i can not believe it would be normal to get an error on the index.

[dcz]

No it's not "normal", but it's not really an error either, it's just that bots auth where not properly set on your forum that's all.
The normal thing is that, if no forums are viewable by bots, it's the same for the forum index.

++

[GoBieN]

No it's not "normal", but it's not really an error either, it's just that bots auth where not properly set on your forum that's all.
The normal thing is that, if no forums are viewable by bots, it's the same for the forum index.

++

Okay i understand that, but this problem just happend recently, without me changing any settings. The only thing i did was update seo to 3.0.2 and installed GYM rss/sitemaps.

Also i would suspect a different error, not an error stating the search is not allowed, this error makes no sens on the index page !

[SeO]

The normal thing is that, if no forums are viewable by bots, it's the same for the forum index.

Have you updated you bot's authorisations ?

[GoBieN]

I have now allowed bots to use the search function. Because i don't know any other solution.

Google is slowly showing correct results again, but I payed big time, my ratings dropped and google is not indexing my pages so good anymore.

[dcz]

Have you as well made sure that bots had the right to visit your public forums ?

I know that what's occurring to you right now is far from funny, but it's not related to our mods, authorisation are always to be checked thoroughly if not redone from scratch (since it's a good occasion) when converting.

Keep faith, your indexing should go back to where it was ... with time.

++