Site has been taken off Google, why?

Discussions about the different Webmastering Tools from here and elsewhere.

Moderator: Moderators

Site has been taken off Google, why?

Postby teeb » Mon May 14, 2012 1:14 pm

Hey guys,

My site uses the premod version of phpBB SEO with default settings for everything. It is still indexed by Google, but it won't show up in the rankings. What are the most common reasons for phpBB SEO users to have this happen? I'm not breaking the guidelines btw (no keyword stuffing, cloaking, link building etc.)

Here's what I wrote on the official webmasters forums:

My domain is http://www.diablo3forum.co.uk

It was on page 1 of google not two days ago, and now has disappeared from the rankings altogether.

I did a site: operator search and my site is still there. However, when I just search my domain name, my site doesn't come up.

I have no messages/alerts in my Webmaster Tools panel indicating that I have breached the rules. I checked the quality guidelines thinking I might have accidentally triggered the site to be down ranked. I found that the forum software I was using was pinging google every time the sitemap was updated, which I thought went against one of the guidelines. I have now turned this feature off and sent a reconsideration request detailing the issue and changes made.

Also, my site uses phpBB - am I likely to have created duplicate content inadvertently? I've run phpBB systems before and never had this issue.

Can someone let me know if I have taken the right steps to getting things back to normal? Apart from the google pinging, my site is in line with the guidelines as far as I can see. It's really frustrating that the site has gone down now of all times, when people will be looking for a Diablo 3 forum just after release date. My site has been a labour of love for 2 years and this is incredibly demoralising.

Thanks,

Teeba


Building on what I've done above, is there anything else that could be problematic?

Also, one more question:

I want Google to crawl my HTML and RSS sitemaps, but I don't want them to show up as search results. How do I configure it so that the sitemaps continue to be crawled yet standard google searchers won't see the sitemaps in the search results?

Thanks,

Teeba
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Advertisement

Re: Site has been taken off Google, why?

Postby Stephen26 » Mon May 14, 2012 5:01 pm

The site loads really slow. Your homepage is indexed and accessible via four different URLs (http://diablo3forum.co.uk/ + http://diablo3forum.co.uk/index.php + http://www.diablo3forum.co.uk/ + http://www.diablo3forum.co.uk/index.php) = duplicate content.

Decide which version you want to keep (www or non-www) and then solve the issue with 301 redirects.
Also use the "Fetch as Google" + "Instant Preview" features in the WMT to diagnose problems and have a look at -> Labs -> site performance.
I install and configure phpBB-SEO (and other mods) for a fair price and very fast. If you are interested simply send me a private message.
Stephen26
phpBB SEO Team
phpBB SEO Team
 
Posts: 434
Joined: Mon May 03, 2010 7:06 am

Re: Site has been taken off Google, why?

Postby teeb » Mon May 14, 2012 5:15 pm

Stephen26 wrote:The site loads really slow. Your homepage is indexed and accessible via four different URLs <snip> = duplicate content.

Decide which version you want to keep and then solve the issue with 301 redirects.
Also use the "Fetch as Google" + "Instant Preview" features in the Webmaster Tools to diagnose problems.


Thanks for getting back to me on this.

As for the loading - yes it is, too much JS going on I reckon. Plus my crappy shared hosting :) I'll be moving to a VPS soon and validating the code. Does this affect my ranking on Google? How fast my page loads?

Okay I understand about the duplicate content, although I was unaware that this was happening. Say I want to keep http://diablo3forum.co.uk/index.php as the one URL. How would I go about arranging these 301 redirects? In my hosting panel? I don't know how to do this.

I've used the Fetch/instant preview tools before and returned no issues whatsoever, I'll check out the site performance one though.
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Re: Site has been taken off Google, why?

Postby Stephen26 » Mon May 14, 2012 5:47 pm

teeb wrote:Does this affect my ranking on Google? How fast my page loads?

Yes, it's a ranking factor, but normally one not too much to worry about, except when load time is really excessive.

teeb wrote:[Say I want to keep http://diablo3forum.co.uk/index.php as the one URL. How would I go about arranging these 301 redirects? In my hosting panel? I don't know how to do this.

This should help: http://www.seo-consultant-services.co.u ... blems.html
So keep the non-www version and also redirect /index.php to root /

Code: Select all
RewriteEngine On
RewriteCond %{HTTP_HOST} !^diablo3forum\.co\.uk$ [NC]
RewriteRule ^(.*)$ http://diablo3forum.co.uk/$1 [R=301,L]

RewriteCond %{THE_REQUEST} index\. [NC]
RewriteRule ^(.*)index\.(.*) /$1 [L,NC,R=301]


The .htaccess file has to be placed in your domain's root folder.
I install and configure phpBB-SEO (and other mods) for a fair price and very fast. If you are interested simply send me a private message.
Stephen26
phpBB SEO Team
phpBB SEO Team
 
Posts: 434
Joined: Mon May 03, 2010 7:06 am

Re: Site has been taken off Google, why?

Postby teeb » Mon May 14, 2012 6:07 pm

Stephen26 wrote:
teeb wrote:Does this affect my ranking on Google? How fast my page loads?

Yes, it's a ranking factor, but normally one not too much to worry about, except when load time is really excessive.


I got 75/100 for the speed page test, so it can't be excessive.

Stephen26 wrote:
teeb wrote:[Say I want to keep http://diablo3forum.co.uk/index.php as the one URL. How would I go about arranging these 301 redirects? In my hosting panel? I don't know how to do this.

This should help: http://www.seo-consultant-services.co.u ... blems.html
So keep the non-www version and also redirect /index.php to root /

Code: Select all
RewriteEngine On
RewriteCond %{HTTP_HOST} !^diablo3forum\.co\.uk$ [NC]
RewriteRule ^(.*)$ http://diablo3forum.co.uk/$1 [R=301,L]

RewriteCond %{THE_REQUEST} index\. [NC]
RewriteRule ^(.*)index\.(.*) /$1 [L,NC,R=301]


The .htaccess file has to be placed in your domain's root folder.


Thanks man, I used your code and put it in the .htaccess file under the comment "HERE IS A GOOD PLACE TO FORCE CANONICAL DOMAIN". I used the exact code - is that right?

So will this solve the duplicate content issue now? As in, will Google's bots find it and improve my ranking again?

Also, could you possibly answer this question for me?

I want Google to crawl my HTML and RSS sitemaps, but I don't want them to show up as search results. How do I configure it so that the sitemaps continue to be crawled yet standard google searchers won't see the sitemaps in the search results?

Thanks,

Teeb
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Re: Site has been taken off Google, why?

Postby Stephen26 » Mon May 14, 2012 6:40 pm

teeb wrote:I got 75/100 for the speed page test, so it can't be excessive.

Try to improve your load time nevertheless with the methods you mentioned above.
I also recommend optimizing all tables of your database if it's larger and you have never done that before. You can do this with phpMyAdmin or use this helpful and very fast to install phpBB mod: Database Optimize & Repair Tool.
You have Gzip enabled which is good.

teeb wrote:Thanks man, I used your code and put it in the .htaccess file under the comment "HERE IS A GOOD PLACE TO FORCE CANONICAL DOMAIN". I used the exact code - is that right?

"RewriteEngine On" is probably twice now in your .htaccess, in that case the code should just be:

Code: Select all
RewriteCond %{HTTP_HOST} !^diablo3forum\.co\.uk$ [NC]
RewriteRule ^(.*)$ http://diablo3forum.co.uk/$1 [R=301,L]

RewriteCond %{THE_REQUEST} index\. [NC]
RewriteRule ^(.*)index\.(.*) /$1 [L,NC,R=301]


teeb wrote:So will this solve the duplicate content issue now? As in, will Google's bots find it and improve my ranking again?

It will resolve the duplicate content issue. Click on the four links to your homepage I posted above and you should see correct redirection to the non-www version of your site. You may also use http://web-sniffer.net/ to check that better (server response must be 301 - Moved Permanently).

Why your ranking got worse is very hard to tell exactly, but solving problems with the site like that is the way to go. Maybe you get more tips on the Google webmaster forums.

teeb wrote:Also, could you possibly answer this question for me?

I want Google to crawl my HTML and RSS sitemaps, but I don't want them to show up as search results. How do I configure it so that the sitemaps continue to be crawled yet standard google searchers won't see the sitemaps in the search results?

I'm sorry, not sure what the correct way is to achieve that. Hopefully one of the other team members can help you with that.
I install and configure phpBB-SEO (and other mods) for a fair price and very fast. If you are interested simply send me a private message.
Stephen26
phpBB SEO Team
phpBB SEO Team
 
Posts: 434
Joined: Mon May 03, 2010 7:06 am

Re: Site has been taken off Google, why?

Postby teeb » Mon May 14, 2012 6:52 pm

Umm, I installed the optimise tool, and now I can't log into my ACP. It just keeps asking me to re-authenticate, I enter my password, and it takes me back to the exact same page. What should I do? Could it be a problem with the .htaccess code? I didn't put a second "RewriteEngine On" btw.

I checked the links in web sniffer and got successful 301 permanently moved on each one.

Np about the second question, you've been very helpful so far :)
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Re: Site has been taken off Google, why?

Postby teeb » Mon May 14, 2012 6:55 pm

I can confirm that this part of the .htaccess file is causing the problem:

Code: Select all
RewriteCond %{THE_REQUEST} index\. [NC]
RewriteRule ^(.*)index\.(.*) /$1 [L,NC,R=301]


Any way to achieve the same redirect with different code? I get the feeling that the above messes with how the redirect from acp login works.

EDIT: I used this:

Code: Select all
Redirect 301 /index.php http://diablo3forum.co.uk
instead of the above and checked it, it works according to web sniffer.
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Re: Site has been taken off Google, why?

Postby Stephen26 » Mon May 14, 2012 6:59 pm

Comment out the two lines regarding index.php in the .htaccess with a #

Code: Select all
#RewriteCond %{THE_REQUEST} index\. [NC]
#RewriteRule ^(.*)index\.(.*) /$1 [L,NC,R=301]


instead of

Code: Select all
RewriteCond %{THE_REQUEST} index\. [NC]
RewriteRule ^(.*)index\.(.*) /$1 [L,NC,R=301]


EDIT: Nevermind
I install and configure phpBB-SEO (and other mods) for a fair price and very fast. If you are interested simply send me a private message.
Stephen26
phpBB SEO Team
phpBB SEO Team
 
Posts: 434
Joined: Mon May 03, 2010 7:06 am

Re: Site has been taken off Google, why?

Postby teeb » Mon May 14, 2012 7:05 pm

Okay cool so I've got that sorted.. here's hoping that Google allow me back :) Thanks so, so much for your help.
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Re: Site has been taken off Google, why?

Postby teeb » Mon May 14, 2012 7:07 pm

Ah crap, so my .htaccess edit didn't actually work on the live site - it says 'infinite redirect loop' so I must have done something wrong. What should I put in?
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Re: Site has been taken off Google, why?

Postby Stephen26 » Mon May 14, 2012 7:12 pm

Remove that:
Code: Select all
Redirect 301 /index.php http://diablo3forum.co.uk

You can't combine it with the other rules. The www to non-www redirect should work fine and is the important one.
I'll have a look at the /index.php to / part tomorrow or maybe you find something on Google.
The guys here http://www.webmasterworld.com/apache/ know that stuff really well by the way.
Last edited by Stephen26 on Mon May 14, 2012 7:14 pm, edited 1 time in total.
I install and configure phpBB-SEO (and other mods) for a fair price and very fast. If you are interested simply send me a private message.
Stephen26
phpBB SEO Team
phpBB SEO Team
 
Posts: 434
Joined: Mon May 03, 2010 7:06 am

Re: Site has been taken off Google, why?

Postby teeb » Mon May 14, 2012 7:14 pm

Stephen26 wrote:Remove that:
Code: Select all
Redirect 301 /index.php http://diablo3forum.co.uk

You can't combine it with the other rules. The www to non-www redirect should work fine and is the important one.
I'll have a look about the /index.php to / part tomorrow or maybe you find something on Google.
The guys here http://www.webmasterworld.com/apache/ know that stuff really well by the way.


Ok, thanks Steven. Let me know if you come up with anything!
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Re: Site has been taken off Google, why?

Postby Stephen26 » Mon May 14, 2012 8:47 pm

The following code works for me:

Code: Select all
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.php\ HTTP/
RewriteRule ^index\.php$ http://%{HTTP_HOST}/ [R=301,L,NC]


Just wanted to add that this has nothing to do with phpBB-SEO (which redirects the forum /index.php to / by default).
It also uses rel-"canonical".
I install and configure phpBB-SEO (and other mods) for a fair price and very fast. If you are interested simply send me a private message.
Stephen26
phpBB SEO Team
phpBB SEO Team
 
Posts: 434
Joined: Mon May 03, 2010 7:06 am

Re: Site has been taken off Google, why?

Postby teeb » Wed May 16, 2012 3:48 pm

Hey again,

Since doing the .htaccess edit my site jumped back onto page 1. So this must be the issue.

However, when using the force canonical code thingy, it also makes any domains pointing at my server use the domain name of the site I'm talking about. So I got an angry call from a customer who runs a very reputable holographics production facility after he found out his site had the URL "diablo3forum.co.uk" :lol:

So what's the workaround for this? I need the other sites to retain their URLs, but also need www.diablo... to redirect to /

Thanks,

Teeb
teeb
 
Posts: 11
Joined: Mon Feb 20, 2012 2:29 pm

Next

Return to Webmaster Tools

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 2 guests