Very Strange Problem - Topics not indexed!

Discussion about the Google Search Engine. SEO, PageRank, AdSense, AdWords, Use and other Services.

Moderator: Moderators

Very Strange Problem - Topics not indexed!

Postby Sarr » Tue Jul 02, 2013 5:51 pm

Hello,

I have very strange problem with google indexing.
My site: http://forum.neverwinter.com.pl/ topics are not indexed. What is indexed are almost _only_ news / map / sitemap pages. And even that not very much.
So I recently blocked the pages in robots.txt:
Disallow: /map.php
Disallow: /news.php

Didn't help. I have 500+ links added via -http://forum.neverwinter.com.pl/sitemap.php (in Webmaster tools).
Only 2 are added.

For you information - I'm using 2 mods that cache pages.
One is Cache guests & bots pages, second is HTTP Guest Cache.
Cache guests & bots pages is file cache and lasts for 360 minutes (set for this period).
HTTP Guest Cache adds parameter ?cache=1 to all forum pages for bots and guests. But I don't think that should matter...
This cache should last for 15 minutes.

Here is our robots.txt from today:
Code: Select all
User-agent: *

Disallow: /adm/
Disallow: /cgi-bin/
Disallow: /docs/
Disallow: /files/
Disallow: /includes/
Disallow: /language/
Disallow: /store/
Disallow: /styles/
Disallow: /jquery_base/
Disallow: /mods/
Disallow: /umil/
Disallow: /ajax_base.php
Disallow: /googled1d6e977ab5a052e.html
Disallow: /jquery_base.php
Disallow: /common.php
Disallow: /config.php
Disallow: /cron.php
Disallow: /mcp.php
Disallow: /memberlist.php
Disallow: /posting.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /web.config
Disallow: /search.php
Disallow: /style.php
Disallow: /ucp.php
Disallow: /viewonline.php
Disallow: /map.php
Disallow: /news.php

Disallow: /*.cfg$
Disallow: /*.css$
Disallow: /*.js$

User-agent: Mediapartners-Google
Allow: /

sitemap: http://forum.neverwinter.com.pl/sitemap.php


But now I will use this simple one to test if maybe I blocked something important:
Code: Select all
User-agent: *

Disallow: /adm/
Disallow: /cgi-bin/
Disallow: /docs/
Disallow: /files/
Disallow: /includes/
Disallow: /language/
Disallow: /store/
Disallow: /styles/
Disallow: /jquery_base/
Disallow: /mods/
Disallow: /umil/
Disallow: /ajax_base.php
Disallow: /googled1d6e977ab5a052e.html
Disallow: /jquery_base.php
Disallow: /common.php
Disallow: /config.php
Disallow: /cron.php
Disallow: /mcp.php
Disallow: /memberlist.php
Disallow: /posting.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /web.config
Disallow: /search.php
Disallow: /style.php
Disallow: /ucp.php
Disallow: /viewonline.php
Disallow: /map.php
Disallow: /news.php

Disallow: /*.cfg$
Disallow: /*.css$
Disallow: /*.js$

User-agent: Mediapartners-Google
Allow: /

sitemap: -http://forum.neverwinter.com.pl/sitemap.php


Can you please help me somehow?

I have also this site on different hosting: -http://forum.ddopl.com/
And when I installed phpBB SEO it got mega-indexing boost. So I don't know what's wrong with -http://forum.neverwinter.com.pl/ :(.

My main site: -http://www.neverwinter.com.pl/ is total leader in google.pl for "neverwinter" phrase (and others too). First or second positions on first page.
But my forum is almost non-existant :|.
What could happen?

More information:
I'm using Cloudflare cdn + GoogleSpeed. It works for my www. site, but not for forum. Don't really know why, maybe that's how it should be.

UPDATE 03.07.2013:


I try hard to know what's the problem. And now I checked my site with Bing Webmaster Tools.
It's been a few days and not indexing! Even sitemap is stuck at "Pending":
Code: Select all
http://forum.neverwinter.com.pl/sitemap.php    2013-06-30    275    2013-06-30       Pending

Now, when I try to fetch forums as Bing Bot (option in Diagnostics & Tools), it shows me this:

Code: Select all
URL: http://forum.neverwinter.com.pl/
Status: Redirection limit reached.

HTTP/1.1 302 Moved Temporarily
Cache-Control: private, no-cache, max-age=0
Connection: Keep-Alive
Date: Wed, 03 Jul 2013 08:36:06 GMT
Keep-Alive: timeout=10, max=100000
Pragma: public
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Expires: 0
ETag: 1372855973
Location: http://forum.neverwinter.com.pl/index.php?cache=1
Server: Apache
Set-Cookie: nwo_forum_u=; expires=Tue, 03-Jul-2012 08:36:07 GMT; path=/; domain=.neverwinter.com.pl; HttpOnly
Set-Cookie: nwo_forum_k=; expires=Tue, 03-Jul-2012 08:36:07 GMT; path=/; domain=.neverwinter.com.pl; HttpOnly
Set-Cookie: nwo_forum_sid=; expires=Tue, 03-Jul-2012 08:36:07 GMT; path=/; domain=.neverwinter.com.pl; HttpOnly

What's going on?
It's most drastic with google - not indexing at all, but shows no errors.

I think I found the problem, but don't know how to fix it. Seems it's PhpBB Seo problem:

Looking to headers via this:
-http://urivalet.com/?http://forum.neverwinter.com.pl/#Report
Or via:
-http://web-sniffer.net/?url=http%3A%2F%2Fforum.neverwinter.com.pl%2Findex.php%3Fcache%3D1&gzip=yes&http=1.1
I see robot get "noindex, follow" o.O
<link rel="canonical" href="http://forum.neverwinter.com.pl/?cache=1" />
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
<title>Polskie Forum Neverwinter!</title>
<meta http-equiv="content-style-type" content="text/css" />
<meta http-equiv="content-language" content="pl-PL" />
<meta http-equiv="imagetoolbar" content="no" />
<meta name="content-language" content="pl" />
<meta name="title" content="Polskie Forum Neverwinter!" />
<meta name="description" content="Polskie Forum Neverwinter! : forum.neverwinter.com.pl" />
<meta name="keywords" content="neverwinter, forum, polskie" />
<meta name="category" content="general" />
<meta name="robots" content="noindex,follow" />
<meta name="distribution" content="global" />
<meta name="resource-type" content="document" />
<meta name="copyright" content="forum.neverwinter.com.pl" />

While if you log in and check (or even when normal user looks in code of page) it's "index, follow".
So it's "index, follow" for users, and "noindex, follow" for robots. How can I fix it?

EDIT 03.07.2013 (Newest) - TEMPORARY SOLUTION:
I've found that "Cache HTTP" Mod is setting all pages to "noindex, follow". If I turn it off, then it's "index, follow" again (and pages lose ?cache=1 parameter, since browser cache is turned off).
Now that's something strange, not sure if it's fault of the Mod or some conflict with phpBB SEO.
Sarr
 
Posts: 7
Joined: Sun May 23, 2010 12:13 pm
Location: Poland

Advertisement

Re: Very Strange Problem - Topics not indexed!

Postby NickJemis » Mon Mar 31, 2014 5:38 pm

I have almost the same problem. The difference is only the meta name robots are not written at all! Could it be the problem of bad indexation? Out of 6-8 pages I have only 1 in the main index :( The site is casino.imcredo.com - could anyone tell what else (if not the code) can be the reason of not-indexing? Would be very grateful for any help!
NickJemis
 
Posts: 1
Joined: Mon Mar 31, 2014 5:14 pm
Location: Belarus


Return to Google Forums

 


  • Related topics
    Replies
    Views
    Last post

Who is online

Users browsing this forum: No registered users and 5 guests