problem in indexing

AmirAbbas · Posted: Mon Nov 27, 2006 4:52 am Post subject: problem in indexing

hi

three month ago i installed most of phpbb-SEO on my friend forum at this address

www.siscenter.com

after three month only a few page of his forum is indexed in google
i think something is wrong with his forum
there is lot of topics (more than 8000 topic) at that forum

i have installed

phpbb SEO advance rewrite mod ( utf-8 )
google sitemap
mx sitemap
zero dupe

my forum is a very new and small forum.
http://forum.persia-cms.com/

my forum has all of phpBB SEO mod and its indexed very well (i think more than 90% of all topic are indexed) Razz

the source code for both forum is the same Rolling Eyes

could you guide me ? i don't know what caused the problem
i hope that this forum is suitable for this topic. if its not suitable for this section, move it to proper place Surprised

thanks

AmirAbbas · Posted: Tue Nov 28, 2006 9:29 am Post subject: Re: problem in indexing

bump

dcz · Posted: Wed Nov 29, 2006 11:45 am Post subject: Re: problem in indexing

All right.

So it looks like some update is going on, because I cannot list the omitted results.

www.siscenter.com has 4 890 URLs listed, but the list ends after 54 results even without the omitted results filter : http://www.google.fr/search?q=site:http://www.siscenter.com/&start=50&sa=N&filter=0

You seem to be allowing profiles indexing, which can make things longer, especially with over 20 000 members (it's 20 000 more URLs with poor content to spider for bots).

Actually I even think this board has reach the size where you should not sow the user online today list any more.

Because, page loading time is not far from being an issue as well it seems, and it's as well an important parameter.
If for example your average page loading time is 2 sec, crawling 20 000 URLs will take 40 000s = 11 hours just waiting to see the content.
As bots to only have a limited time per web site (of course it's taking the size into account, but for a newly indexed webs site, search engine do not through all their bots), you understand lowering page load to 0.5 sec average can help out.

Anyway, I think the first thing to do is disallowing members profile in robots.txt, then take care about page loading time a bit, before it gets to much. Taking of the user online today list can help out a bit. Starting to read this thread too Wink

And its very possible that an update is on it's way.

++

AmirAbbas · Posted: Wed Nov 29, 2006 12:47 pm Post subject: Re: problem in indexing

dcz · Posted: Thu Nov 30, 2006 9:39 am Post subject: Re: problem in indexing

Getting rid of the tpl code won't help much, the list will still be calculated.

The problem is you're querying over 500 users upon each page loads.

The Tweaks for large forum thread is a must to read for every large boards.
It's not a mod nor something you can use before you have spend some time testing around. I know it is a long thread, but take your time to read it, it will be useful in the future anyway, and most of the things told there need some php and sql understanding to be used properly (not to be necessary a geek don't worry).

One simple and efficient one is for if you do only use one style : http://www.phpbb.com/phpBB/viewtopic.php?p=2127995#2127995

This post from drathbun, among others, is a good one : http://www.phpbb.com/phpBB/viewtopic.php?p=2005778#2005778

upgrading server is as well an option.

It's not a real issue right now, but will for sure in the future, 20 000 members is a big board Wink

Testing codes before going online is a must in this case.

++

[EDIT] I forgot to mention a nice clean up of the phpBB search tables could help out quite much as well Wink