google visits and rate of indexing

[AmirAbbas]

hello

i want to ask another question
i have installed all of the SEO mod from this site
i have waited 3 week but only a few page of my forum
are indexed in google database
i've installed robostats script and it shows that google
can see my page and it will come to my forum many times
but only a few number of my pages are indexed in google database
(maybe i'm not patient enough)
how many time i must wait?

my question

direction of my language is right to left but i have used
rtl direction in top of the page and i changed the direction of all
tables and div tags with CSS rules because iranian internet users are get
used to see the scroll bar in right side of the page
can it cause problem in indexing ?

my forum site page is not HTML valid
i have some nonstandard utf-8 character in my page
see this post
http://www.phpbbstyles.com/viewtopic.php?p=77695#77695

can it cause problem in indexing ?

thanks

[dcz]

hello

i want to ask another question
i have installed all of the SEO mod from this site
i have waited 3 week but only a few page of my forum
are indexed in Google database
i've installed robostats script and it shows that google
can see my page and it will come to my forum many times
but only a few number of my pages are indexed in google database
(maybe i'm not patient enough)
how many time i must wait?

Only a few ? 5420 pages indexed is not that few I'd say

You still have URLs with "post" in it, maybe you should check you robots.txt to make sure this duplicate is being taken care of.

And Google is giving you 105 Backlinks, which is quite good since we know Google take a lot more time than other to list them.

Check more things with our SEO webmater tools.
Or are you talking about some other site ?

my forum site page is not HTML valid
i have some nonstandard utf-8 character in my page
see this post
http://www.phpbbstyles.com/viewtopic.php?p=77695#77695

can it cause problem in indexing ?

Well, I don't think a single character could cause so much trouble, did the fix proposed here worked ?

You could basically replace the three dots ('...') with whatever you want there, or just put like 200 as a limit (or get rid of the code limiting it) to make sure no titles will be shorten.

As well W3c validation is something we should all tend to, but there are many cases, especially when dealing with advanced templating (auth, forms and such) where it's very difficult if not impossible.
As long as browsers can display your site correctly, then it's ok.

++

[Peter77]

Wow thats pretty good if your talking about that site. I have just 366 indexed pages and 18 backlinks... even though I know it's more then that. I think it will be another Month or more before I see changes. I have no cache: nor info: for my site yet. Yahoo likes my site better... not by much. just a little more visits then google.

[dcz]

Well,
with Google it can greatly depends, I have seen website, with very few back-links, if not none, needing more than 3 or 4 month just to see their home page indexed and cached by Google.

But I am pretty sure it was due to the fact it was very static with like two updates of content per year and 5 or 6 html pages.

This mean, and it is important for when lunching a new domain, that we need to update frequently our site's content if we want bot to be teased.

And we want backlinks, but not too much, not too fast, you must have heard about the "sandbox"

For example, this domain www.phpbb-seo.com was on line since the 15th of march 2006, but there was basically just one html page accessible, while I started to test the script further online privately.

It was publicly open more or less the day the first member wanted to try on of the pre released phpBB rewrite mod, the 29th of April. But I fully oppened it on the 15th of may (let the mx index show up directly).

I really think the one month and a half sleeping was quite helpful to avoid sandbox

And, I am still not using mx Google Sitemaps, because this is a good occasion to see how much mod rewrite is helpful as it is
I will soon though, I just would like the design to be done a bot more and to be a bit more ready with releases before I start active Search Engine Optimization.

Peter77, maybe you should work on posting some more links to your site, I'll soon open the phpBB SEO Web directories

++

[AmirAbbas]

Only a few ? 5420 pages indexed is not that few I'd say

yes its correct but i had over 5000 page indexed page before i install simple rewrite mod

near half of that URLs are in standard phpbb URLs with .php suffix
and other urls are in able2know rewrite mod form (that many of them are duplicated)
only 100 pages are indexed after simple rewrite mod installation

You still have URLs with "post" in it, maybe you should check you robots.txt to make sure this duplicate is being taken care of.

thank you for remind this issue. i forgot to put /post to my robots.txt file

Or are you talking about some other site ?

im speak about my site. but the site of my friend is like me
his forum was not public for guests. at this time he has 200 page indexed after installing simple rewrite mod

Well, I don't think a single character could cause so much trouble, did the fix proposed here worked ?

You could basically replace the three dots ('...') with whatever you want there, or just put like 200 as a limit (or get rid of the code limiting it) to make sure no titles will be shorten.

the fix didn't work
each UTF-8 character is equal with 2 ISO-8859-1 character
i think this mod some times break a character form middle and make a unusual character.
i removed the limitation and at this time i can validate my page
now there is only 11 error in my page according to W3C html validation report

Well,
with Google it can greatly depends, I have seen website, with very few back-links, if not none, needing more than 3 or 4 month just to see their home page indexed and cached by Google.

its completely true
my site was one of them

I really think the one month and a half sleeping was quite helpful to avoid sandbox

whats the meaning od this sentence ?

direction of my language is right to left but i have used
rtl direction in top of the page and i changed the direction of all
tables and div tags with CSS rules because iranian internet users are get
used to see the scroll bar in right side of the page
can it cause problem in indexing ?

what about this ?

another thing
at the overall_header file

Code: Select all

<title>{SITENAME} :: {PAGE_TITLE}</title>

i changed this part to this form

Code: Select all

<title>{PAGE_TITLE} :: {SITENAME}</title>

which form is better ?

and see this meta tags

Code: Select all

<meta name="category" content="general">
<meta name="robots" content="index, follow">

W3C validator shows that this > ending is not valid for Xhtml 1.0
the ending tags must be in this /> form
howcan i fix it (there are the meta tags that generated by dynamic metatag generator mod






at last i want to say that i don't want to complain about simple rewrite mod and other SEO mod
i know that these mod is very useful and you are working on them to making them more effective

[dcz]

yes its correct but i had over 5000 page indexed page before i install simple rewrite mod

near half of that URLs are in standard phpbb URLs with .php suffix
and other urls are in able2know rewrite mod form (that many of them are duplicated)
only 100 pages are indexed after simple rewrite mod installation

Well, you should tell make sure no duplicate URLs are being listed anymore now, submitting your robots.txt to Google, at least with the basic phpBB SEO simple mod Rewrite recommended disallows, since we put redirection to move from the able2know url to the phpBB seo ones, no need to put more.

Remember this thread ?
This way Google will only spider useful URLs (or get http 301 redirected when hitting an old able2know one)
I'll soon write general articles to cover all of those subjects, I am currently making sure the phpBB SEO mods are well documented and such.
Will as well upgrade soon all of the phpBB SEO mod (minor things don't worry, no bug fix) so that all the support process can start correctly here, and add some more .

the fix didn't work
each UTF-8 character is equal with 2 ISO-8859-1 character
i think this mod some times break a character form middle and make a unusual character.
i removed the limitation and at this time i can validate my page
now there is only 11 error in my page according to W3C html validation report

Have you tried to use an utf-8 character as a break character ?
Anyway, you can just take away the limiting code to solve the issue.

I really think the one month and a half sleeping was quite helpful to avoid sandbox

whats the meaning od this sentence ?

Well, sandbox is a Search Engine Optimization concept. It's known that Google uses some filters when dealing with new domains.
It's meant not to change to much and too fast the search results. When for example a site would start with already 10 000 backlinks, Google would certainly put it in the so called "sandbox".
This last usually about 6 month, and your site, while being spidered is most likely not to show up in the search results, at least at a decent position.
This way, site whiling to SEO a bit too much will not be able to change the search results too fast.

Actually Google has its own idea of what is a rather normal growth pattern (pages & backlinks) for new websites, the idea is to stay in the "official" path.
A web site with too many backlinks obtained too fast, as well as one with not enough or not fast enough (or even if your site stops having more and more Backlinks) will have trouble in Google.

And we know it won't be long, if not already the case, until MSN Search and Yahoo will do something similar.

The idea to Search Engine Optimize the best possible is to maintain a constant growing rate, pages + backlinks, and to never stop

what about this ?

another thing
at the overall_header file

Code: Select all

<title>{SITENAME} :: {PAGE_TITLE}</title>

i changed this part to this form

Code: Select all

<title>{PAGE_TITLE} :: {SITENAME}</title>

which form is better ?

and see this meta tags

Code: Select all

<meta name="category" content="general">
<meta name="robots" content="index, follow">

W3C validator shows that this > ending is not valid for Xhtml 1.0
the ending tags must be in this /> form
howcan i fix it (there are the meta tags that generated by dynamic metatag generator mod

In page_header.php, in plain text, will fix it

Then :

Code: Select all

<title>{PAGE_TITLE}</title>

is better

I'll soon release the phpBB SEO title enhancement mod

at last i want to say that i don't want to complain about simple rewrite mod and other SEO mod
i know that these mod is very useful and you are working on them to making them more effective

They will be more efficient, even though they are already.

IMHO the full Toolkit will be the most efficient ever written for phpBB

Anyway fell free to ask for any thing you would like to know before I add more articles.

++

[AmirAbbas]

Only a few ? 5420 pages indexed is not that few I'd say

now its 5900 records

i find out that google come to your site many time
the statistic shows it visits my forum at least 300 times per day
it seems that googlebot will add your page to database after a priod of time
my forum url shows that these link are not new
there is lot of post with word "post" in url
these links are duplicated as you said

i add /post to my robot test two days ago and google added this new links from yesterday until now

[dcz]

Then Google is really indexing your site.

And it is totally normal that it does not add new visited page (or just new urls, it's the same) the first time it spiders them.

Usually, Google either come on or two time prior to cache new pages or at least wait couple days before including them.

I am pretty sure the number of indexed page in Google for your web site will increase fast now

[Peter77]

I guess I'll share something similiar that i've done to my board as well.

Code: Select all

Replace line...

<title>{SITENAME} :: {PAGE_TITLE}</title>
with line...

<title>{PAGE_TITLE}</title>

Replace the index page "Index" text with site name or keyword text...
Edit file 'language/lang_english/lang_main.php'

Replace line...

$lang['Index'] = 'Index';
with line...

$lang['Index'] = 'Your-site-name Forums or keyword text';

Remove "View Forum - " text...
Edit file 'viewforum.php'

Replace line...
$page_title = $lang['View_forum'] . ' - ' . $forum_row['forum_name'];
with line...

$page_title = $forum_row['forum_name'];

Remove "View topic - " text...
Edit file 'viewtopic.php'

Replace line...
$page_title = $lang['View_topic'] .' - ' . $topic_title;
with line...

$page_title = $topic_title;

Very helpful

dcz, Thank you, I look forward to phpbb-seo web directories. I have been linking with other sites... pretty good quality sites with higher page ranks then mine. also forgot to make clear, that without "www" my site currently has 6,030 backlinks. hopefully I will gain all those back with "www". also things seem to be moving with Google... slowly but it's moving. I now have 836 indexed pages since I posted on this thread.


::edit:: oops, I ment to say 836 indexed.

[AmirAbbas]

now its 5900 records

today its reched to 9480 records in google database

very good

[dcz]

dcz, Thank you, I look forward to linking to phpbb-seo.com. I have been linking with other sites... pretty good quality sites with higher page ranks then mine. also forgot to make clear, that without "www" my site currently has 6,030 backlinks. hopefully I will gain all those back with "www". also things seem to be moving with Google... slowly but it's moving. I now have 836 indexed pages since I posted on this thread.


::edit:: oops, I ment to say 836 indexed.

Yes, the title enhancement you propose is the kind of thing to do with phpBB, I'll soon issue the phpBB SEO version, will be pretty similar but I'll add some other small thing to make it a full link toolkit for phpBB.

Anyway, to come back to the topic, this is the Google Sitemaps effect ( ), since Google can list many URL very fast, and figure out how big is the site, they can organize visits, and page caching accordingly.
The bigger the site is the most bots visits will be made and in the end, we see that Google can spider many many pages very fast with this sitemaps system.
Actually, on the mxBB web site, they got indexed using the mod without even getting rid of the SIDs (since the link provided don't have any, Google spider them without ), same as for Elgobo, from almost nothing to much more in about a month

Then, they will not necessarily all stay cached, actually the Google cached pages list is something that changes a bit, but not that much. It's like a cycle.

In the end with Google, time is very important.

A new domain need time to get Ranked, linked, and known.

Talking about your backlinks for your site, Yahoo is giving you 956, altavista 925, MSN 178 and Google 18. Have you checked ?

So you can see here that Google takes more time to take links into account than the other ones listed here.

But this does not mean that Google did not spider the pages other bots found, it's just telling us that Google is actually using additional parameter to filter backlinks. It's showing us how they fight against spam indexing, Backlinks need to last to be taken int account, and better if they are all on different domains.

That's where phpBB SEO can help quite a bit, the link in your signature and profile will be counted as Baclink with a bit of time, other links will be soon possible in the phpBB SEO directories, and there are two, the French one can be good to use as well for English websites (with a French description of course) as it will come from another domain.

Basically, when phpBB SEO will really start to be fully opened, all domains and sub domains, users will be able to have link backs from their posts (in both forums if they speak English and french ), from the two directories (The translation forum will be used to help out user to translate their site descriptions as well, and random link list will be soon shown on the main domain as well ) and, even more if they write articles published in the phpBB SEO documentation areas. And, articles will be always translated, the author will thus always have a link from both the French and English seo documentation areas.

So you see that phpBB SEO will give very good link backs (as we are SEOed ) from up to 1 domain and 6 sub domains so far. Actually, everything is been though to give back such quality backlinks to our dear users

That's why the spam and Signature part of the TOS are is this precise, because for these links to be good backlinks for user's web-site, we'll need to fight hard against spammers.
Because I am pretty sure this site will soon be very attractive for them, but this is worth it, as it again fulfils the same goal, [url]giving while receiving[/url].
Users are helping this site's PageRank with the links they provide, and this site, by having a good PageRank will give back very good Backlinks to users.

Again, our user's web-sites, by taking advantage of these links, will get better Ranked, and thus provide even better Backlinks to phpBB SEO, which will then give back better ones too. See the idea? I really think we can all enjoy this

now its 5900 records

today its reched to 9480 records in google database

very good

So you start to see the phpBB SEO method in action

[Peter77]

Yes, the title enhancement you propose is the kind of thing to do with phpBB, I'll soon issue the phpBB SEO version, will be pretty similar but I'll add some other small thing to make it a full link toolkit for phpBB.

Cool, I look forward to this

Anyway, to come back to the topic, this is the Google Sitemaps effect ( ), since Google can list many URL very fast, and figure out how big is the site, they can organize visits, and page caching accordingly.
The bigger the site is the most bots visits will be made and in the end, we see that Google can spider many many pages very fast with this sitemaps system.
Actually, on the mxBB web site, they got indexed using the mod without even getting rid of the SIDs (since the link provided don't have any, Google spider them without ), same as for Elgobo, from almost nothing to much more in about a month

Then, they will not necessarily all stay cached, actually the Google cached pages list is something that changes a bit, but not that much. It's like a cycle.

In the end with Google, time is very important.

A new domain need time to get Ranked, linked, and known.

Talking about your backlinks for your site, Yahoo is giving you 956, altavista 925, MSN 178 and Google 18. Have you checked ?


So you can see here that Google takes more time to take links into account than the other ones listed here.

But this does not mean that Google did not spider the pages other bots found, it's just telling us that Google is actually using additional parameter to filter backlinks. It's showing us how they fight against spam indexing, Backlinks need to last to be taken int account, and better if they are all on different domains.

That's where phpBB SEO can help quite a bit, the link in your signature and profile will be counted as Baclink with a bit of time, other links will be soon possible in the phpBB SEO directories, and there are two, the French one can be good to use as well for English websites (with a French description of course) as it will come from another domain.

Yup, i've checked with that tool. doing pretty well with other search engines... latley my focus has been on Google. getting ranked with Alexa seems to be a dificult task as well. I submited my site to DEMOZ about a month ago... still waiting on confermation.

Basically, when phpBB SEO will really start to be fully opened, all domains and sub domains, users will be able to have link backs from their posts (in both forums if they speak English and french ), from the two directories (The translation forum will be used to help out user to translate their site descriptions as well, and random link list will be soon shown on the main domain as well ) and, even more if they write articles published in the phpBB SEO documentation areas. And, articles will be always translated, the author will thus always have a link from both the French and English seo documentation areas.

So you see that phpBB SEO will give very good link backs (as we are SEOed ) from up to 1 domain and 6 sub domains so far. Actually, everything is been though to give back such quality backlinks to our dear users

That's why the spam and Signature part of the TOS are is this precise, because for these links to be good backlinks for user's web-site, we'll need to fight hard against spammers.
Because I am pretty sure this site will soon be very attractive for them, but this is worth it, as it again fulfils the same goal, [url]giving while receiving[/url].
Users are helping this site's PageRank with the links they provide, and this site, by having a good PageRank will give back very good Backlinks to users.

Again, our user's web-sites, by taking advantage of these links, will get better Ranked, and thus provide even better Backlinks to phpBB SEO, which will then give back better ones too. See the idea? I really think we can all enjoy this

Have you thought about installing a simple MOD that requires members to post a minimum set of posts before enabling signatures and website links in profiles? it would be a good idea agianst spammers. anyway, that is very good imformation you gave there, thanks for explaining why Google has taken so long to list my sites backlinks. I always wonderd if links in signatures counted as anything. latley I've seen back links to some forums I've posted in, I just didn't know it could help for PR.

[dcz]

Have you thought about installing a simple MOD that requires members to post a minimum set of posts before enabling signatures and website links in profiles? it would be a good idea against spammers. anyway, that is very good information you gave there, thanks for explaining why Google has taken so long to list my sites backlinks. I always wonderd if links in signatures counted as anything. lately I've seen back links to some forums I've posted in, I just didn't know it could help for PR.

Actually yes, it will be done, I just thought the first users could just obtain more links faster, as a kind of "waiting until this site will be 100 time bigger" welcome package

And of course, the signature links are taken into account as Backlinks.
Actually, I was surprised with my first web-site to see the effects of my first post in a large and well ranked board. Was spidered totally from this unique link and quite fast.
I know of other examples where more links, but not enough to be sandboxed (was like 10 links) where posted but in rather not well Ranked sites, and ending up waiting 6 month for just his home page to be cached

Again, we'll do even better here, because links coming from several domains will be a lot better taken in account than if they'd all been posted on the same unique one.

100 links on one domain, will certainly not end up to be considered as 100 backlinks by Google, 100 links coming from 100 different domains are a lot more likely to be considered as a 100 backlinks, at least with some time.

But of course, the main interest of phpBB SEO is and will be to provide working SEO solutions for all and to have good times while learning and sharing knowledge.
That's what I like about SEO, once the basics will be clearly covered here, we'll start talking about practical examples and will start further experiments. This will need many skills and creativity, so that code will appear less important and everybody will be able to imagine a nice strategy for his web-site.
Entering more into detail will make us cover more subjects.
After all this all about content and every content will need specific approaches, as far as Search Engine Optimization.


And for alexa, phpBB SEO first in alexa from where I stand
Actually these are the MSN Search Results, look at the cache in alexa's result list.

++

[AmirAbbas]

hello DCZ
could you remember, you said that indexing process is a time consuming process and it will take
near 2 month to see the effect of SEO mod on your forum ?

i think something is wrong in my friend forum
after 2 month only 200 page of his forum indexed in google
all SEO mod is installed

simple rewrite mod
remove quest session mod
google sitemap
sitemap module
dynamic meta tag mod

i think something is wrong with dynamic meta tag mod
after installing this mod the rate of indexing reduced dramatically

i had installed simple rewrite mod on my forum
i had 5000 page indexed and in a one google dance it reached to 9000 url
but after installing dynamic meta tag indexing process stoped
i have installed dynamic meta tag on my friend forum at this two address form 2 month ago

-www.siscenter.com
-www.kowsarr.com/forum

this forum using arabic charset for persian language (we can use UTF-8 and arabic charset but standard charse for persian is UTF-8 )

can this mod cause this problem ?
its just my guest.

[dcz]

www.siscenter.com did not installed the robots.txt and www.kowsarr.com/forum did not disallowed the posts URLs, so this is the reason I think.

Because then, Google sees a lot of duplicates and do not like your forum as it could and should.

Then, talking about your site, how many page do you think you have, 9000 is a lot with no dupes (with the correct robots.txt).

That's why reports can help a lot here with phpBB forums, because it's possible to compare more accurately.

Then, Talking about the meta Tags, I don't think they do anything else but better than nothing

I'll soon release a title mod for phpBB forum that will allow us to go even further, you'll see

++