Google Sitemap module update to test

dcz · Last edited by dcz on Thu Nov 16, 2006 12:34 am; edited 2 times in total

Hello,

So here it is, we are going to have soon a beta for the new Google Sitemaps solution for phpBB Forums.
I say new because I think I'll rename it tosomething like Ultimate Google sitemaps & RSS, because yes, there are RSS Feeds, and also, but it's less important, a nice Yahoo! urllist.txt.

Here it goes.

Cache :
A complete cache system configurable from ACP.
All the maps (sitemaps, rss and urllist.txt) are entirely saved in a folder.
When a cached file is up to date and available, the module will just send it as is to the browser without further processing, making the output very, very fast, comparable to a direct physical file access.

Let's talk about time a bit Wink

First page load :

Quote:

This means that the module is building a 11834 URLs list in 5.41892s, and that the cache file is saved in 0.00864s (2 119 631 octets)(the file being sent to the browser right after saving).
The output ended 6.93087s after it was requested.

Second load : And followings until cache expires (configurable in ACP)
And it's more interesting :

Quote:

The first line being itself cached, to recall how hard it was to build up such a long list before sending it. This is to be compared to the 0.00256 s here needed to start the output Very Happy

Then, file transfer is relatively long, but the file is 2mo and if we take into account the large number of URLs and all the work we're asking to the browser, because we do, and I'll talk about this right after, it's pretty fast.

Because of this, Gun-Zip compression is very powerful. The module is both able to save and output Gun-ziped datas. Our 2mo becomes here 48 ko.

As again the cached file is sent as is to the browser, it really become fast to output a Google sitemaps listing 11834 URLs, as fast as sending a 48ko gif file more or less Very Happy

Unfortunately, there are no stats available for this output, the function used to read and send a gun-ziped file makes it impossible it seems. But it is for sure a lot faster, even if the browser is again asked to work even more as it will be the one to uncompress the file which could end up making the page show up a bit later after it was fully sent.

Cache can be updated automatically, after a configurable period of time has passed between two requests on the same file, or manually in ACP.
Each type of page outputted, sitemaps, rss 2.0 feeds and urllist.txt has it's own cache time limit configurable in acp.

URL rewriting :

You can switch mod rewrite type in acp, so far you can select between the three phpBB SEO mod rewrites, but the code is set to allow a lot more URLs standards. Will auto detect the used phpBB SEO mod rewrite when they'll get updated.
Note that title injection does add some weight to the sitemaps. If we continue with our previous example and inject topic title we get :
First request : building cache.

nims

Hey good to see this. Was really waiting for RSS MOD. So will you be coming up with seperate RSS MOD or will this be integrated with Sitemaps only ?

dcz

The module is designed to be highly modular (sic), kind of the way it was before, every type of output is corresponding to an included file.

So, it come with RSS, Yahoo! urllist.txt and Google Sitemaps support, but you can chose to only use one type of output by just not uploading all the files, but rather the one you select to use.

Anyway, IMHO, we all need RSS and Google sitemaps, so I found it handy to provide with both in a single module even easier to install than the the previous mx Google sitemaps Wink

++

nims

Great !!! So when do you suggest I shud switch from earlier Google Sitemap to this one ??? Is it safe to do this. I mean will google ranking etc will not be damaged by this ?

dcz

You'll be able to do so as soon as I publicly release it, eg soon. I am currently working on translating it in English (I wrote this one in French at first) and cleaning up the code (there is some).
Anyway it won't be anything but better than the previous one, will not be an issue to update.

The sitemap url themselves will change if you're using the phpBB SEO mixed or advanced mod rewrite, as there will be title injection in them, but it's no big deal, will just need to re submit the sitemap.

By the way, I added the Yahoo! notification in it Very Happy

Yahoo! will be notified upon each updates of the RSS feeds Wink

++

Peter77

Brilliant... absolutely brilliant, dcz. Very Happy

I like the Yahoo! notification idea also. I look forward for further updates!

nims

Great !!!
I am eagerly waiting for the official release.

kevins · Joined: 16 Nov 2006 Posts: 5

When could we expect the first beta of this one?

Thank you!

dcz

And Welcome Very Happy

Well, as you can see, release has never been closer.

And Good new, Yahoo!, MSN and Google just decided to start using the Google sitemap protocol : http://googleblog.blogspot.com/2006/11/search-engines-united.html

so this mod will even be more useful as it will do the job for all three of them at once now.

I was actually wondering since quite some time why wasn't it the case since Yahoo! and MSN bots where already crawling those, now it's just for sure they'll use them Very Happy

I guess the module is coming at the right time, the cache feature allowing for major linking and usage for these.

Anyway, I uploaded yesterday what is close to last version before a public beta.
I am not sure the total rewriting of the urllist.txt I just did will be this useful any more, but this is on of the difference with the very first version, the module is now able to output thousand of links as well (was no the case in the very fist one) and output paginated links for forums and topics.

Another thing I added is yahoo! notification, another thing made less attractive since the sitemap protocol was shared with Yahoo!, but still, we'll be able to notify yahoo with rss feeds, bringing a bit more information than the Google sitemap is.

Anyway, I am happy you all like this module Wink

++

nims

Good news !!! Now we can just have one common sitemap and that can be used by all the 3 major search engines namely Google, Yahoo and MSN. Wow !!!

So what about this MOD then ??? Wud you be removing yahoo list etc. that r not reqd ?

argenta · Joined: 18 Nov 2006 Posts: 16

aznole · Joined: 19 Nov 2006 Posts: 26

whoops ...wrong forum

dcz

Welcome argenta & aznole Very Happy

So the mod is not available yet, but will soon Wink

This thread is dedicated to present and test the module, you're welcome to tell what you think of how it goes so far.

++

kevins · Joined: 16 Nov 2006 Posts: 5

Oke then we will have wait a bit longer..

Don't you have a realese date or someting like that.

Before December xx Wink

?

dcz

Well this is not really the kind of post that makes it faster.

Maybe you could comment a bit about why are you so impatient, we could find some nice feature to add just before release.

++