Google sitemap index

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

I currently use gsite crawler to spider my site in order to make the google sitemap. It is currently saved as sitemap.xml

The sitemap for my site has stopped at 40,000 url's and will not increase any more then this and so I am going to create two sitemap.xml files and place them in a sitemap index file like this-

dcz · Posted: Sun May 06, 2007 10:01 am Post subject: Re: Google sitemap index

Well yes, a sitemapIndex is just a sitemap listing sitemaps.

Just like http://www.phpbb-seo.com/boards/sitemaps.xml

So as long as you follow the syntax, and list sitemaps in your sitemapindex (not sitemapindexes again for example), it will work nicely and you'll only have to register the sitemapindex at Google. It will spider them all.

++

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

Thanks dcz Wink

I was just a bit concerned and was not sure if deleting the old sitemap first was a good idea! Hopefully I will get around to doing this in the next few days.

I could not find any info on google about how to do this.

Thanks

chez

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

I generated a new sitemap and gsite crawler automatically generated two sitemaps and a sitemap index. I have ftp'd everything to the server, but when I check the sitemap index file it says that there are 0 sitemaps in the index Confused

Have I screwed it up ?

This is the link to the index-

http://www.ultimatehandyman.co.uk/sitemap-index.xml

I have deleted the old sitemap for the site!

Thanks in advance

chez

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

I'm not sure why the link above says there are 0 sitemaps in the sitemap index, but google has accepted it Very Happy

And the number of indexed url's has gone from 40,000 to 49,436 Wink

dcz · Posted: Mon May 07, 2007 11:14 am Post subject: Re: Google sitemap index

Well, I guess it's only because you're attempting to use a sitemap xsl style sheet with a sitemapindex, not using the exact same syntax, because the links really are in it (look up the source code) :

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

Does that mean that I can use gsite crawler to creat the sitemap for my main site and ask it not to crawl the forum and then get GYM sitemap to create a map for the forum?

Thanks

chez

dcz · Posted: Fri May 11, 2007 5:09 pm Post subject: Re: Google sitemap index

Exactly, and it's better, because sitemaps will be updated automatically, and they will not list any duplicate.

I know gsite crawler can be set so it will not spider all links, but I often see duplicates entries in sitemaps created with such tools, mostly because it's hard to think about all filters and to checks huge lists.

Among other benefit, you'll have dynamic lastmod time for each item (topics and forums are sorted by last activity) and nice RSS feeds, all with XSL transforming :

http://www.phpbb-seo.com/boards/sitemaps.xml
http://www.phpbb-seo.com/boards/rss.xml
http://www.phpbb-seo.com/boards/rss-l-m.xml

And the module is fast :

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

That would be great and would save me lots of time Very Happy

Gsitecrawler has been good, but for some reason it always has en error url on line 240 and everytime I create a sitemap with it I have to delete line 240 or Google rejects it.

It also takes it about 12 hours to crawl my site!

dcz · Posted: Fri May 11, 2007 5:41 pm Post subject: Re: Google sitemap index

Even better GYM sitemaps is a module, this mean you won't have to edit a single line of phpBB's code to install it Wink

You'll only need to add few line in your .htaccess to enable mod rewrite for the sitemaps and rss feeds urls, upload the files and run an install script Wink

++

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

I have had a look at the GYM sitemap module, but it looks complicated to me and so I will have to continue using gsitecrawler, until I can either work out how to use it and combine it with Gsitecrawler or get my friend to take a look at it for me.

I have fixed the problem with my desktop computer now and so I can get online more often, but I am busy making some new pages for the site and so I will look at the GYM sitemaps in a few weeks when I have some time to try and understand it properly Wink