Google sitemap problem

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

Hi,

I use Gsitecrawler to generate my google sitemaps and I created a sitemap that was about 7mb in size, when submitted to Google it found 40,000 url's.

Since then I have created two more new sitemaps, which the newest one is about 7.15mb in size but Google still says there are only 40,000 url's Confused

Should I be worried about this?

Thanks in advance

chez

dcz · Posted: Sun Apr 01, 2007 2:34 pm Post subject: Re: Google sitemap problem

Be careful when building sitemaps this way, remember you don't want to submit duplicate links or good links several times.
This could be the case if the Gsitecrawler would not be set to follow robots.txt or if your robots.txt would not be strict enough. And you could list your phpBB forum twice if you'd crawl it with Gsitecrawler and use GYM sitemaps at the same time.

Now, about the size and the number of urls listed, it depends. Could be the reasons above or just Google needing some time to process 7,15mb of data.

++

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

Thanks dcz,

I'm pretty sure that robots.txt is followed.

Someone said that google only accepts up to 40,000 url's and then does not accept any more, but this cannot be right!

Perhaps I should try something else to make the sitemap for the site, the Gsitecrawler takes about 14 hours as well!

The GYM Sitemaps- does this just build sitemaps for phpbb forums or does it build sitemaps for whole sites?

Thanks

chez

dcz · Posted: Mon Apr 02, 2007 9:02 am Post subject: Re: Google sitemap problem

Well, better be sure about robots.txt exclusions.

About Google sitemaps limit, it's 50 000 per sitemap.
Gym sitemaps only takes car about phpBB listings so far, if you use it together with another sitemap, the other one should not list the forum.

++

ultimatehandyman · PR2 Joined: 15 Mar 2007 Posts: 209

Thanks for that, I have just been reading about the sitemap index file and it seems a bit complicated to me Confused

I will have to study this when I have more time.

Thanks

chez