I just received a strange email which appears to have come from Google adsense. I have copied the email below. Has anyone ever had anything similar?

I'm not sure if it is genuine or not


Hello,

While reviewing your ad implementation, we noticed that your robots.txt file is currently preventing our AdSense crawler from reaching a significant number of pages with ads in your account.

In order to serve targeted, paid ads to your sites, our crawler needs to visit your sites’ pages to determine their content. Please update your robots.txt file to allow the AdSense crawler to access all pages showing Google ads. You can allow the AdSense crawler access to your sites by adding the following lines to your robots.txt file:

User-agent: Mediapartners-Google*
Disallow:

Thank you for helping us serve the most relevant ads to your sites. Please note that in the future, if we can't crawl some of your pages, we may disable ad serving to those pages.

Has anyone any thoughts on this?

Looks like adsense is now more strict than before.

Anyway, these urls must be the post urls, could as well be private urls, but since the adsense bot can actually visit the private url, even if it sees the login box there, it could only be post urls.

Pots urls are disallowed in the robots.txt, but should not be used by bots when using the zero duplicate mod.
An in between possibility would be if you set post redirection to "guest", since in such case, registered users can use post urls without being redirected while bots and guest would be.

So you can try to add the bit of code they suggest to in your robots.txt, should be enough, but if not, try to set post redirection to all if you do not really need your registered user to use post urls.

Should be enough, I just hope that google is not actually asking us to stop displaying adsense on private pages.

++

Thanks dcz,

I'll add it to the site asap.

I wasn't sure if it was a genuine email or not and so that is why I asked here.

Thanks

chez