I've been running into links on Yahoo search pages, for example, that are showing my forums index.php with SIDs on it. I will even recognize some of the content that was cached from not to long ago.. that is how I know it is not an old link. Is this normal to still show SIDs in index.php? or does it mean something could be wrong in my functions.php? ( cyberalien MOD )

If they only concern the index.php, then they must have been spidered before you implemented the guest session mod to get rid of SIDs.

Make sure you have :

Code: Select all

Disallow: /phpbb/index.php?

In your robots.txt file, that's the main interest of this rule. Now I have observed that Yahoo keeps old URLs in it's listing for quite a while, but they should not remain in cache too long once they are disallowed with the robots.txt.

Then, Google and MSN should get rid of those faster is they ever had them in cache.

++

Okay I will add that. should I do the same for my portal? because in old indexed pages of my site I see
?start=10?sid=c-ect- where does that "start" come from? I know lmsg_start comes form the 'x' next topics from the recently posted. but "start="

This is the last message pagination (the yellow arrows up and down), when the portal is accessed though www.example.com/ and not www.example.com/index.php.

Otherwise the link would have mention index.php.

You can thus add :

Code: Select all

Disallow: /mxbb/?

in your robots.txt

That's it.. I couldn't think of the name.

Okay thank you very much! this should about do it.

Wait a minute... by /mxbb/? what are you saying? my mxbb is on my root domain.. so would I do Disallow: index.php?

Hmm that has to be it... but please correct me if im wrong.


Thanks.

Code: Select all

Disallow: /mxbb/?

is not

Code: Select all

Disallow: /mxbb/

For mxBB you don't want to disallow index.php? because this would disallow mxBB pages, which you don't want.

Here, we are only talking about links such as this one : www.example.com/mxbb/?var=bla&bli=blo

Those can only come from the last msg module so far

The only reason that I think search engines are finding SIDs, is because of on a search engine page I will see text content that I recognize being made recently. then below it there is a URL with SID's on it. could it be that the preview content has nothing to to with the Link? because you've mentioned that, for example, Google holds cached pages from 'x' past months.

Actually I think it is, Google updated links spidered once with SIDs, the disallow in robots.txt should fix this sometime

++