duplicate VS HTTP 301

[hopster]

Something I did not realize until right now, and I worry about as far as changing the forum names:

With the mod rewrite addition to the forums, any description before the -vfX.html works and directs to the same forum reguardless of what's in the text before....


So,

buggy-bash-2007-information-and-announcements-vf46.html

works at the same times as

buggynews-buggy-bash-2007-information-and-announcements-vf46.html

or even

blah-blah-blah-and-other-stuff-buggy-bash-2007-information-and-announcements-vf46.html

Which could seem to lead towards dupe content issues if I change the forum names, or even if someone on the net has a typo in a forum link and posts it elsewhere! This would actually give folks a tool to bomb your site and post false links hence leading to massive dupe content issues I would think.

I have adjusted the forum names in the past, and maybe that something that I'm running into now?? No idea.


-Steve

[dcz]

Well not really, the zero duplicate makes your url canonical, hard as a rock.
All the possible duplicate you mention will be http 301 redirected to the correct url.
Actually, the good point with this is that old links posted in topics will still work and matter SEO wise if you edit your forum title and thus change its url.
User and bot following it will still see the correct page with the correct url, and bots will acknowledge the older one should not be used any more.

http://www.buggynews.com/is-this-a-dupe-vf6.html ?

Nope, as long as the headers are correct (HTTP 301), which here is the case.

++

[hopster]

I've tested some more with this, and you can tack on anything you want in the beginning of a section or forum tittle and it will stick, but if you change the end or middle of the section or forum tittle it 301 directs to the correct page.

Seems this would be a bug of sorts with mod re-write and a security issue as far as site ranking goes with google dupe content.

On this very forum google-forums-vf12/ works the same as darn-google-forums-vf12/

Alter something in the middle or end... try to change it to google-darn-forums-vf12/ and it redirects to the correct page heading just fine, back to google-forums-vf12/

Give it a shot and you'll see what I mean... I didn't post the full links because I didn't want to cause you an issue here.


-Steve

[hopster]

This becomes an issue if you want to shorten the beginning of a tittle and google still has your longer version:

I wanted to take "buggynews" off of the beginning of a forum tittle, but since the shorter tittle can have things added to the front of it and still work, that's not a good idea. The tittle description with any additional words at all inserted in front of it will still work.

I was trying to go from:

/buggynews-buggy-bash-2007-information-and-announcements-vf46.html

to

/buggy-bash-2007-information-and-announcements-vf46.html

And noticed the issue that the former link (the link that google has) still worked without redirection.

Then I found that

/blahblahblahblahbuggy-bash-2007-information-and-announcements-vf46.html

Works as well.

-Steve

[hopster]

Maybe we should split this topic off and put it in the mod rewrite section? Although it does have particular interest with google since google's penalties for dupe content can be quite severe.

I worry about others that may not realize this quirk, or that could actually be attacked by a competitor, or other malicious person that knows this little issue, including this very site.

I would suspect that this would just be a rather minor tweak of the code to fix this problem?

No 301 redirection or other type when the front of the description has additional text inserted into it.


-Steve

[dcz]

Well, again, this was true before the zero duplicate, all the url you mention do work, you see the page loaded, but, they are all HTTP 301 redirected, meaning, they are all but duplicates, they just do not exist more than the time required to get redirected (ms).

Haven't you noticed the url shown in your browser while trying an url such as : http://www.buggynews.com/blahblahblahblahbuggy-bash-2007-information-and-announcements-vf46.html ?

I mean, I have no fear at all to post links like this here : http://www.phpbb-seo.com/boards/ho-but-what-a-duplicate-vf45/ it's even still a backlink Google can add to his count.

Of course, When you change url, the redirecting is not taken into account live by SE's, they need time to crawl the old url again, and to update it to the new one, but it's genrally a matter of few weeks for several thousand of urls.

On phpBB SEO we changed all url for both forum, now do you see any duplicates here http://www.google.com/search?q=site%3Awww.phpbb-seo.com ?

++

[HB]

It took several months for all my site's URLs to be remapped in Google's index, especially those in the Supplmental Results index. An easy test is to search site:yoursite.com viewtopic. Since my site has nothing to do with phpBB, that's not represented in the forum posts, so any hit was an "old" URL. Now the returned results is less than 50 and are all outbound links to other phpBB-based sites.

As an aside, confirm that your robots.txt is correctly blocking search.php, profile.php, etc. and then go to the "Remove URL" tab in Google's webmaster sitemap tool to explicitly remove them. They're processed in a couple days and makes your site look less "php like". I think that blocking guest access to memberlist.php and making the URLs less phpBB spam-bot friendly can't hurt in the fight against spammers.

[hopster]

That's strange... it wasn't changing the URL in my browser earlier. Wonder if it was a cache issue in the browser?

Let me play some more, I'm on a different system now that may have a different browswer setup.


-Steve

[hopster]

Okay... I see the behavior I was talking about, but it happens when you type directly into the browser bar.

Add anything you want into the browswer address bar previous to the first word in the forum heading, and it comes up with the page you were just looking at... but it seems if it's a link from a page it redirects fine... STRANGE.

The strange thing, is if you enter in words into the address bar anywhere after the first word in the tittle of the forum or section it redirects to the original tittle fine.

Wierd.

So I assumed the address bar would work the same as placing a link somewhere (why wouldn't it??)

I feel better now. Still a little odd that it allows certain modifications, but it seems safe and not as bad as I thought it was.


-Steve

[hopster]

This is a test because I still am finding an issue

http://www.buggynews.com/buggynews-buggy-bash-2007-information-and-announcements-vf46.html

The above link should now redirect to:

http://www.buggynews.com/buggy-bash-2007-information-and-announcements-vf46.html

But for me, it is not.

I checked via the link on the google search results.


-Steve

[hopster]

DCZ-


Well, the next day on different computer, since I again shortened the forum name all the links are no longer re-directing.

Is it something about the cache on the board? Does that need to be cleared or something?

http://www.buggynews.com/blahblahblahblahbuggy-bash-2007-information-and-announcements-vf46.html

No longer re-directs to the correct page, and instead goes to it's own page which worries me.


Check it out, it's not working correctly for me.

When you had tested my forum before, I had already put the forum name back to original because I was worried, and then things appeared to be safe.


-Steve

[hopster]

[hopster]

I tried a 3rd computer, and it does in fact appear to be broken after shortening the forum description.

Above 3 different links up above now all go to the same page and appear as dupe content without redirection.


-Steve

[dcz]

You're right, if you keep exactly the uri, but you add a bit of text before the zero dupe won't redirect, that's pretty much the only case, and you will admit not really suppose to occur too often.

Anyway, this won't be the case with next version, will basically a back-porting of the phpBB3 one, which, with the strict mode and full URL check up (not only the uri), will redirect 100% of the possible duplicates.

Then, this does not change anything to the fact renaming some of your forum will be harmless, if not better with better keywords.

++

[hopster]

Right... so this becomes an issue whenever someone shortens the beginning of their URL... In this case, that was what I wanted to do. I'll have to move it back now, as I have 2 dupe URLS happening here in this thread right now. If I put the original wording back they should redirect, I believe. Strange.

I'm worried that this has happened to me over the past 6 months or so, as I've changed some forum tittles a few times, and I have no idea what I changed from after all this time, so I can't really double check to find the errors.

Google won't forget those links though, and count them as dupes.

I do admit that this won't happen too often, but it can happen and folks should know about it, because if they did what I just tried to do in a systematic way across their site, it could really hurt them.

I never would have thought that removing the first part of the URL would cause a dupe content issue, but it does.

A perfect example is where some webmaster put a forum "section' description in the front of the URL of each forum name... they do this to make it more obvious to the reader where they are posting... such as:

"mini buggy: general discussion"
"mini buggy: parts"
"mini buggy: modifications"
"super buggy: general discussion"
"supper buggy: parts"
"supper buggy: modfications"

Doing this helps the mods out a lot in moving threads to the correct forum sections, etc... As PHPBB doesn't give you the "section names" but only the forum names so it can be quite confusing for an admin or moderator to keep track of where to put the threads on complex boards.

In this case if an admin wanted to remove the section discriptions out of the forum names he would instantly have dupe content across his site because he chopped off the front of a URL in order to remove the section statements.

I understand what you are saying, but it happened to me and I thought I was going crazy when you said it wasn't happening.

-Steve


-Steve