How to Turn non existent posts into 404 errors

[Peter77]

By default our forums will show a "The topic or post you requested does not exist" for a post that is no longer in your forum ( or that was deleted ). last year I was getting annoyed with Google having caches of topics that no longer existed on my site. I thought that this little fix might help my problem and also redirect traffic to my custom 404 page. but not until recently I found a better benefit to this little modification.

Google sitemaps occasionally lists dead URL's ( or posts ) coming from my forum. but in order to Remove dead URLs, the links have to resolve to a true 404 error. otherwise a post that no longer exists will take the Google link removal to phpbb's default "The topic or post you requested does not exist" page and that is not considered a valid 404 page.


So In viewtopic.php

Code: Select all

---- FIND ---- ( two instances  around lines 52 and 160)


message_die(GENERAL_MESSAGE, 'Topic_post_not_exist');


--- REPLACE WITH ---


$header_location = ( @preg_match("/Microsoft|WebSTAR|Xitami/", getenv("SERVER_SOFTWARE")) ) ? "Refresh: 0; URL=" : "Location: ";
header($header_location . 'http://yoursite.com/fake404.html', true);
exit;


--- FIND --- ( around line 192 )


$message = ( !$is_auth['auth_view'] ) ? $lang['Topic_post_not_exist'] : sprintf($lang['Sorry_auth_read'], $is_auth['auth_read_type']);

message_die(GENERAL_MESSAGE, $message);



--- REPLACE WITH ---

if ( !$is_auth['auth_view'] )
{
     $header_location = ( @preg_match("/Microsoft|WebSTAR|Xitami/", getenv("SERVER_SOFTWARE")) ) ? "Refresh: 0; URL=" : "Location: ";
     header($header_location . 'http://yoursite.com/fake404.html', true);
     exit;
}
else
{
     message_die(GENERAL_MESSAGE, sprintf($lang['Sorry_auth_read'], $is_auth['auth_read_type']));
}

Just make sure to point http://yoursite.com/fake404.html into a fake URL from your own domain name so that it comes up with a true 404 error. then you will be able to submit outdated posts to be removed from Google's Index.

Hope this helps anyone who might find this useful.

----

All credit goes to those who contributed toThis thread. and even though there are some differences from the code I provided, it was what best worked for me. feel free to experiment with this and if anyone finds a better solution, please share.

[dcz]

Thanks for this tweak.


Actually, this can be usefull, but 404 may not be the final best answer. I think a good old "410 : the resource is no longer available" would be better here, cause bots do not like to find out too many 404's.
Another way to take advantage of the Back-links to topics that do not exist any more could as well be redirected with a 301 to your home (or whatever) so that they'd still give you some Page Rank

Your tweak would thus become (for the moved permanently version ):

in viewtopic.php :

Code: Select all

---- FIND ---- ( two instances  around lines 52 and 160)


message_die(GENERAL_MESSAGE, 'Topic_post_not_exist');


--- REPLACE WITH ---


header("Status: 301 Moved Permanently", false, 301);
header("Location: http://www.example.com/");
exit();

--- FIND --- ( around line 192 )


$message = ( !$is_auth['auth_view'] ) ? $lang['Topic_post_not_exist'] : sprintf($lang['Sorry_auth_read'], $is_auth['auth_read_type']);

message_die(GENERAL_MESSAGE, $message);



--- REPLACE WITH ---

if ( !$is_auth['auth_view'] )
{
     header("Status: 301 Moved Permanently", false, 301);
     header("Location: http://www.example.com/");
     exit();
}
else
{
     message_die(GENERAL_MESSAGE, sprintf($lang['Sorry_auth_read'], $is_auth['auth_read_type']));
}

Don't forget to change the www.example.com with the url you want to use
For a 410, you could just use :

Code: Select all

header("Status: 410 The resource is no longer available", false, 410);
exit();

Instead.


++

[Peter77]

Hehe, I like the sound of 301 better. it works well, thanks for recommending that. just one thing... works fine with dead viewtopic.php?=123 url's but not with SEO dead links ( or regular oudated links ) instead it redirects to 404. how do we redirect to 301 pages for outdated SEO links? well not spesificly SEO links, but non phpbb viewtoic.php links?

[dcz]

At first, I don't really see why rewriten links could be processed in a different way than the regular one, since php does not know anything about rewritten url.

I have to perform some tests to see if I can experience what you are talking about and then will tell you more.

[Peter77]

Well I was doing some searching about outdated links doing more harm then good and came across this article here. and the author talks about how really 404 pages sends the search bots into a 302 ( The requested resource resides temporarily under a different URI ). and therefore the reason why some of us see a bunch of dead links from our forums on Google pages ( well I do anyway ). could be just a myth.. who knows.

Could we apply the same method to accomplish a 301 with this?

Code: Select all

Step 1
Create a page to display for viewers when they get a 404 error (if you don't already have one). For best usability the page should look like the rest of your site and have the following elements:

- Simple statement that the requested URL was not found on the site,
- List of common URL mistakes for your site (look at your logs for tips)
- Search box so that users can query your site's search engine.

Name the file "404content.php" and upload it to the server.

Step 2
Next we need to create a file to accept and handle the 404 requests. For simplicity we'll name this file 404handler.php and put it in our root directory. The contents of the file should be as follows:

<?php
/* Simple 404 handler */
header("HTTP/1.0 404 Not Found");
include("/path/to/your/404content.php");
?>

How it Works:
The header function sends the 404 response, then include pulls the content of the page you want displayed and feeds it to the browser for viewing.

Step 3
Finally use the Apache ErrorDocument directive in either the servers configuration file (usually httpd.conf), or more commonly in a file called .htaccess in the root directory of the website. It should be as follows:

ErrorDocument 404 /404handler.php

Note: This can also be used for any Error Code by replacing 404 with it's value in the examples above.

Once you have done all of the above, watch your access and error logs for any issues that may arise. Also watch the Googlebot and how it moves around your site.

This should take care of pruning all the outdated links from the search engines and help optimize the way they rank your website.

[dcz]

Well, actually, it is true that some errodocs handling script do not do their jobs, since they redirect to a custom page without the proper header (usually 302 instead of the expected 404 or others).

But this should only happen when your are using some php redirection within your error handling code, because a simple html custom page won't change anything to the header sent by Apache while activating it.

I have to find or built a tool like this here in English, but try sending a 404 link to your site (just put in the dialogue box something like -www.example.com/ksdhkfqsdh ) here and you will find out exactly what is the returned http header.

Here is a sample result for -http://www.phpbb-seo.com/error :

HTTP/1.1 404 Not Found
Date: Wed, 17 May 2006 08:37:28 GMT
Server: Apache/1.3.34 (Unix) mod_gzip/1.3.19.1a PH
P/4.4.2 mod_ssl/2.8.25 OpenSSL/0.9.6m
Connection: close
Content-Type: text/html; charset=iso-8859-1

Anyway, I am working on a page handling script that will do a lot more than just respecting the proper http header. It will take care as well of many security issues, more to come ...

[Peter77]

HTTP/1.1 404Date: Sat, 20 May 2006 22:18:46 GMTServer: Apache Web ServerX-Powered-By: PHP/4.4.1Connection: closeContent-Type: text/html

Yep it's returning a 404 header.


and your 301 script is returning 301 for outdated or deleted viewtpic.php urls

HTTP/1.1 301 Moved PermanentlyDate: Sat, 20 May 2006 22:20:30 GMTServer: Apache Web ServerX-Powered-By: PHP/4.4.1Location: http://www.site.com/index.php?page=2884Connection: closeContent-Type: text/html

Anyway, I am working on a page handling script that will do a lot more than just respecting the proper http header. It will take care as well of many security issues, more to come ...

Sounds good... I look forward to it.

[dcz]

Good

look up here if you want to find out more about what I am currently cooking as a security module