hi there
i've installed advanced url rewrite, zero duplicate mods for my forum
i was having crawler linker mod installed previously on my forums
but now if i open crawler feeds page & try to open 2nd or anyother page in feeds, it gives me 404 error
please help

here is link to my crawler feeds page :
http://www.forums.projectsplanet.com/crawler_linker.php


if i try to open 2nd, 3rd, 4th.. etc. pages like this :
http://www.forums.projectsplanet.com/cr ... er-500.php

i get error!

i think url rewrite mod also rewriting its urls..

hello... somebody plz help.. plz reply...
may b i think disabling url rewrting for crawler linker link on my index.php solve this..
but i dont know what code i'm suppose to add in .htaccess to do this..
kindly help..

My guess is your code is calling append_sid for the pagination links. That's where the phpBB seo code modifies the returned URLs. Since this page is presumably for search engines, you don't need to call append_sid (search engines don't have sessions).

Why don't you use sitemaps instead? It's faster than pages and pages of links, and they include timestamp information the search engine can use to determine if their index is up to date. Pages of links is soooooo yesterday.

well yeh... i'm also using sitemaps..
but dont know why only google, yahoo, msn bots visited the site.. none other bots visited evah.. dont know why.. & this problem persist since i've started using phpbb3..
well watevah.. all i want is is there any solution to make that crawler linker/feeder mod work?? & also how do i get other bots to visit & index ma forum???

Are you referring to crawl_linker here? It generates links based on your search table. Not sure it's much benefit SEO-wise, but you can add "crawl_linker" to the list of ignored files in phpbb_seo_class.php:

Code: Select all

      $this->seo_stop_files = array("posting", "privmsg", "faq", "groupcp", "memberlist", "login", "search");

Replace with:

Code: Select all

      $this->seo_stop_files = array("posting", "privmsg", "faq", "groupcp", "memberlist", "login", "search", "crawl_linker");

As for getting more bots to visit your forum - my #1 recommendation is create more backlink-worthy content. I wouldn't lose much sleep over it though, from my webstats, Google dwarfs the incoming traffic of all the other search engines. Yahoo brings some, MSN a bit less, and the rest are into single digits. I've seriously considered blocking search engines that don't "put out" enough relative to how much server resources they consume. Yahoo is by far the worse offender; its bots spend much more time on my site than the few visitors it brings!

HB wrote:Are you referring to crawl_linker here? It generates links based on your search table. Not sure it's much benefit SEO-wise, but you can add "crawl_linker" to the list of ignored files in phpbb_seo_class.php:

Code: Select all

      $this->seo_stop_files = array("posting", "privmsg", "faq", "groupcp", "memberlist", "login", "search");

Replace with:

Code: Select all

      $this->seo_stop_files = array("posting", "privmsg", "faq", "groupcp", "memberlist", "login", "search", "crawl_linker");

As for getting more bots to visit your forum - my #1 recommendation is create more backlink-worthy content. I wouldn't lose much sleep over it though, from my webstats, Google dwarfs the incoming traffic of all the other search engines. Yahoo brings some, MSN a bit less, and the rest are into single digits. I've seriously considered blocking search engines that don't "put out" enough relative to how much server resources they consume. Yahoo is by far the worse offender; its bots spend much more time on my site than the few visitors it brings!

thanks a lot man.. it was really helpful
btw i've removed that crawler_linker crap outta ma place! thanks again for helping..