Title : Google Yahoo MSN Sitemaps and RSS

Author : dcz / www.phpBB-SEO.com

Description : This module will create United Sitemaps (Aka Google sitemaps), RSS 2.0 feeds and a nice Yahoo urllist.txt for phpBB and mxBB Portal (In case it's installed).
Uses a full support Gun-zip cache for all types of outputs and some XSL-Transform styling for Sitemaps and RSS feeds.

Version : v1.2.0RC4

Installation Level : Easy.

Installation Time : 2 min.

Compatible EasyMOD : n/a.

phpBB : 2.0.22

mxBB : 2.7.x & 2.8.x

Modular Google Sitemaps (United Sitemaps, RSS 2.0 Feeds and Yahoo urllist.txt solution for phpBB and mxBB Portal.

This module will generate several types of outputs :


• United Sitemaps (Google Sitemaps) :
  

  This mod will create a SitemapIndex, listing all the created Sitemaps.
  They will be usable by the Google, Yahoo and MSN search engines.
  Sitemaps are 100% valid, sitemaps 0.9 protocol.
  They can be styled using XSL-Transform style-sheets (configurable ACP).
  
  WARNING

  Please make sure the Sitemaps are working before you submit the sitemap.php file (or sitemaps.xml with mod Rewrite) to Google Sitemaps and Yahoo.
  MSN submission is not handled the same way yet, but you can still notify the MSN Search engine using the sitemaps.org submit procedure.

  
  Sitemaps Created :
  

  :arrow: One SitemapIndex listing all sitemaps
  One general forum sitemap, listing public forums url
  One sitemap per public forum listing topics.

  When mxBB installed :
  

  :arrow: Adds an entry in the sitemapIndex
  One sitemap listing all public mx pages.

  Note :
  

  Knowledge base (KB) mod is until now not supported by the module. Everything is ready to have it plugged in with ACP options, as soon as a stable KB version will be released.
  

  
  
  
• RSS 2.0 Feeds:
  

  You can as well register you main Feeds at Yahoo, like rss.php (or rss.xml).
  Forums RSS Feeds can notify yahoo automatically using the Yahoo! Notifications API.
  You'd have to apply for a Yahoo AppiD.
  Please note that Yahoo will not allow the use of "&" in submited RSS feeds URLs. This mean you won't be able to submit feeds with "&" in their URL.
  This only concern few feeds, and only if you do not use URL rewriting.
  
  phpBB authorisations:
  

  The module is able to build personalized rss feeds according to the user's authorisations.
  If set to yes, users will be able to browse private forum feeds if they have enough permission to do so.

  
  RSS Feeds Created :
  

  :arrow: One Feed listing the last items from all available sources ( plug-in included )
  One Feed listing the last messages from all the forum
  One Feed per forum
  One special Feed, listing all available feeds.

  
  Each feed is available in three configurable versions :
  

  :arrow: A long one,
  A default one,
  A short one.
  All of these thres options can be combined with a extra one, to additionally add the messages content in the feeds.

  
  Advanced content filtering :
  

  It is possible to filter messages content in feeds :
  
  • Links :
    You may choose here to either activate or not links used in posts.
    If deactivated, links will be outputted as part of the content but won't be clickable.
    
  • BBcodes :
    You can deactivate BBcode parsing or filter them (tags and/or content).
    The format is simple :
    You'll just have to enter a comma separated list of bbcode tags to exclude (tags removing) or delete (full bbcode bloc removed and replaced by {bbcode} in output) with the additional ":" option.
    Example :
    img:1,b:0,quote,code:1
    In this example, img bbcode and the img link will be replaced with {img}, bold won't be processed, but the bold-ed text will be kept, quote won't be parsed, but their content will be kept, code bbcode and their content will be replaced with {code} in the output.
  • Smileys :
    You may choose here to either parse the smileys or not in content.
  • Digest :
    You can select between three different method to limit the outputted messages content in feeds : By sentences, by words and by characters.
    None will break word, select the one you feel more comfortable with.

  
  Multiple Char-set handling :
  

  The module uses UTF-8 as output char-set.
  phpBB3 conversion method is integrated in the module to allow conversion from numerous char-sets.
  You'll thus have to set the char-set used by your forum in the ACP, in case the 'auto' parameter fails (which unfortunately shall occur in several cases).
  You can find out what is your forum's char-set in any of your forum's page source code :
  

  <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">

  
  

  
  
• Yahoo! urllist.txt :
  

  The module creates a Yahoo! urllist.txt, listing last active threads from all the forums, one URL per line.

Common features for all the module :


• Cache :
  

  A complete cache system configurable from ACP.
  All the maps (sitemaps, rss and urllist.txt) are entirely saved in a folder.
  When a cached file is up to date and available, the module will just send it as is to the browser without further processing, making the output very, very fast, comparable to a direct physical file access.
  Each type of cache, corresponding to each type of listings, can be configured separately. You can decide for each one how the cache will be updated : auto (each time the cache expires) or manual (files delete from ACP).
  
  Benchmark : (11834 URLs)
  

  • Without Gunz-zip compression :
    First page load : Cache is being build
    

    <!-- URL list generated in 5.41892 s - 25 sql - 11834 URLs listed -->
    <!-- Output started from cache after 5.42756 s - sql -->
    <!-- Output from cache ended up after 6.93087 s - sql -->

    
    This means that the module is building a 11834 URLs list in 5.41892s, and that the cache file is saved in 0.00864s (2 119 631 octets)(the file being sent to the browser right after saving).
    The output ended 6.93087s after it was requested.
    
    Second load :
    

    <!-- URL list generated in 5.41892 s - 25 sql - 11834 URLs listed -->
    <!-- Output started from cache after 0.00256 s - sql -->
    <!-- Output from cache ended up after 1.57475 s - sql -->

    
    The first line being itself cached, to recall how hard it was to build up such a long list before sending it.
    This is to be compared to the 0.00256 s here needed to start the output
    Then, file transfer is relatively long, but the file is 2mo and if we take into account the large number of URLs it's pretty fast.
    
  • With Gunz-zip :
    Unfortunately, there are no stats available for this output, the function used to read and send a gunzipped file makes it impossible it seems.
    But once activated, our 2mo becomes 48 ko !!
    And as the file is sent to the browser as is, it's only a matter to sen a 48ko file to list 11834 URLs
    So no need to search too long to figure out it is even a lot more faster, with a lot fewer server resources spent.
    A work comparable to sending a 48ko image to list only 11834 URLs.
    



• SQL Cycles and perf settings :
  

  All major queries are separated into several cycles, configurable in acp, for all types of listings.
  Default limit settings should be good for most cases.
  The principle is simple, to list 10 000 URLs we cannot query for 10 000 items at a time, the module will perform several.
  The idea is to limit the number of items per query as well as the number of query themselves.
  You should no go over 30 queries per sitemaps, and if your server is not able to build up a 10 000 URLs list in a reasonable time, no need to go for so much.
  We here would deal with the last 10 000 topic to have received an answer in a given forum, not all forums, so we can conclude topics older than that would be quite old.
  In general, there is no need to go above 5000 URLs per sitemaps.
  
  Pleas not as well that it would be totally useless to list thousands of URLs in RSS feeds, even if the module can do it.
  RSS feeds are not meant to be used as sitemaps.
  



• URL Rewriting :
  

  You can switch mod rewrite type in ACP for all listings, phpBB SEO mod rewrite 0.2.x are auto detected.
  Settings for other mod rewrite standards (Webmedic's rewrite mod, able2know rewrite mod, GoogleBB Links) can now be done easily, simply editing a file.
  


Small limitation :

If Gun-zip is activated in phpBB, it will be so in the module, it is though possible to use gun-zip compression on the module if it's turned off in phpBB.

English, French.
French support thread

• [PlugIn] GYM Sitemaps XML :
  
  Will incorporate easily sitemaps (must be valid Google sitemap) to your GYM Sitemaps Google sitemaps listings.
• [PlugIn] GYM Sitemaps TXT :
  
  To incorporate easily url txt lists (one URL per line) to your GYM Sitemaps Google sitemaps.

• RSS Feeds :
  
  
  • The main feed :
    
    www.phpbb-seo.com/boards/rss.xml
    With messages :
    
    www.phpbb-seo.com/boards/rss-m.xml
    Long list :
    
    www.phpbb-seo.com/boards/rss-l.xml
    Long list with message :
    
    www.phpbb-seo.com/boards/rss-l-m.xml
  • Per forum feeds :
    
    www.phpbb-seo.com/boards/gym-sitemaps-rf44.xml
    www.phpbb-seo.com/boards/gym-sitemaps-rf44-m.xml
    www.phpbb-seo.com/boards/gym-sitemaps-rf44-l-m.xml
  • The channel feed :
    Showing all available feeds, with options.
    
    www.phpbb-seo.com/boards/channels-rss-m.xml
    www.phpbb-seo.com/boards/channels-rss-s-m.xml
    www.phpbb-seo.com/boards/channels-rss-l-m.xml
    www.phpbb-seo.com/boards/channels-rss.xml
• Google Sitemaps :
  
  
  • SitemapIndex :
    
    www.phpbb-seo.com/boards/sitemaps.xml
  • Per forum sitemap :
    
    www.phpbb-seo.com/boards/gym-sitemaps-gf44.xml
  • forums sitemap :
    
    www.phpbb-seo.com/boards/forum-sitemap.xml
• Yahoo! urllist.txt :
  
  
  • The urllist.txt :
    
    www.phpbb-seo.com/boards/urllist.txt

Google Yahoo MSN sitemaps and RSS V1.2.0RC4

dcz wrote:The module is now mxBB2.8 compatible.
Better char-set handling, from now-on, all char-set are selected within the same list (utf-8 included), and mb_internal_encoding() suggestions are better handled.
You can as well select between the several conversion method available to take into account the special config cases where only phpBB3 method would work and we need to force it's use.

Went through all SQL again, did some optimization, added time limits to forum RSS feeds (forum=xx) in case the forum is huge.

Optimised all loops, we're winning some more generation time.

Urllist.php fixed.

Better authorisation and exclusion handling.

Added some links on the xsl style-sheet for our user to easily subscribe our feeds to their Google, Yahoo, msn ... accounts.

And some other small enhancements.

1) Delete all files from the module,
2) Uploade all the new ones,
3) Update the .htaccess with the new rewriterules (no change in url standard, only fixes).
4) Connect yourself as admin and run the sql install script again.