| |
|
| :: |
| Author |
Message |
Lucky
Joined: 13 Aug 2006
Posts: 2
|
 Posted: Sun Aug 13, 2006 10:20 am Post subject: Pages not indicized |
|
|
Hi to all,
happy to join in this forum.
I have a website/forum in phpbbxs (a phpbb modded with portal, etc. etc.) that is viewed by bots, the address is www.vivicentro.org
The problem is that if I search in google using the command site:www.vivicentro.org I can see only my home page ... so I have a lot of pages not indicized !!! Why ?
For a better check is important to know that :
1) I have some Back Link with PR 3, but if I type link:www.vivicentro.org nothing appear !!!
2) the rewrite mode enabled correctly on apache server obviously.
3) I have also the meta tags + MOD installed.
4) The website is on line since May, approximately 4 months.
What is the problem ?
If is necessary for a correct check of my website to analyze the .htaccess and robots.txt, you can find a copy of .htaccess typing www.vivicentro.org/ht.txt and obviously www.vivicentro.org/robots.txt for robots.txt file.
Please help me with this problem ...
Thanks. |
|
|
| Back to top |
|
 |
|
|
dcz
Administrateur - Site Admin
Joined: 28 Apr 2006
Posts: 15242
|
| Posted: Sun Aug 13, 2006 11:39 am Post subject: Re: Pages not indicized |
|
|
And Welcome 
I see you are using the good old webmedic mod rewrite.
So for your .htaccess, you should moove all the code after
| Code: | | RewriteRule [.]*chisiamo* chisiamo.php |
at the beginning of it, just after :
It's more efficient than to first redirect once the URL is accepted (mod rewrite rewriterules).
As well, you should add [L] for last at the end of each of the site rewriterules, no need to test topic rewriterules for a forum which already found it's way among the .htaccess (the rule is before, the [L] will just stop any further and useless analysis.
Then your robots.txt seems here to be the one causing us trouble.
Actually you don't need at all to disallow bots in those, because bad bots never follow robots.txt, so no need to bother, and the security rewriterules will take care of that even before. Since those bots are banned in it, they are actually not able to just read your robots.txt 
As well,
| Code: | User-agent: Mediapartners-Google*
Disallow: |
is not the best thing to do if you want Google to be your friend
I' suggest you'd only disallow useless URLs :
| Code: | User-agent: *
Disallow: /post-
Disallow: /updates-topic
Disallow: /stop-updates-topic
Disallow: /viewtopic.php?
Disallow: /viewforum.php?
Disallow: /index.php?
Disallow: /error.php
Disallow: /-br
Disallow: /mark
Disallow: /image-
Disallow: /1-
Disallow: /next
Disallow: /prev
Disallow: /posting.php?
Disallow: /groupcp.php
Disallow: /profile.php?
Disallow: /memberlist.php
Disallow: /search.php
Disallow: /login.php
Disallow: /faq.php |
I see as well you are using the old version of the Google sitemaps mod.
Please check mx Google sitemaps, being a lot more powerful
But was yours submitted and spidered by Google ?
Then, for backlinks, it Take some time for Google to take those into account, it's normal. And you don't seem to have this many, as msn only found 12
and Yahoo 33.
So I'd say fix your robots.txt and wait a bit
++ |
_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________
Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche |
|
| Back to top |
|
|
Lucky
Joined: 13 Aug 2006
Posts: 2
|
| Posted: Sun Aug 13, 2006 12:22 pm Post subject: Re: Pages not indicized |
|
|
Thank You DCZ !!!
| dcz wrote: | And Welcome
I see you are using the good old webmedic mod rewrite. |
In fact !
| dcz wrote: | So for your .htaccess, you should moove all the code after
| Code: | | RewriteRule [.]*chisiamo* chisiamo.php |
at the beginning of it, just after :
|
Something like THIS or like THIS OTHER ?
| dcz wrote: | | It's more efficient than to first redirect once the URL is accepted (mod rewrite rewriterules). |
Sorry I have not understood. could you repeat what you mean ? ... I have typed this code line | Code: | | RewriteRule [.]*chisiamo* chisiamo.php |
because I have before created a new page (chisiamo.php) that doesn't exist inside the phpbbxs package, something like "about us" page, and my intention was to have this page Url Friendly. I don't know if it is the right way but I have created the PHP page and after I have added the code line inside .htaccess file ...
| dcz wrote: | | As well, you should add [L] for last at the end of each of the site rewriterules, no need to test topic rewriterules for a forum which already found it's way among the .htaccess (the rule is before, the [L] will just stop any further and useless analysis. |
Please, could you make an example ?
| dcz wrote: | | Then your robots.txt seems here to be the one causing us trouble. |
| dcz wrote: | Actually you don't need at all to disallow bots in those, because bad bots never follow robots.txt, so no need to bother, and the security rewriterules will take care of that even before. Since those bots are banned in it, they are actually not able to just read your robots.txt
As well,
| Code: | User-agent: Mediapartners-Google*
Disallow: |
is not the best thing to do if you want Google to be your friend
I' suggest you'd only disallow useless URLs :
| Code: | User-agent: *
Disallow: /post-
Disallow: /updates-topic
Disallow: /stop-updates-topic
Disallow: /viewtopic.php?
Disallow: /viewforum.php?
Disallow: /index.php?
Disallow: /error.php
Disallow: /-br
Disallow: /mark
Disallow: /image-
Disallow: /1-
Disallow: /next
Disallow: /prev
Disallow: /posting.php?
Disallow: /groupcp.php
Disallow: /profile.php?
Disallow: /memberlist.php
Disallow: /search.php
Disallow: /login.php
Disallow: /faq.php |
|
Are you suggesting me to use THIS robots file ?
| dcz wrote: | I see as well you are using the old version of the Google sitemaps mod.
Please check mx Google sitemaps, being a lot more powerful
But was yours submitted and spidered by Google ?
Then, for backlinks, it Take some time for Google to take those into account, it's normal. And you don't seem to have this many, as msn only found 12
and Yahoo 33.
So I'd say fix your robots.txt and wait a bit
++ |
|
|
|
| Back to top |
|
|
dcz
Administrateur - Site Admin
Joined: 28 Apr 2006
Posts: 15242
|
| Posted: Sun Aug 13, 2006 2:25 pm Post subject: Re: Pages not indicized |
|
|
Yes for the robots.txt.
And for the .htaccess, here is what I think would be better :
| Code: | Options +FollowSymlinks
RewriteEngine On
#
# HERE YOU JUST PASTE ALL OF THE SECURITY REWRITERULES
#
#make sure the whole site goes to www.mysite.com
#instead of mysite.com. This is good for the search engines
#Edit and uncomment the below lines for your own site.
RewriteCond %{HTTP_HOST} ^vivicentro.org
RewriteRule (.*) http://www.vivicentro.org/$1 [R=301,L]
#this may cause isues with subdirs and so I have not enabled it.
RewriteBase /
RewriteRule [.]*-vf([0-9]*) viewforum.php?%{QUERY_STRING}&f=$1 [L]
RewriteRule [.]*-vp([0-9]*) viewtopic.php?%{QUERY_STRING}&p=$1 [L]
RewriteRule [.]*-vt([0-9]*) viewtopic.php?%{QUERY_STRING}&t=$1 [L]
RewriteRule [.]*-vf([0-9]*) ./viewforum.php?%{QUERY_STRING}&f=$1 [L]
RewriteRule [.]*-vc([0-9]*) forum.php?%{QUERY_STRING}&c=$1 [L]
RewriteRule [.]*-vc([0-9]*) ./forum.php?%{QUERY_STRING}&c=$1 [L]
RewriteRule [.]*-ac([0-9]*) album_cat.php?%{QUERY_STRING}&cat_id=$1 [L]
RewriteRule [.]*-at([0-9]*) album_thumbnail.php?%{QUERY_STRING}&pic_id=$1 [L]
RewriteRule [.]*-apic([0-9]*) album_pic.php?%{QUERY_STRING}&pic_id=$1 [L]
RewriteRule [.]*-apm([0-9]*) album_picm.php?%{QUERY_STRING}&pic_id=$1 [L]
RewriteRule [.]*-full-asp([0-9]*) album_showpage.php?full=&pic_id=$1 [L]
RewriteRule [.]*-asp([0-9]*) album_showpage.php?%{QUERY_STRING}&pic_id=$1 [L]
RewriteRule [.]*-aper([0-9]*) album_personal.php?%{QUERY_STRING}&user_id=$1 [L]
RewriteRule [.]*-dc([0-9]*) dload.php?%{QUERY_STRING}action=category&cat_id=$1 [L]
RewriteRule [.]*-df([0-9]*) dload.php?%{QUERY_STRING}action=file&file_id=$1 [L]
RewriteRule [.]*-kbc([0-9]*) kb.php?%{QUERY_STRING}mode=cat&cat=$1 [L]
RewriteRule [.]*-kba([0-9]*) kb.php?%{QUERY_STRING}mode=article&k=$1 [L]
RewriteRule [.]*-kbsmp kb.php?mode=stats&stats=mostpopular [L]
RewriteRule [.]*-kbstr kb.php?mode=stats&stats=toprated [L]
RewriteRule [.]*-kbsl kb.php?mode=stats&stats=latest [L]
RewriteRule [.]*-pbc([0-9]*) kb.php?%{QUERY_STRING}mode=cat&cat=$1 [L]
RewriteRule [.]*-pa([0-9]*) kb.php?%{QUERY_STRING}mode=article&k=$1 [L]
RewriteRule [.]*-psmp kb.php?mode=stats&stats=mostpopular [L]
RewriteRule [.]*-pstr kb.php?mode=stats&stats=toprated [L]
RewriteRule [.]*-pbsl kb.php?mode=stats&stats=latest [L]
RewriteRule [.]*chisiamo* chisiamo.php [L]
|
This is for the rewriterules part, just add all of your security rules where I told you.
It's logical to proceed this way, because you just don't want banned behaviours to have Apache first loose time interpreting rewriterules before they get denied, it's just better to directly deny them.
And this allows you to add a [L] in the rewriterules, meaning the first rewriterule to match will be the last to be tried. Before, you where interpreting all of them all the time before the request would be filtered by the security rewriterules.
The same apply to the www prefix redirection (I see you installed the 2.31 version I updated ), it's better to do it once security rules ended, meaning we start working on the request, and before the other rewriterules, because this allows you to use the [L] tag in what follows, optimizing all the process.
Then, I did not have time, but I'll update the webmedic mod rewrite once more soon, just to suggest the use of more strict rewriterules and to update the make_url_friendly() function for faster results.
Actually you can directly use the format_url() function's code instead of webmedic's format_url() function, will be quite faster and will allow you to use the short URL feature.
With this one, any part of the title being in between "[" and "]" won't be injected in URLs.
Can be very handy to handle dates or versions for example as "[2.xx]new soft" will be "new-soft" in URLs.
Just download the phpBB SEO adanced mod rewrite pack to grab the format_url() code, and paste it over the make_url_friendly() one in page_header.php.
Then, the rewriterules would be more strict if you'd replace all ocurences of :
with :
Leading the rewrite engine to work faster.
++ |
_________________
Useful links :
SEO Forum || SEO Directory || SEO phpBB || SEO phpBB3 || Search
____________________
Liens Utiles :
Forum référencement || Annuaire référencement || Référencement phpBB || Référencement phpBB3 || Recherche |
|
| Back to top |
|
|
|
| Navigation |
Similar Topics |
|
|
|
|
|
|
|
