Foswiki hogging CPU? - try the PublicCache

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Foswiki hogging CPU? - try the PublicCache

Colas Nahaboo
(forwarded to foswiki-discuss as it may interest others)

On Tue, Mar 31, 2009 at 10:19 PM, Raymond Lutz <[hidden email]> wrote:
> I have installed Foswiki 1.0.4 and am very pleased [...] I'm still getting
> complaints from my
> hosting ISP because it is over using the CPU on ocassion.

Hi Raymond,

I guess you fit the use case for my "Public Cache".  It is geared
towards public sites with not too many edits (for instance personal
sites), but with potentially huge number of reads, or (small) DOS
attacks.

I have not yet put it properly on Foswiki.org (web pages, subversion.
Should upload it this week-end) but you may try the beta which may
work for you. You can see the description at
http://twiki.org/cgi-bin/view/Plugins/PublicCacheAddOn
I have not adapted it with fastcgi yet.

To try it, grab the code ported to Foswiki at:
http://colaz.net/files/fwpc/

uncompress somewhere; then
  cd fwpc
 ./install your-foswiki-install-dir

To uninstall it:
  ./uninstall your-foswiki-install-dir

--
Colas Nahaboo - http://colas.nahaboo.net

------------------------------------------------------------------------------
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Foswiki hogging CPU? - try the PublicCache

Raymond Lutz
I very much want to use PublicCacheAddOn and am willing to make the changes to eliminate per-user information from the pages. However, I tried installing it and I did not have success in my earlier attempts (with TWiki 4.0.0). I looked back at my notes and I said:

Installed per instructions but could not get it to work. Apparently there is a problem with the default extension being treated as perl and not C.
Also, see my comments as the most recent (bottom) of this page:
http://twiki.org/cgi-bin/view/Plugins/PublicCacheAddOnDev

I agree that this is really essential for people like me who use lots of dynamic code which really does not change its result very often, but eliminates maintenance if I can keep it dynamic. For example, building a dynamic site menu based on the topics on the system amounts to several levels of nested SEARCH macros and if used on the home page become absolutely unworkable. However, PublicCacheAddOn may eliminate much of this concern (I hope).

I also eliminated the bookview option from search.pm since googlebot was using that and consuming too much CPU to build the entire site as a single HTML page, and enhanced the robots.txt file. In no place on the site do I have any bookview option available, so I assume googlebot knows about it from past uses or it is indeed a denial of service attack dressed up to look like googlebot. A simple change to search.pm eliminates the bookview option from ever being selected.

I will try the beta version as you suggest with Foswiki as my problems may have been something related to TWiki, I imagine. I'll let you know how it goes (probably this weekend.)

--Raymond

Colas Nahaboo wrote:
(forwarded to foswiki-discuss as it may interest others)

On Tue, Mar 31, 2009 at 10:19 PM, Raymond Lutz [hidden email] wrote:
  
I have installed Foswiki 1.0.4 and am very pleased [...] I'm still getting
complaints from my
hosting ISP because it is over using the CPU on ocassion.
    

Hi Raymond,

I guess you fit the use case for my "Public Cache".  It is geared
towards public sites with not too many edits (for instance personal
sites), but with potentially huge number of reads, or (small) DOS
attacks.

I have not yet put it properly on Foswiki.org (web pages, subversion.
Should upload it this week-end) but you may try the beta which may
work for you. You can see the description at
http://twiki.org/cgi-bin/view/Plugins/PublicCacheAddOn
I have not adapted it with fastcgi yet.

To try it, grab the code ported to Foswiki at:
http://colaz.net/files/fwpc/

uncompress somewhere; then
  cd fwpc
 ./install your-foswiki-install-dir

To uninstall it:
  ./uninstall your-foswiki-install-dir

  

-- 
---------------------------------------
Raymond Lutz
Cognisys, Inc.                
1010 Old Chase Ave., Bldg B             
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com

------------------------------------------------------------------------------

_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Foswiki hogging CPU? - try the PublicCache

Colas Nahaboo
In reply to this post by Colas Nahaboo
On Fri, Apr 3, 2009 at 7:40 PM, Raymond Lutz <[hidden email]> wrote:
> I very much want to use PublicCacheAddOn and am willing to make the changes
> to eliminate per-user information from the pages. However, I tried
> installing it and I did not have success in my earlier attempts (with TWiki
> 4.0.0). I looked back at my notes and I said:
>
> Installed per instructions but could not get it to work. Apparently there is
> a problem with the default extension being treated as perl and not C.

I guess you are using a perl accelerator (mod_perl, speedy_cgi, ..)
You should tell apache that view is not perl anymore, (and neither the
new scripts pcbd pccl pcad)
see:
http://twiki.org/cgi-bin/view/Plugins/PublicCacheAddOn#Use_with_perl_accelerators

I am sorry but I could not find a way to automate install/uninstall in
these cases... and I feel sorry to have overlooked your comments on
the Dev page, I guess in the turmoil of the fork.

> dynamic site menu based on the topics on the system amounts to several
> levels of nested SEARCH macros and if used on the home page become
> absolutely unworkable. However, PublicCacheAddOn may eliminate much of this
> concern (I hope).

Yes, that is the goal. It is also great for all the RSS feeds that are
also %SEARCH-based, the tags on pages, %INCLUDE on external pages,...
It frees you of the performance issues so you can go overboard with
%SEARCHes everywhere.

>
> I also eliminated the bookview option from search.pm since googlebot was
> using that and consuming too much CPU to build the entire site as a single
> HTML page, and enhanced the robots.txt file. In no place on the site do I
> have any bookview option available, so I assume googlebot knows about it
> from past uses

You should disallow all searches and diff in robots.txt
But Your robots.txt seems OK now anyways

> I will try the beta version as you suggest with Foswiki as my problems may
> have been something related to TWiki, I imagine. I'll let you know how it
> goes (probably this weekend.)

I think it is more related to perl accelerators (do you use one, and
which one?). Let me know of problems.

--
Colas Nahaboo - http://colas.nahaboo.net

------------------------------------------------------------------------------
_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Foswiki hogging CPU? - try the PublicCache

Raymond Lutz
I already looked into the mod_perl question, and my server does not have those installed, according to my question to my hosting service provider about this issue:
Is one of the following enabled:
mod_perl or speedy_cgi or persistent_perl
and they said no. Perhaps there is some test to find out if these (or something of another name) is installed, perhaps some sort of simple test program that will not modify foswiki. I was unable to get it to work from a shell command line.

My thought was that since this was so tenuous, I was going to need to create another foswiki install off to the side so I can test the operation before I attempted to deploy it on the real animal.

--Raymond

Colas Nahaboo wrote:
On Fri, Apr 3, 2009 at 7:40 PM, Raymond Lutz [hidden email] wrote:
  
I very much want to use PublicCacheAddOn and am willing to make the changes
to eliminate per-user information from the pages. However, I tried
installing it and I did not have success in my earlier attempts (with TWiki
4.0.0). I looked back at my notes and I said:

Installed per instructions but could not get it to work. Apparently there is
a problem with the default extension being treated as perl and not C.
    

I guess you are using a perl accelerator (mod_perl, speedy_cgi, ..)
You should tell apache that view is not perl anymore, (and neither the
new scripts pcbd pccl pcad)
see:
http://twiki.org/cgi-bin/view/Plugins/PublicCacheAddOn#Use_with_perl_accelerators

I am sorry but I could not find a way to automate install/uninstall in
these cases... and I feel sorry to have overlooked your comments on
the Dev page, I guess in the turmoil of the fork.

  
dynamic site menu based on the topics on the system amounts to several
levels of nested SEARCH macros and if used on the home page become
absolutely unworkable. However, PublicCacheAddOn may eliminate much of this
concern (I hope).
    

Yes, that is the goal. It is also great for all the RSS feeds that are
also %SEARCH-based, the tags on pages, %INCLUDE on external pages,...
It frees you of the performance issues so you can go overboard with
%SEARCHes everywhere.

  
I also eliminated the bookview option from search.pm since googlebot was
using that and consuming too much CPU to build the entire site as a single
HTML page, and enhanced the robots.txt file. In no place on the site do I
have any bookview option available, so I assume googlebot knows about it
from past uses
    

You should disallow all searches and diff in robots.txt
But Your robots.txt seems OK now anyways

  
I will try the beta version as you suggest with Foswiki as my problems may
have been something related to TWiki, I imagine. I'll let you know how it
goes (probably this weekend.)
    

I think it is more related to perl accelerators (do you use one, and
which one?). Let me know of problems.

  

-- 
---------------------------------------
Raymond Lutz
Cognisys, Inc.                
1010 Old Chase Ave., Bldg B             
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com

------------------------------------------------------------------------------

_______________________________________________
Foswiki-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/foswiki-discuss