High CPU usage
Added by Ryan Phung about 14 years ago
Hi everyone,
I have been using Redmine for half a year. At the moment we have close to 30 projects and 150+ users.
Since last week, one group of our users start to make use of Redmine more intensively. Then things start to look not so good. Occasionally we experience high CPU usage (close to 100%). Each time, I had to stop all Redmine service, manually kill the ruby.bin process that refused to stop, and start up again.
Since yesterday, this has happened 3 times.
I suspected that this could be due to Redmine fetching svn change sets (since I've encountered that before). So I disable Autofetch commit in Redmine, and setup a scheduled job that runs every half an hour to fetch change sets from svn.
Today, the same problem happens again. And I can confirm this is not related to svn fetching because from my scheduled job log, I could see that it took no more than a second to fetch svn change sets.
I investigated production.log, but it seems everything was normal at that time (or rather I couldn't spot anything abnormal).
I would like to know how to go about troubleshooting this issue? And whether anyone has encountered this before?
My configuration:
bash-3.2# RAILS_ENV=production script/about About your application's environment Ruby version 1.8.7 (i686-linux) RubyGems version 1.3.5 Rack version 1.0 Rails version 2.3.5 Active Record version 2.3.5 Active Resource version 2.3.5 Action Mailer version 2.3.5 Active Support version 2.3.5 Edge Rails revision unknown Application root /opt/redmine/apps/redmine Environment production Database adapter mysql Database schema version 20100221100219 About your Redmine plugins Redmine Code Review plugin 0.3.0Other than that:
- Red Hat Enterprise Linux Server
- Ruby server used: Mongrel
- Mysql version: 5.1.30
Thanks a lot for any pointers!
Replies (6)
RE: High CPU usage
-
Added by Ryan Phung about 14 years ago
Sorry, I forgot to mentioned that I'm using Red Hat 5.3.
RE: High CPU usage
-
Added by Ryan Phung about 14 years ago
Seems no one has encountered a similar problem.
I think I also forgot to mention that I've recently upgraded to Redmine 0.9.3.
RE: High CPU usage
-
Added by Felix Schäfer about 14 years ago
Several things:
- the rails mysql adapter doesn't support mysql 5.1 yet (or not that I know of at least),
- mongrel is not the worst rails server, but not the best either, far from it, see if you can switch to something else, passenger for example,
- if not, make sure you have enough processes (higher concurrency),
- in any case, make one server process serve only so many requests and have it restart then,
- make sure you have enough memory.
RE: High CPU usage
-
Added by Ryan Phung about 14 years ago
Hi Felix, thanks a lot for your suggestions.
- Yes, from http://www.redmine.org/wiki/redmine/RedmineInstall it seems that the Ruby MySQL gem currently does not support MySQL 5.1. My installation was a Bitnami Stack (http://bitnami.org/stack/redmine). For some reason, MySQL 5.1 was bundled in the stack. I guess I need to live with it unless I can find time to downgrade to MySQL 4. So far it has not given any problem though.
- Currently I'm running 2 mongrel processes. Memory doesn't seem to be an issue as at the time the server hangs, memory usage was still pretty low. But I'll try out with some of your suggestions and report back. Thanks a lot!
RE: High CPU usage
-
Added by Felix Schäfer about 14 years ago
Can't say much for the bitnami stack, but I'd expect the configuration to be geared rather for desktops/small servers than possibly many concurrent users. Regarding MySQL: 5.0 works fine too, no need to go down to 4.x, though if the bitnami people ship it as a package, I wouldn't be searching there.
Regarding the bitnami stack: again, I have no first-hand knowledge of their packs, but if you/your company is committed to using redmine and you have a large-ish user base (150 users is not that much, though it heavily depends on usage, the relevant metrics would more likely be reqs/s and peak concurrent requests), I'd strongly suggest to move away from "black box" package install that you might be able to install fast but gets a little tricky to tweak/maintain. Take the time to get a basic notion of how rails "servers" work (the passenger docs are quite good, though some of the config/idioms is specific to them), install the rails stack "by hand" (my suggestion here would be ruby and MySQL from yum provided they are "recent" enough (means 1.8.6-ish and 5.0-ish at least) and rubygems from source, passenger from wherever you like depending on what yum has and if it works when rubygems is not form yum), and so on.
Long story short: bitnami is fine but sort of a black box, you will have more success streamlining your installation if you do some stuff by yourself and understand what you do, it does take a little learning curve, but it's worth it :-)
RE: High CPU usage
-
Added by Sergi de Pablos about 13 years ago
Yes, it seems there's some kind of high CPU usage problem with bitname stacks regarding Redmine. I'm using one with this same problem - the ruby.bin process takes up all CPU - but I'm planning to move away asap.