Mongrel consuming all memory

Added by Brad Schick about 12 years ago

I setup my server using Mongrel to run Redmine, but unfortunately the Mongrel processes grow to consume all memory on my box within 2 days. The site has modest user traffic and I a lot of the load is from indexing. I'm creating a custom robots.txt file to see if it well help, but in a bug report Jean-Philippe noted that he has found Mongrel to be unsuitable for production environments. Clearly some people make it work, however. I'm wonder if others have hit this problem and determined where the problem lies. Seems like it could be either Mongrel, Rails, or Redmine itself. There seem to be plenty of complaints around about Mongrel hosting Rails apps consuming tons of memory, but most of the discussions have no resolution.

Replies (11)

RE: Mongrel consuming all memory - Added by Brad Schick about 12 years ago

One thing I found in some forum was an issue with rails keeping log files in memory. I see that my production.log file grew to 1G in size quickly (a few weeks). I'm new to the ruby/rails world... any easy way to drastically reduce logging? Do I have some debug logging flag set by mistake perhaps?

RE: Mongrel consuming all memory - Added by Brad Schick about 12 years ago

Below is what my log is mostly filled with (looks like ActiveRecord logs of DB activity):

[[0mSELECT * FROM `roles` WHERE (`roles`.`builtin` = 2) LIMIT 1^[[0m
[[4;36;1mIssue Load Including Associations (0.006213)[[0m ^[[0;1mSELECT `issues`.`id` AS t0_r0, `issues`.`tracker_id` AS t0_r1, `issues`.`project_id` AS t0_r2, `issues`.`subject` AS t0_r3, `issues`.`description` AS t0_r4, `issues`.`due_date` AS t0_r5, `issues`.`category_id` AS t0_r6, `issues`.`status_id` AS t0_r7, `issues`.`assigned_to_id` AS t0_r8, `issues`.`priority_id` AS t0_r9, `issues`.`fixed_version_id` AS t0_r10, `issues`.`author_id` AS t0_r11, `issues`.`lock_version` AS t0_r12, `issues`.`created_on` AS t0_r13, `issues`.`updated_on` AS t0_r14, `issues`.`start_date` AS t0_r15, `issues`.`done_ratio` AS t0_r16, `issues`.`estimated_hours` AS t0_r17, `users`.`id` AS t1_r0, `users`.`login` AS t1_r1, `users`.`hashed_password` AS t1_r2, `users`.`firstname` AS t1_r3, `users`.`lastname` AS t1_r4, `users`.`mail` AS t1_r5, `users`.`mail_notification` AS t1_r6, `users`.`admin` AS t1_r7, `users`.`status` AS t1_r8, `users`.`last_login_on` AS t1_r9, `users`.`language` AS t1_r10, `users`.`auth_sou...

RE: Mongrel consuming all memory - Added by Brad Schick about 12 years ago

If anyone else runs into this, I solved the problem with two solutions:

  1. Switched from Mongrel to Thin. On my Ubuntu system, Thin's memory consumption doesn't blow up like Mongrel.
  2. Created a logrotate rule to rotate, compress, and eventually delete the logs. Here is the configuration (the daily restarting of thin help also)
/path/to/log/*.log {
        daily
        missingok
        rotate 14
        compress
        delaycompress
        notifempty
        create 666 user group
        sharedscripts
        postrotate
                /etc/init.d/thin restart > /dev/null
        endscript
}

RE: Mongrel consuming all memory - Added by Thomas Pihl about 12 years ago

Your logs look wrong. There shouldn't be any database calls in production.log. Please verifiy that you havent uncommented the "config.log_level" in environment.rb or changed it somewhere else.

Also verify that you really run rails in production-environment-state. There should be an "environment: production" in mongrel_cluster.yml. If you run it in development, it'll eat your memory in no-time flat.

/T

RE: Mongrel consuming all memory - Added by Karl Heinz Marbaise about 12 years ago

Hi there,

I'm currently running into the same problem. I have configured an Apache using a balancer (using five mongrels cluster nodes), but they ran into the same problem after a few days...
The contents of the mongrel_cluster.yml in /etc/mongrel_cluster

---
cwd: /usr/local/vhosts/redmine
log_file: log/mongrel.log
port: "3000" 
environment: production
address: 127.0.0.1
pid_file: tmp/pids/mongrel.pid
servers: 5

I have checked that the correct environment is working and so on...

RE: Mongrel consuming all memory - Added by Thomas Pihl about 12 years ago

How is the memory allocated when you run out (or perhaps take the same check over time to see what grows)?

You can used "ps v -e"

/T

RE: Mongrel consuming all memory - Added by Sergi de Pablos about 12 years ago

I've got mine installed in a VPS, sharing memory with other RoR and Wordpress applications, and I've found the best performance is obtained using Phusion Passenger (aka modrails, http://www.modrails.com/). Even better, it runs inside my apache2, so I don't need additional web servers and I can configure it with a simple virtualhost.

Here http://www.railsplugins.net/passenger.mov you could find a screencast explaining how to install Passenger.

Some benchmarks:

S.

RE: Mongrel consuming all memory - Added by Thomas Pihl about 12 years ago

Thats interesting. I have made that very same journey and i am also very impressed by passenger. In my case i changed the timeouts to huge values to avoid the 3-5 seconds of startup-time since my users are used to mongrels "always allocated" state.

/T

RE: Mongrel consuming all memory - Added by Sergi de Pablos about 12 years ago

Interesting. Where do you change the timeout for the process?

S.

RE: Mongrel consuming all memory - Added by Thomas Pihl about 12 years ago

http://www.modrails.com/documentation/Users%20guide%202.0.html#_resource_control_and_optimization_options

In my Q&D testenviron i just added it behind the cut/pasted section from installation in httpd.conf

PassengerMaxPoolSize 2
PassengerPoolIdleTime 86400

Max 2 since i am testing it with 256mb ram, timeout equals 24 hours.

/T

RE: Mongrel consuming all memory - Added by Sergi de Pablos about 12 years ago

Just tried. Working like a charm. Thanks Thomas.

S.

(1-11/11)