Passenger processes restart even though PassengerPoolIdleTime is 0

Question

I have set PassengerPoolIdleTime to 0, with the expectation that this means I can "warm" up a bunch of passenger processes on my server, and the next time I have a burst of traffic (even if it is days later), they will all be warmed up and ready to accept requests.

What I'm seeing instead is that every morning when I get up, passenger-status shows only a handful of processes and they have all only been up since midnight. The previous day I'd warmed up a bunch of processes and the last time I looked at passenger-status (before midnight) there were 50.

Here's the entire Passenger-related snippet from my httpd.conf (I'm on CentOS):

LoadModule passenger_module /usr/local/lib/ruby/gems/1.8/gems/passenger 2.2.11/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.11
PassengerRuby /usr/local/bin/ruby
PassengerMaxPoolSize 60
PassengerPoolIdleTime 0

I've checked the crontabs for root and apache, to see if there might be something triggering an apache restart, but I don't see it.

Here's a snippet of passenger-status, about 11hours and 46minutes after midnight:

----------- General information -----------
max      = 60
count    = 3
active   = 0
inactive = 3
Waiting on global queue: 0

----------- Domains -----------
/var/www/myapp/current: 
  PID: 20704   Sessions: 0    Processed: 360     Uptime: 11h 44m 16s
  PID: 20706   Sessions: 0    Processed: 4249    Uptime: 11h 44m 9s
  PID: 20708   Sessions: 0    Processed: 14189   Uptime: 11h 44m 9s

And here's what I see if I do a ps aux | grep apache:

apache   13297  0.0  0.0 546652  5312 ?        Sl   14:28   0:00 /usr/sbin/httpd.worker
apache   13332  0.0  0.0 546652  5336 ?        Sl   14:28   0:00 /usr/sbin/httpd.worker
apache   13334  0.0  0.0 546652  5328 ?        Sl   14:28   0:00 /usr/sbin/httpd.worker
root     16841  0.0  0.0   6004   628 pts/0    S+   15:48   0:00 grep apache
root     20478  0.0  0.0  88724  3640 ?        Sl   04:02   0:01 /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.11/ext/apache2/ApplicationPoolServerExecutable 0 /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.11/bin/passenger-spawn-server /usr/local/bin/ruby  /tmp/passenger.30916
apache   20704  0.0  1.7 251080 135164 ?       S    04:02   0:06 Rails: /var/www/apps/myapp/current                                                                                                                                                
apache   20706  0.2  1.7 255188 137704 ?       S    04:02   1:52 Rails: /var/www/apps/myapp/current                                                                                                                                                
apache   20708  0.9  1.7 255180 139332 ?       S    04:02   6:26 Rails: /var/www/apps/myapp/current

The server is on UTC, so 04:02 corresponds to 12:02am my time (EDT).

Answer 1

I discovered what was happening. Here is my logrotate conf file for httpd:

/var/log/httpd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript
}

It's the postrotate script that is doing it. Reloading apache causes the passenger processes to die off.

Anyone have any good suggestions for how to do this without having to reload apache? Or a way to reload apache without killing off the passenger processes (if that's possible)?

Answer 2

You could send your logs to a program which logs to a file based on date and eliminates logrotate...

CustomLog "|/usr/local/bin/my_log_script" combined