views:

487

answers:

3

I have a PHP application that uses Zend Framework, jQuery's ajax, and Zend_Session. This application has been around for about 7 months and working the way it should. When the application initializes after the user logs in, about 10 ajax requests are fired off to load up relevant data to a dashboard type page. After those have requests have finished, ajax requests are mainly user-initiated from that point on. When the application was working properly, we didn't have a truly load balanced environment, we had 3 app servers that handles requests, but each one stored PHP session data locally. Recently we changed that so that each app server is connected to a central NFS share where PHP session data is stored. That is when the application broke.

Now what happens is the page initializes, I can see the ajax requests waiting, but half of them timeout. If I wait long enough (about 3-10 mins), then all user-initiated clicks respond fast. We have verified that the problem was caused by our change in session handling.

Anyone have an explanation of what might be going on, how to troubleshoot, and/or a solution?

I greatly appreciate any help you can give. I've been pulling my hair out over this one.

A: 

This is a common problem. You have likely run into a session locking issue. The default session save handler that PHP uses, the file handler (and the one that Zend_Session uses by default), uses the flock() syscall to lock the session files.

How are you choosing to which server to send the individual HTTP requests? If it is just random, and any given request can be handled by any of the servers, then you could easily imagine a scenario where server 1 handles the initial request, creates the session file on the NFS share, and gets and keeps a lock on that file.

The next AJAX request comes in to a different server, where the PHP process on that server reads the session ID from the client's cookie and tries to get a lock on the session file on the NFS share. Since the first server still has the lock on the file, you are locked out and waiting.

That could be what's causing it. It's pretty common when you start trying to load balance PHP.

Some load balancers have a mode that provides 'sticky sessions' where the balancer is smart enough to look at the session ID over the wire and ensure that the same physical server handles all of the requests for a given session. It can make your load balancing less efficient, but it's worth it for getting rid of the problem.

Or, the problem might be related to something else. I don't know.

RibaldEddie
That is the conclusion that we can to as well, but we weren't 100% sure. I'm just not sure what the options are to fix it. I would love a quick and easy way, but it seems like we might need to reengineer a good portion of the app.
Bob
In another one of my posts on this site I suggest a small method for storing session data in the browser's cookie. The best solution is to avoid storing anything in the server-side session. Link to that answer is here: http://stackoverflow.com/questions/617793/what-is-the-best-way-to-persist-an-object-using-forms-in-php/617915#617915
RibaldEddie
+1  A: 

There could be several problems.

  • Mount NFS share with noac option to try to prevent stale copies (What is close-to-open cache consistency?).
  • NFS is slow
  • File locked (is session_write_close() called?) or not written out to file but only in local memory of server
  • There are known problems if you use framesets (check php session documentation)
  • There could also be problems if you change domains (a.domain.com, b.domain.com)
jitter
+1  A: 

To me quick and easy solution is to store sessions in DB, isn't it? :) http://framework.zend.com/manual/en/zend.session.savehandler.dbtable.html

Tomáš Fejfar