Strategies for handling memory consumption in PHP5 ?

Question

We have a large management software that is producing big reports of all kinds, based on numerous loops, with database retrievals, objects creations (many), and so on.

On PHP4 it could run happily with a memory limit of 64 MB - now we have moved it on a new server and with the same database - same code, the same reports won't come up without a gig of memory limit...

I know that PHP5 has changed under the hood quite a lot of things, but is there a way to make it behave ?

The question at the end is, what strategies do you apply when you need to have your scripts on a diet ?

Answer 1

Since moving to the new server, have you verified that your MySQL and PHP system variables are identical to the way they were on your old server?

PHP5 introduced a lot of new functionality but due to its backward compatibility mantra, I don't believe that the differences between PHP5 and PHP4 should be causing this large an affect on the performance of an application who's code and database has not been altered.

Are you also running on the same version of Apache or IIS?

It sounds like a problem that is more likely related to your new system environment than to an upgrade from PHP4 to 5.

Answer 2

Thanks Gabriel for your answer, and I've thought about that. The original server was a wamp4, and I've got the same memory issue on a wamp5 and on a lamp5.

It's very difficult to compare both versions because on Windows PHP4 did not have get_memory_size available.

Although I agree there can be things outside the code that can change the way memory goes, there should be ways to do some refactoring to help things ?

Answer 3

Have you checked the error logs? You might have a tonne of warnings that are both slowing things down and adding to the memory usuage.

Answer 4

Bertrand,

If you are interested in refactoring the existing code then I would recommend that you first monitor your CPU and Memory usage while executing reports. Are you locking up your SQL server or are you locking up Apache (which happens if a lot of stress is being put onto the system by the PHP code)?

I worked on a project that initially bogged down MySQL so severely that we had to refactor the entire report generation process. However, when we finished the load was simply transferred to Apache (through the more complex PHP code). Our final solution was to refactor the database design to provide for better performance for reporting functions and to use PHP to pick up the slack on what we couldn't do natively in MySQL.

Depending on the nature of the reports you might consider denormalizing the data that is being used for the reports. You might even consider constructing a second database that serves as a data warehouse and is designed around OLAP principles rather than OLTP principles. You can start at Wikipedia for a general explanation of OLAP and data warehousing.

However, before you start looking at serious refactoring, have you verified that your environments are sufficiently similar by looking at phpinfo(); for PHP and SHOW VARIABLES; in MySQL?

Answer 5

A big problem we have run into was circular references between objects stopping them from freeing memory when they become out of scope.

Depending on your architecture you may be able to use __destruct() and manually unset any references. For our problem i ended up restructuring the classes and removing the circular references.

Answer 6

When I need to optimize resources on any script, I try always to analyze, profile and debug my code, I use xDebug, and the xDebug Profiler, there are other options like APD, and Benchmark Profiler.

Additionally I recommend you this articles:

• Make PHP apps fast, faster, fastest..
• Profiling PHP Applications (PDF)
• PHP & Performance (PDF)

Answer 7

A gig!?!

even 64MB is big.

ignoring the discrepancy between environments, (which does sound very peculiar), it sounds like the code may need some re-factoring.

any chance you can re factor your code so that the result sets from database queries are not dumped into arrays. I would recommend that you construct an iterator for your result sets. (thence you can treat them as array for most purposes).there is a big difference between handling one record at a time, and handling 10,000 records at a time.

secondly, have a look at weather your code is creating multiple instances of the data. Can you pass the objects by reference. (use the '&'). We had to do a similar thing when using an early variant of the horde framework. a 1 MB attachment would blow out to 50MB from numerous calls which passed the whole dataset as a copy, rather than as a reference.