Finding cause of memory leaks in large PHP stacks

Question

I have CLI script that runs over several thousand iterations between runs and it appears to have a memory leak. I'm using a tweaked version of Zend Framework with Smarty for view templating and each iteration uses several MB worth of code. The first run immediately uses nearly 8MB of memory (which is fine) but every following run adds about 80kb.

My main loop looks like this (very simplified)

$users = UsersModel::getUsers();
foreach($users as $user) {
  $obj = new doSomethingAwesome();
  $obj->run($user);
  $obj = null;
  unset($obj);
}

The point is that everything in scope should be unset and the memory freed.

My understanding is that PHP runs through its garbage collection process at it's own desire but it does so at the end of functions/methods/scripts. So something must be leaking memory inside doSomethingAwesome() but as I said it is a huge stack of code.

Ideally, I would love to find some sort of tool that displayed all my variables no matter the scope at some point during execution. Some sort of symbol-table viewer for php.

Does anything like that or any other tools that could help nail down memory leaks in php exist?

Answer 1

You could have a look at the Memory Manager functions to help narrow things down, that article has a pretty decent tutorial of using them to locate a leak.

Answer 2

There are several possible points of memory leaking in php:

• php itself
• php extension
• php library you use
• your php code

It is quite hard to find and fix the first 3 without deep reverse engineering or php source code knowledge. For the last one you can use binary search for memory leaking code with memory_get_usage

Answer 3

PHP doesn't have a garbage collector. It uses reference counting to manage memory. Thus, the most common source of memory leaks are cyclic references and global variables. If you use a framework, you'll have a lot of code to trawl through to find it, I'm afraid. The simplest instrument is to selectively place calls to memory_get_usage and narrow it down to where the code leaks. You can also use xdebug to create a trace of the code. Run the code with execution traces and show_mem_delta.

Answer 4

If what you say about PHP only doing GC after a function is true, you could wrap the loop's contents inside a function as a workaround/experiment.

Answer 5

I noticed one time in an old script that PHP would maintain the "as" variable as in scope even after my foreach loop. For example,

foreach($users as $user){
  $user->doSomething();
}
var_dump($user); // would output the data from the last $user

I'm not sure if future PHP versions fixed this or not since I've seen it. If this is the case, you could unset($user) after the doSomething() line to clear it from memory. YMMV.

Answer 6

I recently ran into this problem on an application, under what I gather to be similar circumstances. A script that runs in PHP's cli that loops over many iterations. My script depends on several underlying libraries. I suspect a particular library is the cause and I spent several hours in vain trying to add appropriate destruct methods to it's classes to no avail. Faced with a lengthy conversion process to a different library (which could turn out to have the same problems) I came up with a crude work around for the problem in my case.

In my situation, on a linux cli, I was looping over a bunch of user records and for each one of them creating a new instance of several classes I created. I decided to try creating the new instances of the classes using PHP's exec method so that those process would run in a "new thread". Here is a really basic sample of what I am referring to:

foreach ($ids as $id) {
   $lines=array();
   exec("php ./path/to/my/classes.php $id", $lines);
   foreach ($lines as $line) { echo $line."\n"; } //display some output
}

Obviously this approach has limitations, and one needs to be aware of the dangers of this, as it would be easy to create a rabbit job, however in some rare cases it might help get over a tough spot, until a better fix could be found, as in my case.