Have PHP dump heap on OutOfMemory exception

views:

answers:

+1 Q:

Have PHP dump heap on OutOfMemory exception

I am currently debugging a script that constantly runs into OutOfMemory exceptions. It is run as a cronjob and usually runs fine, but when the cronjob wasn't run for a while (for whatever reason) the script has to handle to many elements that queued up and will run into a OutOfMemory exception.

From examining the code I was not able to spot the problem. I believe one of the iterative function calls might leak memory, but I am not sure which one and where. Is there an option to get PHP to dump the heap, when an OutOfMemory exception occurs? I might be able to spot the problem from there (most likely).

+1 A:

Well, easiest approch would be to use a try-catch block around that part of your script where the error possibly occurs and you will have to dump the stack in the catch part. Proble might be that the machiene won't be able to react cause the memory is full and it terminates. I do not know if it helps to discard some variables to free up some mermory to output some data.

EDIT: For this purpose use the php function debug-backtrace. This will give you a stack trace. So finding the error will be much likely in case the machine is still up.

Thariama 2010-10-25 12:01:20

I think his question is about *how* to dump the heap.

Pekka 2010-10-25 12:01:57

Pardon me, but how do I "dump the heap" in the catch part? I am of course not talking about SplHeap, but the PHP scripts memory heap.

Daniel Baulig 2010-10-25 12:03:14

Daniel Baulig 2010-10-25 12:16:45

I also already know in which part of the script it dies. It's not always the exact line of code, but always within a certain loop. I suspect a memory leak anywhere within that loop, however, the stack of that loop is pretty deep (so lots of function calls, within function calls, within function calls) and the point where the script crashes must not necessary be the place where the script leaks. In fact it crashes at various points all the time (propably due to non-deterministic gc).

Daniel Baulig 2010-10-25 12:22:55

how big (in lines of code) is this loop?

Thariama 2010-10-25 12:24:46

Hard to grasp because of nested function calls - roughly 1000 lines, maybe a bit more.

Daniel Baulig 2010-10-25 12:32:51

did you try to find out the number of loops at which the error occurs?

Thariama 2010-10-25 12:41:02

No, I did not. How could that help me?

Daniel Baulig 2010-10-25 14:25:44

it would help if you would know that the error occured after the 435th loop - it would make debbuging easier - in that case you could check for the 435th loop and print out debugmessages (then you can find the error in O (log n) time when n is the amount of lines of code to check). Make sure to restart your server everytime you check for the number of loop in order to empty the memory used by your server.

Thariama 2010-10-25 15:10:13

I dont think knowing that the error occurs on the 435th iteration will help me, since what the loop is doing is processing a bunch of table rows and updating them back again. The table rows are basicly the same (of course different values, but nothing fancy or of variant length like blobs). Each iteration is basicly the same, except that one row gets value A and another row gets value B. Of course this is simplyfied, but in end it boils down to exactly that. Determining if a row gets value A or B of course is complex and involves lots of stuff.

Daniel Baulig 2010-10-25 19:14:33

But again, nothing like "Oh, that produces an overflow on my array_size variable if $i is 435 and my array suddenly is 4 billion elements long", that could cause a sudden OOM exception on the 435th iteration. I dont want to rule that kind of stuff out completely, after all I might miss something, but a memory leak is the most likely cause for the problem (maybe a recursive reference somewhere ) and I will not start hunting for witches until I can rule out a memory leak. Anyway, thank you for your precious time and your willingness to help, +1 for that.

Daniel Baulig 2010-10-25 19:19:31

thanks, to count the iterations of the loop is fast (takes 2 lines of code). You can never have enough information regarding a bug or other problem :)

Thariama 2010-10-26 07:10:28

Just do not load all objects together to memory, but read-as-you-process-them?

joni 2010-10-25 12:10:18

That is what the script seems to do, from visually inspecting the code. As I said, I suspect a memory leak. Maybe due to circular references (this is PHP 5.0), but debugging thousands of lines of code by reading them line for line to maybe spot the problem is just a waste of time.

Daniel Baulig 2010-10-25 12:15:44

You need to be creative in PHP to create a memory leak, except you just take an array and put in data (possibly using some strange associative keys) without removing it again...

joni 2010-10-25 12:37:15

You mean as creative as creating recursive associations? They, for example, will create memory leaks in pre 5.3 because the gc cannot detect them. Since the code I am debugging is not mine, I do not know what sick stuff might be going on.

Daniel Baulig 2010-10-25 14:24:02

I've had lots of problems with simpleXML and memory leaks. They are a pain in the are to track down... took me days to figure out that simpleXML was causing then and then fix them. As far as i know you cand programatically set a handled for OOM:)

Also, PHP's functions for displaying memory info fails to detect the memory leaks, i had scripts eating up ~1Gb of ram, but PHP's functions reported only 100Mb used:)

Quamis 2010-10-25 13:10:23

ansaurus

tags:

views:

answers:

Have PHP dump heap on OutOfMemory exception

related questions