php/symfony/doctrine memory leak?

Question

I'm having problems with a batch insertion of objects into a database using symfony 1.4 and doctrine 1.2.

My model has a certain kind of object called "Sector", each of which has several objects of type "Cupo" (usually ranging from 50 up to 200000). These objects are pretty small; just a short identifier string and one or two integers. Whenever a group of Sectors are created by the user, I need to automatically add all these instances of "Cupo" to the database. In case anything goes wrong, I'm using a doctrine transaction to roll back everything. The problem is that I can only create around 2000 instances before php runs out of memory. It currently has a 128MB limit, which should be more than enough for handling objects that use less than 100 bytes. I've tried increasing the memory limit up to 512MB, but php still crashes and that doesn't solve the problem. Am I doing the batch insertion correctly or is there a better way?

Here's the error:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /Users/yo/Sites/grifoo/lib/vendor/symfony/lib/log/sfVarLogger.class.php on line 170

And here's the code:

public function save($conn=null){

    $conn=$conn?$conn:Doctrine_Manager::connection();

    $conn->beginTransaction();


    try {
        $evento=$this->object;


        foreach($evento->getSectores() as $s){

            for($j=0;$j<$s->getCapacity();$j++){

                $cupo=new Cupo();
                $cupo->setActivo($s->getActivo());
                $cupo->setEventoId($s->getEventoId());
                $cupo->setNombre($j);
                $cupo->setSector($s);

                $cupo->save();

            }
        }

        $conn->commit();
        return;
    }
    catch (Exception $e) {
        $conn->rollback();
        throw $e;
    }

Once again, this code works fine for less than 1000 objects, but anything bigger than 1500 fails. Thanks for the help.

Answer 1

Try to unset($cupo); after every saving. This should be help. An other thing is to split the script and do some batch processing.

Answer 2

Try to break circular reference which usually cause memory leaks with

$cupo->save();

$cupo->free(); //this call

as described in Doctrine manual.

Answer 3

I tried setting $cupo to null, freeing it and unsetting it, but that didn't work.

Answer 4

I got the same problem, none of the above solutions work for me.

Answer 5

Periodically close and re-open the connection - not sure why but it seems PDO is retaining references.

Answer 6

I have just did "daemonized" script with symfony 1.4 and setting the following stopped the memory hogging:

sfConfig::set('sf_debug', false);