pcntl_fork and then pcntl_exec or pcntl_exec and then pcntl_fork

Question

Explanation: Part of the app that I'm creating requires checking of thousands of records and acting on them in a timely manner. So for every record, I want to fork a new process. However, I need a DB connection to do some more checking of that record. As I understand it, the child inherits the DB connection. So subsequent forks have DB errors.

I thought I could pcntl_exec('php /path/script.php'); and then pcntl_fork so that the calling process is not held up.

Or I can pcntl_fork and then pcntl_exec in the child process. Or maybe I should be using exec() instead of pcntl_exec().

My question: Are there any drawbacks or advantages to either order?

Notes: Maybe I'm imagining this issue, as I thought that the calling php process would wait for pcntl_exec to return. But that's not what the docs state:

Returns FALSE on error and does not return on success.

How can a function return a value sometimes and none other times? That sounds like poorly written docs.

fahadsadah's comments state:

Once the executed process ends, control returns to the webserver process.

If that is the case, then I need to fork.

Edit: code for the confused - including me ;)

<?php

class Process
{

    public function __construct($arg = false)
    {
        if ($arg == "child")
        {
            $this->act();
        }
        else
        {
            $this->run();
        }
    }

    public function run()
    {
        echo "parent before fork:", getmypid(), PHP_EOL;
        $pid = @ pcntl_fork();
        echo $pid, PHP_EOL;

        if ($pid == -1)
        {
            throw new Exception(self::COULD_NOT_FORK);
        }
        if ($pid)
        {
        // parent
            echo "parent after fork:", getmypid(), PHP_EOL;
        }
        elseif ($pid == 0)
        {
        // child
            echo "child after fork:", getmypid(), PHP_EOL;
            //echo exec('php Process.php child');
            echo pcntl_exec('/usr/bin/php', array('Process.php', 'child'));
        }
        return 0;
    }

    private function act()
    {
        sleep(1);
        echo "forked child new process:", getmypid(), PHP_EOL;
        return 0;
    }
}

$proc = new Process($argv[1]);

If you uncomment the exec and comment the pcntl_exec, you will see that pcntl_exec replaces the process. Which I'm guessing saves some resources.

Answer 1

This doesn't make sense. Once you exec() you're running different code so you can't fork() afterwards. Does not return on success.

Answer 2

This is really confused - you're trying to apply very sophisticated techniques - but you are applying them in completely the wrong way.

fork creates a new running copy of the current process. Exec starts a new process. You would not use them both to start a single process.

But before I get into a en explanation of how to use fork and exec correctly, I should point out that they are not the right tools for addressing this problem.

Batch processing should be avoided wherever possible. Data typically arrives at a finite rate (albeit that the rate may be stochastic) - usually the right approach to avoid batching is to deal with requests synchronously or via queueing. Where batch processing is unavoidable, parallelizing and/or pipelining processing usually improves throughput. While there are many sophisticated methods for achieving this (e.g. map-reduce) simply sharding the data is usually adequate. While your basic idea amounts to sharding into single pieces, this:

1) will be less efficient than dealing with small batches

2) makes it very difficult to limit resource consumption by the system (what if you spawn 500 processes and your DBMS only supports 200 concurrent connections?)

Assuming that you can't deal with the processing synchronously and runiing a queue with multiple subscribers is not practical, I'd suggest just splitting the data into (a limited number of) smaller batches and spawning processes to deal with those. Note that popen(), proc_open() and pcntl_fork() do not block for the duration of execution of the spawned process. (hint - use the modulus operator)

If you want to to launch the processing from an HTTP request (or have another reason for running them in seperate session groups) then have a google for 'PHP long running processes setsid).