Concurrent execution in apache/mod_php?

Question

Hello,

i am currently working on an MVC webframework with a very special feature. Layouts defined in XML files are seperated into content blocks that can be arranged/cached/loaded individually.

Every content block (this could be e.g. the footer which is re-used on several pages) has some sort of Controller of its own. I call it Blockcontroller.

This controller executes relevant code (including DB requests via resource Models, external API requests via an HTTPclient model and so on).

The framework is designed to be used in very complex environments where multiple datasources are involved. This will expecially be several Database servers and REST apis.

Now the goal is to execute such Content blocks concurrently to speed up page delivery.

Example:

A Webinterface needs to fetch your address book from Google Mail, Facebook, Twitter and 10 other sources, do some matching and calculations. Every requests takes about 1 second. This adds up to 15 seconds if done sequentially.

The goal is to define a block for everything (yeah this should rather go in to a model but i want to keep it on the block level as most of the logic is defined in this xml files. they just generate no output then.) and set the "concurrent execution" flag to 1.

The last content block that assembles everything those concurrent Blocks as required attribute, so it begines to execute when they are all done and the data is available. Something like this:

<block template="blank.phtml" block="twitter.php" concurrent="1" name="twitter"/>
<block template="blank.phtml" block="gmail.php" concurrent="1" name="gmail"/>
<block template="blank.phtml" block="facebook.php" concurrent="1" name="facebook"/>
<block template="stats.phtml" block="statistics.php">
    <depends>twitter</depends>
    <depends>gmail</depends>
    <depends>facebook</depends>
</block>

The parser essentially has a look at the layout xml fiels first and scans for blocks that are marked for concurrent execution and assembles an array with a queue, and every queue point can have multiple entries. they are arragend in Dependency order.

So if something has no dependency it definately goes into the queue group at the very first position. Everything that depends something from this group goes into second positino and so on...

Data is stored into a singleton model.

My problem now is that it works fine with mod_cgi or mod_fastcgi for php, but it just messes all up with apaches mod_php.

I googled a bit and read that you should never use pcntl_fork with mod_php ... is there any alternative?

Answer 1

Instead of forking, use proc_open to run each block as different scripts on the command-line. This is not possible on all web hosts but maybe you have your own server anyway?

You cannot fork in mod_php because that will create an entirely different Apache process. It's really difficult to do concurrent work in mod_php, but it's doable (as I said) with proc_open.