How to read multiple files in parallel?

Question

I have a number of text files that I need to open, then allocate certain fields to set strings inside PHP to eventually write into MySQL.

Each txt file does have the same unique value for app_id, eg

So text file 1 has

app_id name description

text file 2 has

app_id category

text file 3 has

app_id price

I want to get name, description, category and price for each record, and write that into mysql.

I can read the first one using code :

$fp = fopen('textfile1','r');
if (!$fp) {echo 'ERROR: Unable to open file.</table></body></html>'; exit;}
while (!feof($fp)) {
$line = stream_get_line($fp,4096,$eoldelimiter); 
if ($line[0] === '#') continue;  //Skip lines that start with # 
 $field[$loop] = explode ($delimiter, $line);
list($app_id, $name, $description) = explode($delimiter, $line);
 $fp++;
}

fclose($fp);
?> 

My question is how to read the 2nd and 3rd files? Can I somehow do it in parallel, or do I need to finish reading text file 1, write each line into mysql then open text file 2 and write category into mysql using replace on the app_id key, then do the same with text file 3 opening and ?

regards,

Greg

Answer 1

I would recommend the following approach:

• create a simple class containing the following fields: name, description, category, and price
• create an empty array that will be indexed with the app_id, each corresponding to an instance of the aforementioned class

• read each file and affect the result to the correct element in the array

  if (!isset($array[$app_id])) $array[$app_id] = new MyClass();

  array[$app_id]->name = $name;

  array[$app_id]->description = $description;

• once you're done reading all files, you can iterate through the array and write each element to the SQL table.

Answer 2

Simply open all three files in sequence and then have three stream_get_line's in your while loop, one for each file:

$fp1 = fopen('textfile1','r');

$fp2 = fopen('textfile2','r');

$fp3 = fopen('textfile3','r');

while (!feof($fp1)) {

$line1 = stream_get_line($fp1...)

$line2 = stream_get_line($fp2...)

$line3 = stream_get_line($fp3...)

...

You'll have to take care that each file has exactly the same number of lines, though. Best is probably to check for feof on each stream before reading a line from it, and aborting with an error message if one of the streams runs out of lines before the others.

Answer 3

Try this:

     file1.txt contents:
            id 13 category test description test
    file2.txt:
       id 13 name test description test
       id 15 name test description test
       id 17 name test description test


    $files = array('file1.txt','file2.txt');

            foreach($files as $file) {
                flush();
                preg_match_all("/id\s(?<id>\d+)\s(?:name|category|price)?\s(?<name>[\w]+)\sdescription\s(?<description>[^\n]+)/i",@file_get_contents($file),$match);
                    print_r($match);
            }

returns:
Array
(
    [0] => Array
        (
            [0] => id 13 category test description test
        )

    [id] => Array
        (
            [0] => 13
        )

    [1] => Array
        (
            [0] => 13
        )

    [name] => Array
        (
            [0] => test
        )

    [2] => Array
        (
            [0] => test
        )

    [description] => Array
        (
            [0] => test
        )

    [3] => Array
        (
            [0] => test
        )

)
Array
(
    [0] => Array
        (
            [0] => id 13 name test description test
            [1] => id 15 name test description test
            [2] => id 17 name test description test
        )

    [id] => Array
        (
            [0] => 13
            [1] => 15
            [2] => 17
        )

    [1] => Array
        (
            [0] => 13
            [1] => 15
            [2] => 17
        )

    [name] => Array
        (
            [0] => test
            [1] => test
            [2] => test
        )

    [2] => Array
        (
            [0] => test
            [1] => test
            [2] => test
        )

    [description] => Array
        (
            [0] => test
            [1] => test
            [2] => test
        )

    [3] => Array
        (
            [0] => test
            [1] => test
            [2] => test
        )

)