PHP throws 'Allowed memory exhausted' errors while migrating data in Drupal.

Question

I'm trying to setup a tiny sandbox on a local machine to play around with Drupal. I created a few CCK types; in order to create a few nodes I wrote the following script:

chdir('C:\..\drupal');

require_once '.\includes\bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
module_load_include('inc', 'node', 'node.pages');

$node = array('type' => 'my_type');

$link = mysql_connect(..);
mysql_select_db('my_db');

$query_bldg = '
  SELECT stuff
  FROM table
  LIMIT 10
';

$result = mysql_query($query_bldg);

while ($row = mysql_fetch_object($result)) {

 $form_state = array();

 $form_state['values']['name'] = 'admin';
 $form_state['values']['status'] = 1;
 $form_state['values']['op'] = t('Save');

 $form_state['values']['title'] = $row->val_a;
 $form_state['values']['my_field'][0]['value'] = $row->val_b;
 ## About another dozen or so of similar assignments...

 drupal_execute('node_form', $form_state, (object)$node);

}

Here are a few relevant lines from php_errors.log:

[12-Jun-2010 05:02:47] PHP Notice:  Undefined index: REMOTE_ADDR in C:\..\drupal\includes\bootstrap.inc on line 1299
[12-Jun-2010 05:02:47] PHP Notice:  Undefined index: REMOTE_ADDR in C:\..\drupal\includes\bootstrap.inc on line 1299
[12-Jun-2010 05:02:47] PHP Warning:  session_start(): Cannot send session cookie - headers already sent by (output started at C:\..\drupal\includes\bootstrap.inc:1299) in C:\..\drupal\includes\bootstrap.inc on line 1143
[12-Jun-2010 05:02:47] PHP Warning:  session_start(): Cannot send session cache limiter - headers already sent (output started at C:\..\drupal\includes\bootstrap.inc:1299) in C:\..\drupal\includes\bootstrap.inc on line 1143
[12-Jun-2010 05:02:47] PHP Warning:  Cannot modify header information - headers already sent by (output started at C:\..\drupal\includes\bootstrap.inc:1299) in C:\..\drupal\includes\bootstrap.inc on line 709
[12-Jun-2010 05:02:47] PHP Warning:  Cannot modify header information - headers already sent by (output started at C:\..\drupal\includes\bootstrap.inc:1299) in C:\..\drupal\includes\bootstrap.inc on line 710
[12-Jun-2010 05:02:47] PHP Warning:  Cannot modify header information - headers already sent by (output started at C:\..\drupal\includes\bootstrap.inc:1299) in C:\..\drupal\includes\bootstrap.inc on line 711
[12-Jun-2010 05:02:47] PHP Warning:  Cannot modify header information - headers already sent by (output started at C:\..\drupal\includes\bootstrap.inc:1299) in C:\..\drupal\includes\bootstrap.inc on line 712
[12-Jun-2010 05:02:47] PHP Notice:  Undefined index: REMOTE_ADDR in C:\..\drupal\includes\bootstrap.inc on line 1299
[12-Jun-2010 05:02:48] PHP Fatal error:  Allowed memory size of 239075328 bytes exhau   sted (tried to allocate 261904 bytes) in C:\..\drupal\includes\form.inc on line 488
[12-Jun-2010 05:03:22] PHP Fatal error:  Allowed memory size of 239075328 bytes exhausted (tried to allocate 261904 bytes) in C:\..\drupal\includes\form.inc on line 488
[12-Jun-2010 05:04:34] PHP Fatal error:  Allowed memory size of 262144 bytes exhausted (tried to allocate 261904 bytes) in Unknown on line 0

At this point any action php takes results in the last error shown above. I tried increasing the value of memory_limit in php.ini before the final Fatal error which obviously didn't help.

How can the error be eliminated? Am I on a correct path to migrating data into Drupal or should the cck tables be operated on directly?

Windows XP
PHP 5.3.2 VC6
Apache 2.2

Answer 1

WHat exactly is that 'node_form' call doing? I'm guessing it's that $form_state data and building some huge in-memory structure from it. That means:

a) You have to increase (or eliminate) the memory limit - Drupal's building something big, and you're not giving it the headroom to work

b) Change the node_form stuff so it builds smaller chunks at a time. Maybe flush output to the client at intervals? Save to disk?

Answer 2

If you're trying to migrate data from sql (or anywhere else), you'll find doing your own direct import is fairly complicated. Have a look at this method: http://www.lullabot.com/articles/drupal-data-imports-migrate-and-table-wizard