views:

24

answers:

3

Hi guys,

I am parsing an XML file with PHP and inserting the rows in a MYSQL database.

I am using PHP simplexml_load_files to load the XML and a foreach to loop through the array and insert the rows into my database.

It works perfectly fine with small files i am testing, but it comes to reality I need to parse a large 500mb XML file and nothing happens.

I was wondering what was the right Php.ini config for this case ?

I have a VPS Linux Cent OS, with 256 mb of dedicated Memory and MYSQL 5.0.5. I have also set php memory_limit = 256M (maximum of my server)

Any suggestions, similar experiences will be greatly appreciated

Thanks

A: 

gosh, a 500 MB XML file? I guess you run out of memory. try XMLReader, that’s a pull parser (parsing the XML (more or less) linewise), so that you don’t run out of memory so easily.

Dormilich
Yes, it may work but that requires rewriting my code again.. Is there any chance to use simplexml_load_file for large files ?
Matias
SimpleXML (like DOMDocument) parse the XML as a whole and have to create a DOM tree. this easily requires more than 100 times the file size in memory.
Dormilich
A: 

I suggest you use xmlreader ... its more efective as memory is concerned

solomongaby
A: 

The memory config used for xml is located in a third party library, not at php.ini config. Besides that you should XMLReader or ext/xml to process such a large file, that means streaming access and not use dom data access (SimpleXML).

Dez