Convert large XML file to CSV in PHP

Question

I have a 50MB XML file. I want to convert it to a CSV file, but most methods I have found exhaust the server memory. Is there a good way to do this using a stream method such as XMLreader.

Answer 1

the SAX-style expat-based parser is the most space-efficient option:

http://php.net/xml_parse

it will execute your $start_element_handler and $end_element_handler callbacks whenever an element tag is opened or closed, rather than keeping the entire document in memory.

but still, 50 MB is not a lot, maybe your provider can up the limit.

php_value memory_limit 100M

in .htaccess/httpd.conf, or set it in php.ini.

Answer 2

Have you tried to increase memory limit ? ini_set('memory_limit', '256M')

(That's a very bad solution btw)

Answer 3

I don't know much about PHP API, but seems this class can help you: XML Parser

Basically you're looking for a parser based on events, like old SAX. This parser type will fire an event, or something similar. It'll be memory efficient, as it doesn't need to load your entire document into memory.

Answer 4

You'd want to use XmlReader to parse the XML, as it works as an event based parser - Eg. it doesn't load everything into memory, but rather reads as it advances through the input file.

Answer 5

If the XML file is rather simple and could avoid going through a full-fledged XML parser, and could instead be read line-by-line by PHP and export each line as it goes, that would save having the whole file in memory at once. What's the XML structure?

Answer 6

I've written this algorithm some time ago.. Feel free to give it a shot.

http://sites.google.com/site/soichih/q-a/xml-to-csv-converter