Store a large PHP array, plain php, or gzipped serialized data?

Question

I have a static large array-of-arrays, witch represents tree structure with about ~100k nodes, this array is only read-only reference for value lockups.

Now, witch of this methods will perform better?

First, simply array definition in pure PHP file, for including in requests

Second, serialize this array, gzip serialized output, and load gzipped file and unserialize for every request

Or convert array to SQLite or somethin' similar, but storage must be capable of fast lockup of long "ID path" ie. 1->5556->123->45->455->Node_name (With actually PHP table doing very good)

Memory limit on server is not a problem

Answer 1

You'll need to turn the array into a PHP value a some point anyway, so gzip is out.

So if you are going to decide between keeping it on disk using something like sqlite, or just let php load it in every time (preferably having APC enabled), the real question is whats more important to you, memory or CPU. If you don't know yet, you're probably suffering from a case of premature optimization.

When it does become relevant to you to either cut down on memory or cpu, (or io) the answer will be more obvious, so make sure you can easily refactor.

If you want to predict what's better for you, do a benchmark.

Update I just saw memory is apparently not a concern. Go for the PHP array and include the file. Easy. Keep in mind though that if the total data size is 10MB, this will be 10MB per apache process. At a 100 apache processes this is already 1GB.

Answer 2

My benchmarks are telling everything:

Pure PHP file size: ~5 MB

Pure PHP file import: avg load time: ~210 ms
Pure PHP file import with APC: avg: ~60-80 ms
Unserialize(gzuncompress(file_get_contents()) : almost const, every request: ~ 40 ms

SQlite i've not tested, because lack of time, to port this tree structure into SQL DB

~host is on 4 core Intel Xeon with HT, and hardware RAID 5