Large File Upload Memory Issue PHP

Question

I recently took over a file hosting site (similar to rapidshare, megaupload etc.) and currently averaging about 75k visitors/day. After the migration, we wrote our own system from scratch. It's now time to upgrade the infrastructure and have been toying around with web servers and scaling. Now the issue is that after file upload is complete, the web server is not "flushing" the used-up memory. We uploaded multiple 300MB+ files and before the upload, we had about 3.7G of memory free and after uploading 3 files of 300M each, the memory usage stands at around 1.2G. Initially, we thought it could be apache issue, but upgrading to Litespeed didn't fix it either. Here's the "memory usage" from 'top'

Mem: 4015996k total, 1206036k used, 2809960k free, 114984k buffers

I am scratching my head to figure out how to "flush" the memory used up by the web server or by php during the upload, because this way the system will run out of memory and crash in no time.

Here are my php.ini values:

max_execution_time = 6000 max_input_time = 6000 memory_limit = 2048M post_max_size = 2047M upload_max_filesize = 2047M

On Apache, php runs on fcgi and on Litespeed, it's Litespeed SAPI.

Thanks, Bill

Answer 1

I just figured out that it is the "cache memory" and I am able to free it up by this command: echo 3 > /proc/sys/vm/drop_caches

cat /proc/meminfo

MemTotal: 4015996 kB MemFree: 2490780 kB Buffers: 118232 kB Cached: 1204168 kB SwapCached: 0 kB Active: 358748 kB Inactive: 991220 kB Active(anon): 28024 kB Inactive(anon): 0 kB Active(file): 330724 kB Inactive(file): 991220 kB Unevictable: 0 kB Mlocked: 0 kB HighTotal: 3179664 kB HighFree: 1918528 kB LowTotal: 836332 kB LowFree: 572252 kB SwapTotal: 48827384 kB SwapFree: 48827384 kB