Best approach for speedy debug and efficiently fixing large image files upload failure

Question

Consider a normal PHP image upload functionality (not using AJAX) and there occurs this problem of large image upload failing occasionally - less frequently on one test server and more frequently on another test server. Assuming the debugger has not yet started debugging the problem and there are no file/folder permission issues, how to proceed?

I am sure I have file_uploads on. I do not want to just blindly set some safe values or increase the values until it works. Basically, I want the values to be exactly as per my concerned modules. I am ready to override the settings in my concerned modules, if that is the best approach.

According to settings related to file upload, these are all the relevant/related settings -
* file_uploads
* upload_max_filesize
* max_input_time
* memory_limit
* max_execution_time
* post_max_size

1. Finding parameters/values for concerned script -
   So that I can find out which one of them is/how many of them are actually being violated by my script and causing the failure, I need to first find the corresponding values for my script. How to find the following values for my script:

   • Total uploaded files size
   • Input time
   • Memory usage
   • Script execution time
   • Posted data size

   Which tool(s) can be used for the same. Using PHP code, I think, I can find out a few:

   • Script execution time - Difference between microtime(true) at script start and end.
   • Total Uploaded file size - Foreach loop on $_FILES to find the sum of ['size'] attribute

   How to find out the rest like Memory Usage, Input time etc.?

2. Where/How to override
   Finally, when I have found the violating setting(s), suppose I need to increase/override values for 2 of the settings. Where to apply the override? I guess it is not correct to set memory_limit etc. for all the modules in htaccess or in PHP script. Rather, applying only in the concerned module is better. Am I correct?

3. Settings for Less demanding modules
   Also, for other modules, where much resources are not needed, is it good/wise to override the settings to reduce them, after carefully studying the resource requirements of the modules? Will it reduce unnecessary resource consumption? If so, how about having 2 or 3 combinations of these settings (depending on the project requirements, naming them normal-script, heavy-file-upload) and calling a single function to load any one combination for every module?

4. memory_limit precautions
   Regarding memory_limit it is mentioned here that -

   Setting too high a value can be very dangerous because if several uploads are being handled concurrently all available memory will be used up and other unrelated scripts that consume a lot of memory might effect the whole server as well.

What general precautions to take about this?

Thanks,
Sandeepan

Answer 1

A few ideas for debugging:

For manual testing, I would prepare a series of images with different dimensions whose net size (width x height) increases in small steps: 100 x 100, 100 x 200, 100 x 300 .... and try them. At some point, they could start failing if the problem is the memory limit. You could turn on error_reporting() for yourself only (maybe using a debugging cookie of some sort) so you see what exactly fails.

If that's not an option, I would set up a mechanism of some sort for long-term logging that stores the image's dimensions into a log file or table before the resizing starts, and also the contents of the $_FILES array. At the successful end of the script, add an "OK" to that entry. That way, you will be able to find out more about the failed uploads, if they make it through to the script (and don't fail beforehand due to a timeout setting).

Also, for other modules, where much resources are not needed, is it good/wise to override the settings to reduce them

I think the answer is always "no". As far as I know, the memory limit is the maximum limit of memory that can be allocated, but that amount is not reserved for every request. I have never heard of anybody fine-tuning the memory limit in this way.

However, if some parts of the system (e.g. the image resizer) require an enormously high memory limit, it may be wise to apply specific memory_limit settings only to them, e.g. through a .htaccess setting.