Magento export script using up all memory -- memory leak?

Question

Hello,

I have a script that processes Magento's product catalog and exports it.

Unfortunately it is killed halfway because it ends up using too much memory.

I am afraid I have incurred in the problem with the php garbage collector not being able to free circular references, hence causing a memory leak.

I've tried the fix suggested at http://ringsdorff.net/2009/07/23/guest-post-fix-for-memory-leaks-in-magento/ but unfortunately this breaks the script (no data is exported at all).

Does anybody know if there are any other fixes for this problem? Thank you!

Answer 1

Any time you are using catalog/product objects, there is in fact a memory leak, and your app will not be able to export everything at once. This is most likely the reason for the structure of the core Dataflow module. In the past, I have used two solutions:

1. Break the export up into chunks, as DataFlow does.
2. Write your own wrappers for the data, rather than using Magento's

If you take the implicit third option of fixing Varien's memory leak, please remember to contribute it back as a bugfix. :)

Hope that helps!

Thanks, Joe