LAMP and memory/swap space problem

Question

My LAMP application seems to eventually use up all of my server's memory and swap space. My gut feeling is that it has something to do with the external processes I have to call (as that is the only time the problem manifests).

I need to call GhostScript, ImageMagick's "convert", PDFTK, etc. constantly. When those processes are running, that's when I see my memory running out. So, questions:

1. Which techniques should I use to conclusively identify which process is actually causing the memory problems? My plan right now is to run the processes individually and just observe the memory usage via the *nix command "top". Is there a way I can do this programatically?

2. Is there any "memory flushing" solutions I can use? Would this be a good idea to do?

Answer 1

Another problem you might face is the fact that when forking, the application where you fork from is "doubled" so its memory consumption doubles. If you have a app server which is resident and keeps a lot of data cached this can be very significant.

A solution to this problem is to run a small resident script/program listening on a socket or named pipe to launch the external programs.

You can use top -b (or similar) to receive computer readable output and monitor the memory consumption there with a script.

BTW: Do not count swap space as "real" memory, your app should run without ever hitting swap space. Once you start hitting swap space performance goes down so fast that request start piling up, causing more memory to be used, causing more stuff to be swapped. If you see significant swap space being allocated then increase memory (or buy a bigger hosting plan)