php memory overflow multi curl

Question

Hi all,

I have a PHP routine that I call shell_exec to traverse my own website and build an index.

However since I do this multi-threaded I have been running into problems with memory storage...server memory spikes to 250MB then back down to 100MB randomly.

I am constantly accessing the mysql database during this time; and the php slows down.

Is this possible using php or should I use jsp/java.