running several background threads in succession on iphone / ipad

Question

I have to download a ton of images and I am doing it on a background thread, problem is all of the downloaded data is not released until I go back to the main thread which is fine a for couple hundred images but when I get into thousands the app runs out of memory and crashes.

So I need to run several background threads in succession so I can batch the download of the images in groups of say 200 so my autorelease pools can clear and memory gets released.

I cannot wrap my head around how to do this. I need some sort of recursive function on the main thread to call the background threads with and keep track of the batches so it knows which to call next. I know that passing values back and forth between threads will cause me some problems so I am not sure how to approach this?

Anyone tackle a problem like this before?

Answer 1

You don't need to wait for everything to download before releasing memory. You could just pass each image or small batch of images to the main thread using performSelectorOnMainThread, and let the main thread release memory as it caches the data in storage or uses the data. Then continue on in the background thread until done and pass a "done" message, again using performSelectorOnMainThread.

Answer 2

NSOperationQueues solve this problem for you. You create a class that descends from NSOperation that does the download for you. You then create an NSOperationQueue and add the downloads to the queue. You can control the max concurrency of the queue, etc.

http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/NSOperationQueue_class/index.html