concurent background donwloads on iphone

Question

Hi there!

I am trying to create class that will handle multiple downloads at same time (I need to download a lot of small files) and I have problems with "disappearing" connections.

I have function addDonwload that adds url to list of urls to download, and checks if there is free download slot available. If there is one it starts download immediately. When one of downloads finishes, I pick first url form list and start new download.

I use NSURLConnection for downloading, here is some code

- (bool) TryDownload:(downloadInfo*)info
{
    int index;
    @synchronized(_asyncConnection)
    {
        index = [_asyncConnection indexOfObject:nullObject];
        if(index != NSNotFound)
        {
            NSLog(@"downloading %@ at index %i", info.url, index);
            activeInfo[index] = info;
            NSURLRequest *request = [NSURLRequest requestWithURL:info.url cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:15];

            [_asyncConnection replaceObjectAtIndex:index withObject:[[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:TRUE]];
            //[[_asyncConnection objectAtIndex:i] scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];           

            return true;
        }
    }

    return false;
}

- (void)connectionDidFinishLoading:(NSURLConnection*)connection
{
  [self performSelectorOnMainThread:@selector(DownloadFinished:) withObject:connection waitUntilDone:false];
}

- (void)DownloadFinished:(id)connection
{
    NSInteger index = NSNotFound;
    @synchronized(_asyncConnection)
    {
        index = [_asyncConnection indexOfObject:(NSURLConnection*)connection];
    }

    [(id)activeInfo[index].delegate performSelectorInBackground:@selector(backgroundDownloadSucceededWithData:) withObject:_data[index]];
    [_data[index] release];
    [activeInfo[index].delegate release];
    @synchronized(_asyncConnection)
    {
        [[_asyncConnection objectAtIndex:index] release];
        [_asyncConnection replaceObjectAtIndex:index withObject:nullObject];            
    }
    @synchronized(downloadQueue)
    {
        [downloadQueue removeObject:activeInfo[index]];
        [self NextDownload];
    }
}

- (void)NextDownload
{
    NSLog(@"files remaining: %i", downloadQueue.count);
    if(downloadQueue.count > 0)
    {
        if([self TryDownload:[downloadQueue objectAtIndex:0]])
        {
            [downloadQueue removeObjectAtIndex:0];
        }
    }
}

_asyncConnection is my array of download slots (NSURLConnections) downloadQueue is list of urls to download

What happens is, at the beginning everything works ok, but after few downloads my connections start to dissapear. Download starts but connection:didReceiveResponse: never gets called. There is one thing in output console that I don't understand I that might help a bit. Normaly there is something like 2010-01-24 21:44:17.504 appName[3057:207] before my NSLog messages. I guess that number in square brackets is some kind of app:thread id? everything works ok while there is same number, but after some time, "NSLog(@"downloading %@ at index %i", info.url, index);" messages starts having different that second number. And when that happens, I stop recieving any callbacks for that urlconnection.

This has been driving me nuts as I have strict deadlines and I can't find problem. I don't have many experiences with iphone dev and multithreaded apps. I have been trying different approaches so my code is kinda messy, but I hope you will see what I am trying to do here :)

btw is anyone of you know about existing class/lib I could use that would be helpfull as well. I want paralel downloads with ability o dynamically add new files to download (so initializing downloader at the beginning with all urls is not helpful for me)

Answer 1

This snippet can be the source of the bug, you release the object pointed to by the activeInfo[index].delegate pointer right after issuing async method call on that object.

[(id)activeInfo[index].delegate performSelectorInBackground:@selector(backgroundDownloadSucceededWithData:) withObject:_data[index]];
[_data[index] release];
[activeInfo[index].delegate release];

Answer 2

Do you use connection:didFailWithError: ? There may be a timeout that prevents the successful download completion.

Try to get rid of the @synchronized blocks and see what happens.

The string inside the square brackets seems to be thread identifier as you guessed. So maybe you get locked in the @synchronized. Actually, I don't see a reason for switching thread - all the problematic code should run in the main thread (performSelectorOnMainThread)...

Anyhow, there is no need to use both the @synchronized and the performSelectorOnMainThread.

BTW, I didn't see the NSURLConnection *connection = [[NSURLConnection alloc] initWithRequest:request delegate:self]; line. Where do you initiate the connection?

As for the parallel downloads - I think that you can download more than one file in a time with the same code that you use here. Just create a separate connection for each download.

Answer 3

You've got a bunch of serious memory issues, and thread synchronization issues in this code.

Rather than go into them all, I'll ask the following question: You are doing this on a background thread of some kind? Why? IIRC NSURLConnection already does it's downloads on a background thread and calls your delegate on the thread that the NSURLConnection was created upon (e.g., your main thread ideally).

Suggest you step back, re-read NSURLConnection documentation and then remove your background threading code and all the complexity you've injected into this unnecessarily.

Further Suggestion: Instead of trying to maintain parallel positioning in two arrays (and some sketchy code in the above relating to that), make one array and have an object that contains both the NSURLConnection AND the object representing the result. Then you can just release the connection instance var when the connection is done. And the parent object (and thus the data) when you are done with the data.

Answer 4

Consider just keeping a download queue along with a count of active connections, popping items off the top of the queue when downloads complete and a slot becomes free. You can then fire off NSURLConnection objects asynchronously and process events on the main thread.

If you find that your parallel approach prohibits doing all of the processing on the main thread, consider having intermediary manager objects between your main thread download code and NSURLConnection. Using that approach, you'd instantiate your manager and get it to use NSURLConnection synchronously on a background thread. That manager then completely deals with the downloading and passes the result back to its main thread delegate using a performSelectorOnMainThread:withObject: call. Each download is then just a case of creating a new manager object when you've a slot free and setting it going.

Answer 5

I recommend that you take a look at this: http://allseeing-i.com/ASIHTTPRequest/

It's a pretty sophisticated set of classes with liberal licensing terms (free too).

It may provide a lot of the functionality that you are wanting.