views:

1491

answers:

3

In my application I am executing 10 asynchronous NSURLConnections within an NSOperationQueue as NSInvocationOperations. In order to prevent each operation from returning before the connection has had a chance to finish I call CFRunLoopRun() as seen here:

- (void)connectInBackground:(NSURLRequest*)URLRequest {
 TTURLConnection* connection = [[TTURLConnection alloc] initWithRequest:URLRequest delegate:self];

 // Prevent the thread from exiting while the asynchronous connection completes the work.  Delegate methods will
 // continue the run loop when the connection is finished.
 CFRunLoopRun();

 [connection release];
}

Once the connection finishes, the final connection delegate selector calls CFRunLoopStop(CFRunLoopGetCurrent()) to resume the execution in connectInBackground(), allowing it to return normally:

- (void)connectionDidFinishLoading:(NSURLConnection *)connection {
    TTURLConnection* ttConnection = (TTURLConnection*)connection;
    ...
    // Resume execution where CFRunLoopRun() was called.
    CFRunLoopStop(CFRunLoopGetCurrent());
}

- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error {  
    TTURLConnection* ttConnection = (TTURLConnection*)connection;
    ...
    // Resume execution where CFRunLoopRun() was called.
 CFRunLoopStop(CFRunLoopGetCurrent());
}

This works well and it is thread safe because I bundled each connection's response and data as instance variables in the TTURLConnection subclass.

NSOperationQueue claims that leaving its maximum number of concurrent operations as NSOperationQueueDefaultMaxConcurrentOperationCount allows it to adjust the number of operations dynamically, however, in this case it always decides that 1 is enough. Since that is not what I want, I have changed the maximum number to 10 and it seriously hauls now.

The problem with this is that these threads (with the help of SpringBoard and DTMobileIS) consume all of the available CPU time and cause the main thread to become latent. In other words, once the CPU is 100% utilized, the main thread is not processing UI events as fast as it needs to in order to maintain a smooth UI. Specifically, table view scrolling becomes jittery.

Process Name  % CPU
SpringBoard   45.1
MyApp         33.8
DTMobileIS    12.2
...

While the user interacts with the screen or the table is scrolling the main thread's priority becomes 1.0 (the highest possible) and its run loop mode becomes UIEventTrackingMode. Each of the operation's threads are 0.5 priority by default and the asynchronous connections run in the NSDefaultRunLoopMode. Due to my limited understanding of how threads and their run loops interact based on priorities and modes, I am stumped.

Is there a way to safely consume all available CPU time in my app's background threads while still guaranteeing that its main thread is given as much of the CPU as it needs? Perhaps by forcing the main thread to run as often as it needs to? (I thought thread priorities would have taken care of that.)

UPDATE 12/23: I have finally started getting a handle on the CPU Sampler and found most of the reasons why the UI was becoming jittery. First of all, my software was calling a library which had mutual exclusion semaphores. These locks were blocking the main thread for short periods of time causing the scroll to skip slightly.

In addition, I found some expensive NSFileManager calls and md5 hashing functions which were taking too much time to run. Allocating big objects too frequently caused some other performance hits in the main thread.

I have begun to resolve these issues and the performance is already much better than before. I have 5 simultaneous connections and the scrolling is smooth, but I still have more work to do. I am planning to write a guide on how to use the CPU Sampler to detect and fix issues that affect the main thread's performance. Thanks for the comments so far, they were helpful!

UPDATE 1/14/2010: After achieving acceptable performance I began to realize that the CFNetwork framework was leaking memory occasionally. Exceptions were randomly (however, rarely) being raised inside CFNetwork too! I tried everything I could to avoid those problems but nothing worked. I am quite sure that the issues are due to defects within NSURLConnection itself. I wrote test programs which did nothing except exercise NSURLConnection and they were still crashing and leaking.

Ultimately I replaced NSURLConnection with ASIHTTPRequest and the crashing stopped entirely. CFNetwork almost never leaks, however, there is still one very rare leak which occurs when resolving a DNS name. I am quite satisfied now. Hopefully this information saves you some time!

+1  A: 

It sounds as though NSOperationQueueDefaultMaxConcurrentOperationCount is set to 1 for a reason! I think you're just overloading your poor phone. You may be able to mess around with threading priorities -- I think the Mach core is available and part of the officially blessed API -- but to me it sounds like the wrong approach.

One of the advantages of using "system" constants is that Apple can tune the app for you. How are you going to tune this to run on an original iPhone? Is 10 high enough for next years quad-core iPhone?

Stephen Darlington
I am actually optimizing the design for the user rather than the device. Our brains don't double in capacity every 2 years or do they?). Even if the machine is capable of fetching 1000 objects at the same time I wouldn't want to. If I throw the wrong amount of content at the user they will either become overwhelmed, or worse, bored.
James Wald
If you are consuming so much CPU the main (UI) thread gets starved - then you are not optimizing for the user, you are optimizing for network throughput. They are two very different things. Optimizing for the user means balancing the available resources, but most often means "keep the UI responsive". That is the primary rule above all, a user can sit for a few more seconds as long as the UI is still responsive.
Kendall Helmstetter Gelner
Forgot to answer your question about the original iPhone. I want it to go as fast as possible and consume all available CPU time. If it can't keep up with 10 connections at a time, so be it. I can always optimize later to squeeze out some extra performance. I need to just make sure that all of this is done without adding latency to the main thread.
James Wald
The default is actually `-1`. The docs say: "Specify the value `NSOperationQueueDefaultMaxConcurrentOperationCount` if you want the receiver to choose an appropriate value based on the number of available processors and other relevant factors."
ohhorob
I have tried that and it only ever creates 1 operation at a time on my 3GS. I still have acceptable UI performance, no noticeable jitters, with about 7 or 8 though. Not sure how the OS decides on "other relevant factors," I really wish the documentation in this area was more complete.
James Wald
+4  A: 

In practice you simply cannot have more than two or three background network threads and have the UI stay fully responsive.

Optimize for user responsiveness, it's the only thing a user really notices. Or (and I really hate to say this) add a "Turbo" button to your app that puts up a non-interactive modal dialog and increases concurrent operations to 10 while it is up.

Kendall Helmstetter Gelner
"add a "Turbo" button to your app" I never even thought of that but that is a sweet idea! I would like to design the app to go as fast as possible, up to 10 connections if possible, backing off as necessary to maintain UI performance. The "Turbo" button would be a great option for users on older devices who don't mind sacrificing a few frames per second to get the content faster. Thanks for the idea!
James Wald
A: 

James, although I haven't experienced your problem, what I've had success with is using the synchronous connection for downloading within an NSOperation subclass.

NSData *response = [NSURLConnection sendSynchronousRequest:request returningResponse:&urlResponse error:&requestError];

I use this approach for grabbing image assets from network locations and updating the target UIImageViews. The download occurs in NSOperationQueue and the method that updates the image-view is performed on the main thread.

ohhorob
I have actually tried using synchronous requests too but it appears to perform the same. That makes sense because Apple's documentation says that the synchronous connection is built on top of the asynchronous connection.
James Wald