NSURLConnection crashing under 10.5.7

Question

I have a little app that downloads stock prices and was working perfectly (for years) until my recent upgrade to 10.5.7. After the upgrade, the program would crash on this call:

NSString *currinfo = [NSString stringWithContentsOfURL:[NSURL URLWithString:[NSString stringWithFormat:@"http://finance.yahoo.com/d/quotes.csv?s=%@&f=l1c1p2", escsymbol]]];

Oddly, the crash doesn't happen right away. This line of code is called many times, with no problems, and then the program eventually fails after 1-2 hours due to a crash on this call.

I originally had a long post here describing my attempts to investigate this problem. I received two suggestions: (i) make the call asynchronous (probably better anyway) and (ii) use NSZombieEnabled to investigate the possibility that an Objective-C object is getting deallocated early (this comment was made in response to stack traces showing failure in objc_msgSend).

I spent a good deal of time making the call asynchronous (using [[NSURLConnection alloc] initWithRequest:theRequest delegate:self]) and this didn't help. The program still failed eventually, usually after 10-15 minutes. During this interval before failure, many asynchronous calls were made without any problem, data was returned, etc. Everything was fine. Then the program crashed suddenly again.

I then turned on NSZombieEnabled. Sure enough, when the program eventually crashed I got the message:

-[CFArray count]: message sent to deallocated instance 0x16b90bd0

"info malloc 0x16b90bd0" then yielded:

0: 0x93db810c in malloc_zone_malloc
1: 0x946bc3d1 in _CFRuntimeCreateInstance
2: 0x9464a138 in __CFArrayInit
3: 0x946cd647 in _CFStreamScheduleWithRunLoop
4: 0x932d1267 in _Z16_scheduleRStreamPKvPv
5: 0x946bf15c in CFSetApplyFunction
6: 0x932b0e2b in CFNSchedulingSetScheduleReadStream
7: 0x9331a310 in _ZN12HTTPProtocol19createAndOpenStreamEv
8: 0x9332e877 in _ZN19URLConnectionLoader24loaderScheduleOriginLoadEPK13_CFURLRequest
9: 0x9332d739 in _ZN19URLConnectionLoader26LoaderConnectionEventQueue33processAllEventsAndConsumePayloadEP20XConnectionEventInfoI12XLoaderEvent18XLoaderEventParamsEl
10: 0x9332dbdd in _ZN19URLConnectionLoader13processEventsEv
11: 0x932d8dbf in _ZN17MultiplexerSource7performEv
12: 0x946ba595 in CFRunLoopRunSpecific
13: 0x946bac78 in CFRunLoopRunInMode
14: 0x9058c530 in +[NSURLConnection(NSURLConnectionReallyInternal) _resourceLoadLoop:]
15: 0x90528e0d in -[NSThread main]
16: 0x905289b4 in __NSThread__main__
17: 0x93de8155 in _pthread_start
18: 0x93de8012 in thread_start

I am no expert in reading stack traces, but doesn't this trace indicate a problem in Apple code, rather than my code? Or could I somehow be responsible for de-allocation of the CFArray in question? Is there any way for me to further investigate the cause of the problem?

---

(Here's the rest of my original post)

Seeing that stringWithContentsOfURL is deprecated, I switched to this code:

pathURL = [NSURL URLWithString:[NSString stringWithFormat:@"http://finance.yahoo.com/d/quotes.csv?s=%@&f=l1c1p2", escsymbol]];

NSURLRequest *request = [NSURLRequest requestWithURL:pathURL cachePolicy:NSURLRequestReturnCacheDataElseLoad timeoutInterval:30.0];

responseData = [ NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];

NSString *currinfo = nil;

if ([error code]) {  dNSLog((@"%@ %d %@ %@ %@", [ error domain], [ error code], [ error localizedDescription], request, @"file://localhost/etc/gettytab"));  }

This didn't help. The program still crashes on the sendSynchronousRequest line after an arbitrary length of time, with this information in the debugger:

0   0x93db7286 in mach_msg_trap
1   0x93dbea7c in mach_msg
2   0x946ba04e in CFRunLoopRunSpecific
3   0x946bac78 in CFRunLoopRunInMode
4   0x932b53eb in CFURLConnectionSendSynchronousRequest
5   0x905dca4b in +[NSURLConnection sendSynchronousRequest:returningResponse:error:]

...etc.

The real crash might actually be in a different thread:

0   libobjc.A.dylib                 0x965c3688 objc_msgSend + 24
1   com.apple.CoreFoundation        0x946cc581 _CFStreamSignalEventSynch + 193
2   com.apple.CoreFoundation        0x946ba595 CFRunLoopRunSpecific + 3141
3   com.apple.CoreFoundation        0x946bac78 CFRunLoopRunInMode + 88
4   com.apple.Foundation            0x9058c530 +[NSURLConnection(NSURLConnectionReallyInternal) _resourceLoadLoop:] + 320
5   com.apple.Foundation            0x90528e0d -[NSThread main] + 45
6   com.apple.Foundation            0x905289b4 __NSThread__main__ + 308
7   libSystem.B.dylib               0x93de8155 _pthread_start + 321
8   libSystem.B.dylib               0x93de8012 thread_start + 34

which I presume is the thread spawned to download the URL. By the way, the error handling code works fine--when I intentionally cause an error by disconnecting from the internet, the error is just reported in the console and the program doesn't crash.

This is incredibly frustrating. I would be very happy to spend as much time as necessary tracking down the problem, but I'm kind of at the limits of my knowledge with gdb and especially with assembly language. I don't know how to find out what is the actual problem for the Foundation code. At first I thought that maybe the autoreleased NSString escsymbol is somehow being deallocated, but sending it a retain message didn't help. If this were the case, how could I prove it?

Is anybody else having this problem?

Answer 1

I'd suggest not using synchronous URL connections. It does require some code restructuring, but it's really bad behaviour to block the main thread on network. (Assuming you're doing this in the main thread).

Also, I'm guessing it's code that Apple is planning to deprecate or stop maintaining, which might be what you're seeing here.

Hope that helps….

Answer 2

We're seeing the same issue, using both the non-deprecated stringWithContentsOfURL and sendSynchronousRequest (in different places). We see the same type of behavior; the call can work fine for many iterations, and then will randomly fail (on identical requests).

Given that the synchronous calls are built on top of the async mechanism, there's no reason these calls shouldn't still be working properly. Unfortunately, we don't have a solution yet either.

Answer 3

Colin, thanks for letting me know that you see this problem too--lets me know I'm not crazy. It seems impossible that Apple could have broken something so fundamental.

Looking more closely, it actually seems that it is the main thread that crashes, not the thread that is trying to use NSURLConnection. Here is the trace from that thread:

Thread 1 Crashed:
0   libobjc.A.dylib                 0x965c3688 objc_msgSend + 24
1   com.apple.CoreFoundation        0x946cc581 _CFStreamSignalEventSynch + 193
2   com.apple.CoreFoundation        0x946ba595 CFRunLoopRunSpecific + 3141
3   com.apple.CoreFoundation        0x946bac78 CFRunLoopRunInMode + 88
4   com.apple.Foundation            0x9058c530 +[NSURLConnection(NSURLConnectionReallyInternal) _resourceLoadLoop:] + 320
5   com.apple.Foundation            0x90528e0d -[NSThread main] + 45
6   com.apple.Foundation            0x905289b4 __NSThread__main__ + 308
7   libSystem.B.dylib               0x93de8155 _pthread_start + 321
8   libSystem.B.dylib               0x93de8012 thread_start + 34

Does this tell anybody anything? What is NSURLConnection(NSURLConnectionReallyInternal)? What does _resourceLoadLoop do?

Answer 4

Crashes in objc_msgSend are usually due to improper object life-cycle (i.e. object has been released and it's being sent a message).

Run your code with NSZombieEnabled to figure out if this is indeed your problem and to see which object is being release early.

Answer 5

At The Omni Group, we are seeing this new crash in 10.5.7 in all of our NSURL-using applications too, in asynchronous use (so I think the synchronous thing is a red herring). We've also had it happen in TextMate (presumably in the software update).

I've logged Radar 6932684 on this issue, and if you are seeing it too, I strongly suggest you all report it, with whatever details you can gather.

Answer 6

This seems to be related to redirects. My app downloading about 500 megs of data directly (few hundred separate files) doesn't crash. Same app downloading smaller set of urls, all of which are redirected will randomly crash several times in exactly that point (restart resumes, and restarting over and over again will actually result in successful download).

EDIT: BTW, Colin's suggestion about reimplementing redirects doesn't seem to work for NSURLDownload :(.

EDIT2: Ok, this seems to be a race condition. Adding cerr << "redirect" << endl; in the callback "fixes" it for me with NSURLDownload. Sleeping for 1 second or locking a local static mutex have no effect however...

Answer 7

I think Eugene's reply in this thread is properly describing the problem; after some testing, here is what I've concluded seems to be going on, with hopefully some details to help others stuck with this issue:

Redirecting URLs will periodically cause failures. This happens both in the sync and async usages of NSURLConnection. I've created a test project to track down this bug, and this crash will consistently occur (typically between 25-500 iterations). Running the same test on 10.5.6 or without redirecting URLs does not fail (have run it up to 20,000 iterations).

There are two possible work-arounds:

1. Don't use redirecting URLs:
   Obviously this isn't always possible, but if it is a known URL, you can still use the simple calls (like stringWithContentsOfURL:) and this will work fine. Dennis, in your case, the proper server URL is download.finance.yahoo.com, not finance.yahoo.com, so I believe this would fix your particular problem. Using curl you can see that you get a 301 redirect when you hit the latter address.
2. Use the async calls and implement connection:willSendRequest:redirectResponse:
   If you implement even the most basic handling of this delegate method, everything seems to work. If you leave out this call (and thus let the system use its default implementation), you'll get the crash. For a basic implementation, just return the request that is passed in:

- (NSURLRequest *)connection:(NSURLConnection *)connection 
             willSendRequest:(NSURLRequest *)request 
            redirectResponse:(NSURLResponse *) redirectResponse
{
    return request;
}

All of this seems to suggest to me that there is something broken in Apple's implementation in 10.5.7, but if anyone else has any insights as to what might be going on, please chime in.

I have submitted a bug with my test project to Apple as rdar://6936109 and referenced tjw's report (Radar 6932684).

Answer 8

I have also seen this exact same crash appearing in 10.5.7 in several applications.

Given that implementing the simplest connection:willSendRequest:redirectResponse: delegate method will solve the problem, I think might be highly related to radar #6700222.

Answer 9

Does anyone know if this affects the iPhone as well?

Answer 10

i spend hours yesterday handling this issue - with no positive result. now, i searched for it again and found your post and i thought "da*n thats it"... i come back to you after trying your solutions... regards torsten

Answer 11

For what it's worth, this seems to be fixed on 10.5.8 --- but it's been replaced by a new bug which prepends the redirect's response-body to the real response about 2% of the time. (Easy to reproduce by spinning on -stringWithContentsOfURL:, but we've also seen it in the wild.) Reported that bug as radar #7169953.

Answer 12

FYI, this is still happening on 10.6.1, just at a much-reduced rate (my test shows it down to one request in thousands, rather than 2/100 as it was in 10.5.7).