NSCFArray mutated while being enumerated - but I don't use enumerators

Question

One thing I really don't like about Objective-C is that it's very difficult to find out what's going on under the hood.

The latest problem this has caused me is a random exception that occurs in random places in my code and breaks everything - it's the one listed above, about concurrency problems with NSArray.

The thing is that I never use NSEnumerator myself, instead if I do iterate through an array I do it like this:

for (int i = 0; i < [array count]; i++)
{
    id obj = [array objectAtIndex:i];
    //blah blah
}

The question is whether or not some Obj-C code creates an NSEnumerator under the hood and I just can't tell, or whether I should spend hours tracing through my code trying to figure this one out.

Any of the following could potentially be called on the array:

objectAtIndex:
exchangeObjectAtIndex: withObjectAtIndex:
replaceObjectAtIndex: withObject:

It doesn't seem like any of those, since they are index operations, should require enumerators. But do they? Do any of the functions in NSMutableArray?

Also, I am indeed using two threads in my program - and when I added in the other thread the problem started. I just don't want to mess around with putting locks and things in my code until I know exactly where the problem is.

PLEASE don't give me some comment telling me I'm lazy, that's not the question here. I will put in locks and better thread safety if first know the problem is from my end. I don't want to waste time doing all that if it's because of Apple's API. The question is whether or not it is in fact from my end. That's it. I want to know when NSArray creates an NSEnumerator.

Jerret Hardy gets the win for this one: http://developer.apple.com/mac/library/documentation/Cocoa/Conceptual/Multithreading/ThreadSafetySummary/ThreadSafetySummary.html provides a reasonable explanation that I simply can't use NSMutableArray in multiple threads. Fair enough, I guess.

EDIT: Looks like the problem was being caused by [NSMutableArray addObject:], which was being called in another thread while [NSMutableArray objectAtIndex:] was being called. The generated NSEnumerator exception was throwing me off - if anyone else gets this exception then they should check for any concurrent NSMutableArray access and mutation that can occur in their program, with or without NSEnumerator being involved.

Answer 1

I am indeed using two threads in my program - and when I added in the other thread the problem started.

QED.

I just don't want to mess around with putting locks and things in my code until I know exactly where the problem is.

Sucks to be you, I guess.

Answer 2

First off, yes, tons of things use enumerators under the hood. Any API you pass a collection into could very well use an enumerator, as is the case with everything single other framework that has enumerators.

Next off, the error you are seeing is because you have 2 threads and are mutating a collection on one while there is an inflight enumerator. Thread safety is not an optional thing you add when you need it. You either need to deal with it, or not use threads.

The specific issues you are hitting is documented in the Cocoa Threaded Programming Guide:

Mutable Versus Immutable Immutable objects are generally thread-safe; once you create them, you can safely pass these objects to and from threads. Of course, when using immutable objects, you still need to remember to use reference counts correctly. If you inappropriately release an object you did not retain, you could cause an exception later.

Mutable objects are generally not thread-safe. To use mutable objects in a threaded application, the application must synchronize access to them using locks. (For more information, see “Atomic Operations”). In general, the collection classes (for example, NSMutableArray, NSMutableDictionary) are not thread-safe when mutations are concerned. That is, if one or more threads are changing the same array, problems can occur. You must lock around spots where reads and writes occur to assure thread safety.

Even if a method claims to return an immutable object, you should never simply assume the returned object is immutable. Depending on the method implementation, the returned object might be mutable or immutable. For example, a method with the return type of NSString might actually return an NSMutableString due to its implementation. If you want to guarantee that the object you have is immutable, you should make an immutable copy.

So in other words, unless it is specifically documented as otherwise a mutable object is not thread safe and using one on multiple threads can (and will) result in all sorts of bugs. The fact that you are seeing this particular enumerator issue just a symptom, if the framework was not using an enumerator it just wouldn't be able to detect the mutation, and would instead smash the collections data structure and crash.

Answer 3

"Mutable objects can be used if you synchronize access." - so it should not be a problem if you use it only in one thread without a lock.

I wonder what you do in the second thread that references the array in the first one.

Jolly aka Patrick

Answer 4

Though it's hard to tell from what you've provided, there is an implicit assumption about 'what' is using, and possibly accessing, the NSMutableArray. If there's 'another' reference to the array out there, and there very obviously is, then it's hard to say what's actually causing the problem.

In short, you've made the assumption that something you're doing, ostensibly with the NSMutableArray mutating methods you list, is what's 'creating' a NSEnumerator. What I think is much more likely is you are either passing the NSMutableArray to another method / function and it's enumerating the contents via NSEnumerator / for(... in ...) { }. Even more likely, IMHO, is that there's a reference to the NSMutableArray that's 'live' and being used in the another thread. Therefore the most likely explanation is that you're assumption that Cocoa is 'creating' a NSEnumerator behind your back on you is totally wrong.. The other thread is enumerating the contents of the array through its normal course of execution, which is (obviously) when you're mutating it. I can almost guarantee, from years of multi-threaded programming experience, that the code path in the other thread that's actually enumerating the array, not the thread that's doing the mutating, is in a part of your code that you thought could "never, ever" be called / executing during the time you're doing the mutating. Or that a pointer to the array managed to find its way deep in to the system where you would never, ever expect it to be. Even doing something simple like NSLog(@"array: %@", array); in the other thread is likely to use a NSEnumerator to print out the arrays contents, and therefore randomly throw a mutating exception if the timing is just right, which invariably happens much more often than you'd think.

In other words, absolutely no other thread can be using, accessing, enumerating, etc the NSMutableArray that you happen to be mutating. You must absolutely, unconditionally, and provably guarantee that no other thread is using the NSMutableArray, in any way what-so-ever, or things will not work correctly. Anything short of this will cause problems and will likely crash in strange and mysterious ways... which sounds like what's happening to you now.

The very short version: The fact that you thought you didn't need to provide proper multi-threaded concurrency control virtually guarantees that the problem is on your end, not Apples API, and you really do need to do all that complicated locking stuff you're trying to avoid.