subarrayWithRange with large data

Question

iPad application

I have a large text file which I want to divide into several pieces and process them in a loop. I used the following code:

NSArray * contentArray = [stringFromFile componentsSeparatedByString:@" "];

for(...){
    contentRange = NSMakeRange(lastPosition , newLength);
    NSArray * subContentArray = [contentArray subarrayWithRange:contentRange];
    NSString *subContent = [subContentArray componentsJoinedByString:@" "];
    ....
    //Process sub content here...

}

After running, I receive malloc error, code 12

Observing the Activity monitor, the memory and the VM size increases and until system memory is exhausted and the application crashes.

Is there any way to prevent this?

Answer 1

One way to work around this issue will be to use a custom NSAutoreleasePool to constantly clear temporarily allocated memory. Like so:

NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSUInteger i=0;

for (...) {
  ... // your method contents

  if (++i % 10 == 0) {
    // you might want to play with the frequency of the releases, depending on the size of your loop
    [pool release];
    pool = [[NSAutoreleasePool alloc] init];
  }
}

[pool release];

Answer 2

The autorelease pool thing will take care of the memory problem for the most part, but if the file is any size it might be slow. It seems like scanning the original string in a loop getting substrings directly from the string would be faster. Also use at least 1/2 the memory, since you duplicate the string in the components array.

The code would use rangeOfString on the original string, until a substring of appropriate length was found, then process the substring, and onto the next one.

iPhone 3G phones only have about 5 - 15 MB of memory total for your application. You are on an iPad, though I see which does give you about 4x that amount.