Bulk update & occasional insert (coredata) - Too slow

Question

Update: Currently looking into NSSET's minusSet links: http://stackoverflow.com/questions/1475636/comparing-two-arrays

Hi guys,

Could benefit from your wisdom here..

I'm using Coredata in my app, on first launch I download a data file and insert over 500 objects (each with 60 attributes) - fast, no problem.

Each subsequent launch I download an updated version of the file, from which I need to update all existing objects' attributes (except maybe 5 attributes) and create new ones for items which have been added to the downloaded file.

So, first launch I get 500 objects.. say a week later my file now contains 507 items..

I create two arrays, one for existing and one for downloaded.

    NSArray *peopleArrayDownloaded = [CoreDataHelper getObjectsFromContext:@"person" :@"person_id" :YES :managedObjectContextPeopleTemp];
NSArray *peopleArrayExisting = [CoreDataHelper getObjectsFromContext:@"person" :@"person_id" :YES :managedObjectContextPeople];

If the count of each array is equal then I just do this:

    NSUInteger index = 0;
if ([peopleArrayExisting count] == [peopleArrayDownloaded count]) {
 NSLog(@"Number of people downloaded is same as the number of people existing");
 for (person *existingPerson in peopleArrayExisting) {
  person *tempPerson = [peopleArrayDownloaded objectAtIndex:index];
  // NSLog(@"Updating id: %@ with id: %@",existingPerson.person_id,tempPerson.person_id);
  // I have 60 attributes which I to update on each object, is there a quicker way other than overwriting existing?
  index++;
 }
} else {
 NSLog(@"Number of people downloaded is different to number of players existing");

So now comes the slow part.

I end up using this (which is tooooo slow):

      NSLog(@"Need people added to the league");
  for (person *tempPerson in peopeArrayDownloaded) {
   NSPredicate *predicate = [NSPredicate predicateWithFormat:@"person_id = %@",tempPerson.person_id];
   // NSLog(@"Searching for existing person, person_id: %@",existingPerson.person_id);
   NSArray *filteredArray = [peopleArrayExisting filteredArrayUsingPredicate:predicate];
   if ([filteredArray count] == 0) { 
    NSLog(@"Couldn't find an existing person in the downloaded file. Adding..");
    person *newPerson = [NSEntityDescription insertNewObjectForEntityForName:@"person" inManagedObjectContext:managedObjectContextPeople];

Is there a way to generate a new array of index items referring to the additional items in my downloaded file?

Incidentally, on my tableViews I'm using NSFetchedResultsController so updating attributes will call [cell setNeedsDisplay]; .. about 60 times per cell, not a good thing and it can crash the app.

Thanks for reading :)

Answer 1

I'll begin by saying that I'm still new to using the Core Data framework, but my guess is that your problem lies in the for loop you've posted.

If you look at your loop, each time it executes it creates a new NSPredicate object and then filters your existing array looking for matches. On a small data set this technique would work with seemingly small performance losses; however, with your large data set you will end up spending a lot of time creating NSPredicate objects that only differ in the name you've provided. I would suggest that you look at how to create a single predicate and then use variable substitution to perform the search. For information about variable use in predicates check out: http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/CoreData/Articles/cdImporting.html#//apple_ref/doc/uid/TP40003174

As a side note, you may also consider how you've sorted your data and how you are performing the search operation. And another thing I noticed is that you don't release your NSPredicate object, so you're just tossing memory away too.