Is Core Data more efficient than custom indexing for "as-you-type" searching ?

Question

Hello everyone,

Actually, I'm working on a Core Data based iPhone application. I have two entities which contain more than 200000 rows each and I'm experiencing some performance issues during retrieval of data. For each fetch request, I must wait between 1 and 2 seconds before getting results. I'm thinking of implementing a custom search engine to index my data but the problem is that the whole database is editable. The contents can be changed at anytime so indexing a dynamic content database is stupid.

I'm wondering if Core Data is efficient enough to provide instant search. In the Apple documentation, a thousand rows entity is considered as small. Is that right?

Does anyone have a solution to improve the speed of Core Data...? Or should I implement my own search engine?

The goal is to provide an instant search, a search as you type mecanism.

[UPDATE] Below is a snippet of one of my fetch request...

NSString *predicateString = [NSString stringWithFormat:@"^(.*\\b)?%@(\\b.*)?$", searchString];
NSString *predicate = [NSString stringWithString:@"text MATCHES[cd] %@"];
NSArray *arguments = [NSArray arrayWithObjects:predicateString, nil];

NSFetchRequest *fetchRequest = [[[NSFetchRequest alloc] init] autorelease];
[fetchRequest setEntity:[NSEntityDescription entityForName:@"MyEntity" inManagedObjectContext:[[HYDataManager instance] managedObjectContext]]];
[fetchRequest setPredicate:[NSPredicate predicateWithFormat:predicate argumentArray:arguments]];
[fetchRequest setSortDescriptors:[NSArray arrayWithObjects:[[NSSortDescriptor alloc] initWithKey:@"length" ascending:YES], [[NSSortDescriptor alloc] initWithKey:@"subEntity.attr1" ascending:YES], [[NSSortDescriptor alloc] initWithKey:@"subEntity.attr2" ascending:YES], nil]];
[fetchRequest setResultType:NSDictionaryResultType];
[fetchRequest setReturnsDistinctResults:YES];
[fetchRequest setPropertiesToFetch:[NSArray arrayWithObject:@"subEntity"]];

NSFetchedResultsController *controller = [[NSFetchedResultsController alloc] initWithFetchRequest:fetchRequest managedObjectContext:[[HYDataManager instance] managedObjectContext] sectionNameKeyPath:nil acheName:nil];

Answer 1

You can try several things.

1) try using setFetchBatchSize: to reduce the working set of data in your app. In combination with an NSFetchedResultsController this will transparently faults batches on demand, showing the objects on your table as needed.

2) verify if you can use setResultType: with NSManagedObjectIDResultType. This will return only the ID of the matching objects; if you need to access just one or a few of them after performing the fetch request, then the additional overhead is really small, but the fetch is much faster. If you need to access all of the returned objects, then this is not the way to go.

3) if you instead need to retrieve the properties stored in your objects, use setPropertiesToFetch: as suggested by Hunter to retrieve only the ones you really need

4) if your model includes subentities of your entity, then verify if you can use setIncludesSubentities: passing NO as argument.

5) if you do not need to deal with all of the objects matching the predicate associated to your fetch request, then use setFetchLimit: to retrieve a fixed number of objects.

Hope this helps.

Answer 2

Have you tried marking all editable elements indexed in the data model? As you said it seems really odd to index everything but Core Data should be smart enough to handle this appropriately.