Best datastore for storing 20,000 items to be queried by UISearchBar

Question

I need to store 20,000 records that are brief (map identifiers to names) and allow the user to search for one either by ID or name in a UISearchBar.

Approach 1 (Core Data): To conserve memory, I tried implementing this with CoreData with SQLite as the backend but found that storing 20,000 records is slow when the initial import is done. Then, as the user types in the UISearchViewBar letter by letter, the response is also noticeably slow.

Approach 2 (NSMutableArray): When I implemented the same thing in memory using NSMutableArray, the response time on the UISearchBar is very quick.

In the first approach, only a small subset of records is in memory at a time but the response time is slow. In the 2nd approach, The 20,000 items take up just 700KB (under a MB) in memory and the response time is fast.

Since Core Data provides persistant storage and makes querying very flexible with predicates, I'd like to use if it possible. Does anyone have suggestions on if there's a way of to still use Coredata but get quicker results?

Thanks

Answer 1

Doing keystroke queries on CoreData is just not a good idea.

CoreData was not meant to be hit that hard repeatedly. You have to remember the device that's actually running this database. Put CoreData on a 10 year old desktop and see how fast it runs.

Also, I am pretty sure there is a way to use NSPredicate with NSArrays... no links for you but I think I've seen it done before.

Also yes, the initial import into CoreData is bone crushingly slow. IIRC, it's 2 separate queries per CoreData record creation.

If you want to lower the memory consumption, just call up what you need in the objects to do the searching, then pull the whole object when you actually need it.

Answer 2

Why not do both? Use Core Data for persistence. Pre-fetch the managedObjectID and properties into in-memory caches that you can use for searching with the search bar. When the item is selected, you can use the managedObjectID to fetch the Entity from the MOC.

Use NSArray's filteredArrayUsingPredicate: and NSMuatableArray's filterUsingPredicate