Fastest way to store/retrieve a dictionary - SQL, text file...?

Question

Hi all,

This is a really really super dumb question, so I apologise, but I'd be grateful for some advice.

I've got a text file of words and word frequencies. It's very large - theoretically we're talking millions of rows.

I just want to retrieve values from the file, and do it as quickly and efficiently as possible (for a web app, in Django).

My question is: what is the best way to store and retrieve the values? Should import them into SQL? Or keep the file and use grep? Or put them into a JSON dictionary...? Or some other way?

Sorry for the dumb question, would be very grateful for advice!

Answer 1

putting them in a json dictionary would be a bad idea unless you want to load the entire thing into memory when you search through it.

sql is basically built for this kind of thing, so i would use that. a file and grep would also work fine, but you wouldn't gain any benefits from indexing etc that sql would give you.