complete beginner trying to create a flat file database in python

Question

trying to keep it stupidly simple. Is it a bad idea to move a txt file into and out of a python list? the txt files will probably get to about 2-5k entries. what is the preferred method to create a simple flat file databse?

Answer 1

It might or might not be a bad idea. It depends on what you are trying to achieve, how much memory you have and how big those lines are on average. It also depends on what you are doing with that data. Maybe it is worth it to read and process file line by line? In any case, database assumes indexes, what are you going to do with a list of strings without an index? You cannot search it efficiently, for example.

In any case, if you feel like you need a database, take a look at SQLite. It is a small embedded SQL server written in C with Python interface. It is stable and proven to work. For example, it is being used on iPhone in tons of applications.

Answer 2

If you're looking for a very simple file database, maybe you should look at the shelve module. Example usage:

import shelve

with shelve.open("myfile") as mydb:
    mydb["0"] = "first value"
    mydb["1"] = "second value"
    # ...