views:

146

answers:

3

I have been toying with this idea for quite awhile now, but haven't seen any information on people doing it. I have a small website project where I need to load and modify 1 object. This object is pretty simple, and shouldn't be more than a few kb. Instead of running a DB for this small amount of data, why not just use pickle and/or shelve to save this data, and load it? I am planning on using a micro web framework like Bottle or Flask for the project.

Are there any reasons to not use this method to load the data? It will only load the pickle file at the time Apache starts up, so I don't think speed will be effected (faster than querying a db).

Thanks for any input!

+1  A: 

I wouldn't write a pickled string to a file directly. There are too many low-level details to worry about. Check out Durus, ZODB, or this post from FriendFeed about storing Python objects in MySQL.

Don't discard relational databases, though, they give you a lot of bang right out of the box (even for simple projects).

Sean Woods
Thanks for the links. If I'm going to use any DB, it's going to be MongoDB. I did not think about the problem of the framework running multiple python processes and not having access to the same object if something is modified. It would have to trigger some kind of flag and have each process reload the pickled file? What are some other "low-level details to worry about"? Thanks!
Shane Reustle
-1: "There are too many low-level details to worry about"? Can you be more specific? We do this all the time.
S.Lott
+3  A: 

There is no reason why you can't implement object persistence via the standard Python pickle or shelve modules. Just make sure your objects are cleanly and securely picklable. Scalability may become a concern if your site grows beyond your current scope, but until then your idea should work just fine. If that day comes, the next obvious step would be to consider using Python's excellent SQLite module that comes pre-packaged with recent versions of the language.

Kevin Jacobs
Something like MongoDB would be an easier transition from Pickle than would SQL be.
David M.
Yep, there's nothing wrong with getting it up and running with pickle. And who knows, maybe pickle will suffice for your problem. If not, you can always switch over to a database when it becomes a problem. Keep it simple until it demands more.
wybiral
@David M I have worked with SQL a lot before so either wouldn't be a problem, I would just choose to use MongoDB to get more experience in NoSQL :)
Shane Reustle
I'm not saying that SQL is hard, just that MongoDB et al are more similar to pickle.
David M.
Re SQL vs. NoSQL: there is no generic right answer to this debate. It all devolves into personal preference and religion for all but the most extreme use-cases. Best to give both a try and benefit from the broader perspectives gained only from experience.
Kevin Jacobs
+1  A: 

In addition to the concurrency issues you are already aware of, you also must ensure that the file is always in a consistent state. For example, if the server crashes in the middle of writing the file, what happens then? It's a case you need to consider and implement a solution for if you go this route.

Nathan Davis