SQLite for client-server

views:

1153

answers:

+4 Q:

SQLite for client-server

Hi, I've seen a couple of SQLite performance questions here on Stackoverflow, but the focus was on websites, and I'm considering using this DB in a client-server scenario:

I expect 1-10 clients for one server for now, could go up to 50 or more in the future.
slightly more reads than writes
the DB would sit behind a server process (i.e: not using direct DB access through a network)

Would using SQLite make the app less responsive as opposed to using PostgreSQL? My intuition tells me that it should be ok for these loads, but maybe someone has some practical experience with this kind of scenario.

+2 A:

You didn't mention what operating system and Postgres versions you are using. However, before considering change of database engine, try to do some logging and benchmarking your current database with typical usage, then optimize "heaviest" questions. And maybe your backend processing load makes DB question time irrelevant? ~~As SQLite is a file-based DBMS, concurrent access from multiple processes will degrade performance when client number grows up~~ (edited after comment)

Following question may be helpful: http://stackoverflow.com/questions/54998/how-scalable-is-sqlite

tomash 2009-08-24 10:29:49

The performance degredation will only happen with direct concurrent access -- but as the OP said, the accesses are indirect with one server process inbetween ... this might give a little slowdown, but no degredation.

Juergen 2009-08-24 10:35:03

Right, I missed the one server per multiple clients part. Anyway, if DB construction is so simple it can be migrated to SQLite (no DB-side processing, triggers and stuff), I don't think DBMS performance can be serious problem when DB is reasonably indexed.

tomash 2009-08-24 10:59:51

Haven't written any code yet. I'm working on the project's risk list atm, and scalability is one of them. Not using any Postgres, but the OSes will most likely be XP, Vista at first, later Linux.

rpg 2009-08-24 11:02:36

@rpg: As I said in my answer, all depends on the actual strain on the DB. I also would say, that SQLite is in a similar performance range as PostgreSQL, when you can cross out concurrent accesses -- so with your described setup I would claim (without knowing Postgress to much) that the difference will be small. The dependence of scalability on your actual coding will be most likely much higher. When you don't need all the higher DB-stuff (eg. triggers) I would recommend SQLite -- and 50 users/clients don't seam many to me!

Juergen 2009-08-24 11:42:23

@Juergen: maybe 50 users/clients don't seem much to you but you need to consider that every one of them can potentially generate multiple SQL statements that need to run concurrently with queries that are ran by other clients. In my experience - SQLite locking causes major delays and GUI timeouts even when ~10 users are working concurrently. It is true that SQLite is very fast database (when only a single user is working on it). It is also very easy to administer. However - SQLite is EVERYTHING BUT a scalable database solution (and I've learned this lesson the hard way)...

Liron Levi 2009-08-26 10:42:37

+1 A:

I would confirm to S.Lott's answer.

I dont know how SQLite performs in comparison to PostgreSQL, since I don't know any newer meassurements, but my own experience with SQLite in a rather similar environment is rather good.

The only thing that might cause troubles in my view is that you have rather many writes. But it all depends on the total number per second I would say.

Also your setting to have one server process is optimal for SQLite in my opinion -- so you circumvent its weakness in multi-tasking.

Juergen 2009-08-24 10:32:43

+5 A:

I did use SQLite for a major client/server product used with ~10 concurrent users and I deeply regret that decision. In my opinion - PostgreSQL is much more suitable for client/server scenarios than SQLite due to its fine locking granularity.

You simply can't get very far when the entire database is locked whenever someone needs to write something ..

I like SQLite very much (I even wrote a commercial utility for comparing SQLite databases - SQLite Compare but I don't think it fits the bill when you have client/server scenarios.

Even SQLite's author says that it should be used as a replacement for custom file formats and not as a full blown database server. I wish I took his advice more seriously..

Best Regards

Liron Levi

Liron Levi 2009-08-25 09:02:52

In my opinion you can only get into trouble with fine locking granularity when either you have multiple server processes accessing one DB file or your clients access the DB directly. When you have one server process -- how can you get locking problems (Ok, I must add, that you don't use threads ...)?

Juergen 2009-08-25 15:32:44

Actually - I have a single server that access the database in the name of multiple client processes. So what? In my case - every client is represented by a worker thread in the server and when you have multiple clients working on the server you'll also have multiple threads that are working on the database engine concurrently.Since it is very hard to predict in advance how database access will actually be performed (i.e., read vs. write access patterns) I strongly suggest not to make such a limiting decision (using SQLite instead of PostgreSQL) right from the beginning.

Liron Levi 2009-08-25 16:36:20

maybe sqlitening can answer your problem :)

tkg 2010-02-21 16:13:16

ansaurus

tags:

views:

answers:

SQLite for client-server

related questions