Is it bad to not use a DB but use in memory objects?

Question

I've been tasked to write a small app to be used by a single user. This app will pull in ~500 employee names/departments from our master employee DB. Then the user will enter like 5 fields for each employee. Those 5 fields will typically only change once a year, but could be once a month worst case. I only am supposed to keep track of 2 years worth at any given time.

I've looked at SQLite and SQL CE and I'm just not thrilled by either of them. SQL CE doesn't want to allow the data file to reside on a network share. (Only a single user but they store all their documents on their private share that is backed up daily).

SQLite seems like it would fit the bill better but it doesn't integrate as well into Visual Studio without wrappers or anything.

The other thing to consider is that our people are versed in MS' SQL Server and little else so having something that they understand vs SQLlite will be an important thing to my boss.

So my question is: What if I store the data in Objects in memory and serialize them to disk when saving. I've done a quick test and with 10k people (our use will only be 500-1000 max) and 10 years each (or 10 months if they update their data every month, highly unlikely) only caused my demo app to use 30MB of memory. Also populating that data was instantanous even with using GUID's to randomly fill all the strings. Is this a bad idea? Its a fairly simple app and in this case it seems ok to me.

Answer 1

By serializing the data you lose:

• SQL-style search
• ability to insert/update/delete individual records
• well known and understood
• not language specific
• accessibility for other and non-local applications

You gain

• easy to code
• simple backup (make sure you've thought about backup!)
• fewer dependencies

From your description of your goals and constraints I can't see any specific issues with your approach.

Another thought. It sounds like you're saving a simple table-like data structure, so you might want to thing about saving it in a human-readable form like a comma-separated-values file or even XML. That way you're not dependent on the language you're currently using.

Answer 2

Your own testing shows it should work. However, I would highly suggest abstracting out your data layer so that if it needs to be replaced, you can do so quickly and easily. If you find you want just a hint of SQL with your objects you can look at Linq to Objects

Answer 3

Over the years I have seen many small apps that are to be used by a single user. (For a long time, they all had Access back-ends for some reason). They invariably become large apps used by several users.

But the real question isn't how big it's going to be, but how is the data going to be used. Normally, people store data because they later want to see the data, and that usually means reports, and that means that a relational database system would be an awesome thing to have!

Answer 4

Storing the data in database has some other benefits like reading the data from other application or from a simple database editor. If any other application in future want to reuse the data it will be easier to do than deserializing from a java object. Also you have to deal with versioning of the objects too and querying will be difficult if you use it using java objects.

Answer 5

I see a few issues with the idea of persisting business data using object serialization:

These aren't necessarily show-stoppers for the idea, but rather something to think about...

1. The data can't be queried, reported or inspected. It's entirely opaquely captured by the application.
2. Debugging serialized data is harder than being able to view the corresponding data in a database, or even a format like CSV.
3. There's no atomicity - it possible to corrupt your entire "database" with one power failure or application crash.
4. If the data model changes, updating the existing persisted entities requires a version of the app that can read both the old and new format. With a database, you can just add a column (or sub table).
5. There's no clean way to implement concurrent access. What happens if more than one user want to view or edit the data?

One thing I've learned, is that small apps tend to grow and become "large apps". When organizations guess incorrectly about the potential value of an application, they tend to incur the costs of this kind of unexpected, organic growth later.

You also mention that you liked at SQLLite and didn't like it. What is it that you didn't like? What kinds of problems did you anticipate?

If you're just looking for a way to "cut corners" to get this done quicker - that may be ok in the short term - but be careful - these kinds of decisions have way of coming back to bite you.

Answer 6

If the original data is in SQL Server that you load in, won't one of the users versed in MS' SQL Server try to SELECT out the changes that your app is making? If you store it somewhere else they will never find it.

How will storing this data is something other than the original database help the business? Or will it hinder the business? you need to decide that, and then solve the problem from there. Not on how is it easier/best for me to do this assignment.

Answer 7

Store your data in a database! Decouple it using a Data Layer. You can use an ORM if you want or just a plain old repository but never use a text file, xml file, json file to store that kind of information.

You can use SQL Server if you have it, use it, it's there for a reason.

Answer 8

If you can store these 5 extra data fields in the master database, then that would seem ideal.

If you can't, you could probably just give them an Excel file with some kind of extension or macro for fetching data from the DB. Excel gives you many well known functions for printing, sorting, graphing and so on. You haven't told us much about what the app should do besides updating these 5 fields.

If you're dead set on writing a .net app for this, and not using a DB, at least put your data in XML format by using XML serialization.

Answer 9

From your justification, you can certainly use file but just seem to me that with a database backend, it will be more extensible. We often start with a project that we think we know all its requirements and will never change, but reality is not. In your design, if a user ask for an additional field, you will have to custom your code again to deal with the serialized fixed length/binary field. SQLite has good support and they are even LinQ provider for it.

I seriously think that a easiest solution to implement your project for the requirement above is to create Access Form. Define the field, and then use the wizard to create the form in access. With very little effort, you can get this whole task implemented with extensibility in mind.