Why would someone need an in-memory database?

Question

I read that a few databases can be used in-memory but can't think of reason why someone would want to use this feature. I always use a database to persist data and memory caches for fast access.

Answer 1

A common use case is to run unit/integration tests.

You don't really care about persisting data between each test run and you want tests to run as quickly as possible (to encourage people to do them often). Hosting a database in process gives you very quick access to the data.

Answer 2

Does your memory cache have SQL support?

How about you consider the in-memory database as a really clever cache?

That does leave questions of how the in-memory database gets populated and how updated are managed and consistency is preserved across multiple instances.

Answer 3

In-memory databases are roughly at least an order of magnitude faster than traditional RDBMS for general purpose (read side) queries. Most are disk backed providing the very same consistency as a normal RDBMS - only catch the entire dataset must fit into RAM.

The core idea is disk backed storage has huge random access penalties which does not apply to DRAM. Data can be index/organized in a random access optimized way not feasible using traditional RDBMS data caching schemes.

Answer 4

Applications, which require real time responses would like to use an in memory database, perhaps application to control aircraft, plants where the response time is critical

Answer 5

They are used as an advanced data structure to store, query and modify runtime data.

Answer 6

Searching for something among 100000 elements is slow if I don't use tricks like indexes. Those tricks are already implemented in a database engine (be it persistent or in-memory).

A in-memory database might offer a more efficient search feature than what you would be able to implement yourself over a self-written collection of arrays and objects.

Answer 7

You may need a database if several different applications are going to access the dataset. A database has a consistent interface for accessing / modifying data, which your hash table (or whatever else you use) won't have.

If a single program is dealing with the data, then it's reasonable to just use a data structure in whatever language you are using though.

Answer 8

Cache is also a kind of database, like a file system is. 'Memory cache' is just a specific application of an in-memory database and some in-memory databases are specialized as memory caches.

Other uses of in-memory databases have already been included in other answers, but let me enumerate the uses too:

1. Memory cache. Usually a database system specialized for that use (and probably known as 'a memory cache' rather than 'a database') will be used.
2. Testing database-related code. In this case often an 'in-memory' mode of some generic database system will be used, but also a dedicated 'in-memory' database may be used to replace other 'on-disk' database for faster testing.
3. Sophisticated data manipulation. In-memory SQL databases are often used this way. SQL is a great tool for data manipulation and sometimes there is no need to write the data on disk while computing the final result.
4. Storing of transient runtime state. There are application that need to store their state in some kind of database but do not need to persist that over application restart. Think of some kind of process manager – it needs to keep track of sub-processes running, but that data is only valid as long as the application and the sub-processes run.