Where is the future of databases?

Question

I'm a bit frustrated with my MySQL database at the moment, so I've been thinking about all the things I'd like to see in the database of the future. But I thought it would be fun to hear other people's thoughts too--I'm not a pro by any means.

Answer 1

Cloud computing might be interesting and Object Database (ODBMS)

Answer 2

What are you frustrated with? Performance? I think if you were more clear on what you find wrong with a current Relational Database, we might be able to have a discussion (although that isn't really what stackoverflow is about, so maybe a better question would be "What one feature is the most lacking in current relational databases?")

Answer 3

Flat files!

Answer 4

Some solution like SQL Server Data Services could be the future of storage paradigm

Answer 5

I think much of what's missing in current database usage isn't in the database itself; it's in the ORMs, frameworks and applications that are interacting with the databases. It's extremely common to see code that just doesn't know how to interact with a database in a way that leverages the capabilities of the database.

To turn that around and try and answer your question; what I think is missing from databases is an easy way to create an actual API between a database and the applications using it. SQL isn't really an API that's useful here, because it means intimately tying the application in with the structure of the database. An example of a useful API is creating stored procedures/functions for every action that the applications need to perform, but that currently requires a lot of extra work in the application.

What I'd also like to see is more powerful procedural languages in the database that still make it very easy to interact with the database. Postgres allows you to use languages like Perl and Python inside the database, but the interaction between the language and the database is nowhere near as clean and easy as it is in plpgsql. On the other hand, plpgsql is not a very powerful language; it's only real benefit is how easy it is to interact with the database. Having something like a more powerful plpgsql would make it easier to create a database API, at least on the database side.

Answer 6

WebCould is another new technology. Cloud computing is going to change the way we deal with Backend systems, or in most of the cases we might not need a database for the next generation(Cloud aware) applications.

Of course I am not talking about enterprise level business applications.

Answer 7

abandoning SQL and implementing the relational model correctly!

Answer 8

decibel: Which ORMs and frameworks are you specifically talking about when you say that they don't know how to interact with the database? I'd be very interested to hear your opinions as I work with a number of them.

Answer 9

The largest problem with current database systems (imho) is the fact that you have to think in terms of the tables and structures of the database system. You can't work in objects or save them directly (yes you can build abstractions above and around them). I think the future of databases is probably going to a more object-centric view (think like serialization). I would love to spend more time thinking in terms of my problem space objects rather than how should this normalize to a relational system...

Answer 10

Depending on the application you could get away with using a full text search engine (apache lucene / apache solr) to store your data. With this solution you can configure which "columns" or data fields are indexed and simply search all fields with one call. No more generating sql statements based on attribute values in a data object.

Also, if you want to add additional fields to your data index (or storage) you can simply add the field and re-index. It's likely you won't need to change any code to add an additional field to your search.

It's also handy that these full text search engine's usually have some type of paging system built in.

Answer 11

I wish I could say OODB, but no OODBs have really made a break in the market.

Answer 12

A temporal database would be nice. No more having to do time joins by hand anymore. Automatic auditing too. Sigh.

For progress prediction/reporting.... wouldn't that be nice.

I have an open bug to implement something like an accurate progress bar myself. I'm going to record command object execution times. We'll have a reference system's command execution times recorded as well, so users can see if there "seems to be a problem" or not.

All I have to do is work a a neat way to parametize the query to predict execution tiem for the GUI. Maybe a polynominal. In a perfect world it could be self-fitting to the actual data.

(I already do something like this to predict print progress rates.)

Answer 13

I prefer Oracle and consider MySQL to be a toy database, although MySQL has got (few) useful features that don't exist in Oracle.

My favourite feature is PL/SQL, a very powerful programming language that can be used for Stored Procedures, triggers and "Packages", units of procedures and other stuff inside of the DB. Meanwhile I tend to writing code that does mass data processing mostly inside of the database without costly interaction between Db and app.

I don't believe that we will see a switch to object-oriented databases soon because relational databases do quite well what's sufficient for 90% of all applications and can be made working for the rest.

By the way, Oracle supports flat files, like requested by Shog9 here. You can mount them as "external tables". I've used that for mass data import yet, works quite nicely.

Answer 14

What the database of the future is depends a lot on your need. I think we will see a broader landscape with more options than using the classical relational databases. New technology like data stream management systems for contigous real time queries, xml databases for storing document-centric documents, column stores, data warehouses etc.

But the relational model is far from dead yet. It would be best if you could give more details about your frustrations so we could find a future for your problem, not just the whole broad scope of databases.

Answer 15

The future is not with SQL, the lowest common denominator with a crippled functional language. Okay, at least it is somewhat standardized.

Try object prevalence (cl-prevalence, Hibernate) or on-disk object databases (e.g. Elephant).

Answer 16

I'll divide the answer in two:

From a conceptual level (relational model)

First-order predicate calculus, out of which came relational algebra is something that will not go away. Conceptually such algebra has no problem in formalizing any and all 'object models', hierarchical and network data structures.
The room for improvement here still exists and it goes towards higher order logic and type theories.

This has nothing to do (and never will have) with flat files, xml, SQL, objects, ORM or any of the implementation details. And that fact is very important (SQL is based on relational, but SQL is not equal to relational; even if SQL fails the concepts of relational algebra will stand).

From a point of usage (SQL database management systems)

SQL database management system have many, many areas where improvement could be achieved.

Initial selling point of database systems - letting database decide how to store and retrieve data is still a good thing, however it has (on average) failed to recognize how wide usage scenarios have become (sole reason for MySQL, which at the right time recognized that there is a whole industry that does not care, that much, about data integrity and consistency; also historically SQL databases were really mostly data capture systems and people failed to realize how important mostly-read system have become).
Indexing, or more broadly data storage and retrieval, should be more flexible then it currently is, allowing to optimize any of the following: read speed, write speed, consistency, durability (in memory transactions), atomicity, integrity, with very fine granularity and detail (especially under read and write scenarios).

Object-relational integration could also have a lot of improvements - even though the discrepancy is not so huge; database entities are, in essence, objects, so it mostly comes down to tools. (most ORM griefs are there because lot of people are not sure if their object class is a row or a table)

Temporal database management systems are definitively much closer to how we humans store data that we process; and in this respect it would be good to achieve higher level of mental ergonomy on every level of data design and management. Usability of any system that support full history and naturally works with time concepts and time related operations grows considerably.

This temporal aspect applies not only to data, but to metadata, including integrity rules.

Also, the separation of data and metadata is a bit overemphasized - or to put it in another words - designing database systems (and maintaining them) is manipulation of metadata, and for this purpose it would be beneficial if this data was managed using the same concepts as the regular data (this, in my mind, points towards higher-order logic).

For this to be possible database management system need much stronger symbolic manipulation of expressions, which again points to concepts such as lambda calculus.

Unfortunately, SQL standard is not likely to drastically change, because of data and systems inertia (costs), until adequate savings could be proven or at least argued.

Answer 17

I think the NoSQL databases show that the future of databases will be more heterogenous than the present. Relational databases will not go away in either the short perspective nor the long. After all, if your data is naturally relational (i.e. tabular), like economic data, the relational model is quite convenient.

However, for some types of data: hierarchical, many-to-many relations etc. the relational model is less useful. Or, it is useful, but the database model becomes rather contorted. That's where key-value stores, document databases, graph databases come in.

Often, a combination can be the most efficient. Over-simplifying for a moment, you use a database in two ways:

• Storing data (and retrieving it by id/primary key)
• Searching data

A relational database can do both, but depending on the structure of the data might not do either very well. When that's the case, you might be better off with a document database for storage and an external search index like Lucene for searching.

Unfortunately I don't think semantic databases (aka triple stores and associative databases) will see widespread use. But, if I'm wrong here the better, as the associative data model is IMO the most intuitive and expressive.