Property-values database

Question

I have a number of objects, each one have an arbitrary number of shared, and distinct property-value pairs (more specifically: files, and their related properties -such as width, and height values for images, album/artist/length for music files, etc). I'd like to be able to search for objects having specific property/values (such as: by album), group by property, etc.

What kind of database would you suggest for this scenario? Due to modularity (ability to add more properties on-the-fly), as well as the fact of common properties are <20% of all properties, the standard SQL with normalized tables wouldn't really cut it. I have already tried to approach the problem using a "skinny data model"; however I have faced with serious scalability issues.

Are there any specialized databases tuned for this scenario (BSD-licensed solutions preferred)? Or any alternative way to tweak standard RDBMs for this?

Answer 1

Searching for an object having some properties makes me think about a RDF datastore. Have a look a a RDF API (see JENA , sesame, virtuoso ).

Or BerkeleyDB ?

Answer 2

Have a look at the databases offered by various Cloud services:

• Google AppEngine Datastore (based on Bigtable)
• Amazon SimpleDB
• Microsoft SDS
• Apache CouchDB

If Cloud databases aren't an option, BerkelyDB could be a good choice.

Answer 3

What you're talking about is called EAV model or triple store. Later can be queried with SPARQL

Answer 4

Pierre is right; a triplestore is what you want, and RDF is the standard for that. SPARQL is the standard language for querying it (a lot like SQL for RDBMS's).