I need a way to do key-value lookups across (potentially) hundreds of GB of data. Ideally something based on a distributed hashtable, that works nicely with Java. It should be fault-tolerant, and open source. The store should be persistent, but would ideally cache data in memory to speed things up. It should be able to support concur...
I am looking to write a Key/value store (probably in python) mostly just for experience, and because it's something I think that is a very useful product. I have a couple of questions. How, in general, are key/value pairs normally stored in memory and on disk? How would one go about loading the things stored on disk, back into memory? Do...
Given a search engine like Lucene and a set of XML documents which need to be fully preserved, what are the advantages and disadvantages of using the search engine as key value store for returning XML doucments given a unique primary key which each document contains? ...
http://www.infoq.com/presentations/newport-evolving-key-value-programming-model is a video about KV stores, and the whole premise is that redis promotes a column-based style for storing the attributes of an object under separate keys rather than serialising an object and storing it under a single key. (This question is not redis-specifi...
I'm reviewing my code and realize I spend a tremendous amount of time taking rows from a database, formatting as XML, AJAX GET to browser, and then converting back into a hashed javascript object as my local datastore. On updates, I have to reverse the process (except using POST instead of XML.) Having just started looking at R...
NoSQL refers to non-relational data stores that break with the history of relational databases and ACID guarantees. Popular open source NoSQL data stores include: Cassandra (tabular, written in Java, used by Cisco WebEx, Digg, Facebook, IBM, Mahalo, Rackspace, Reddit and Twitter) CouchDB (document, written in Erlang, used by BBC and En...
It has been recommended to me that I investigate Key/Value pair data systems to replace a relational database I have been using. What I am not quite understanding is how this improves efficiency of queries. From what I understand you are going to be throwing away a lot information that would help to make queries more efficient, by simpl...
There is a table: doc_id(integer)-value(integer) Approximate 100.000 doc_id and 27.000.000 rows. Majority query on this table - searching documents similar to current document: select 10 documents with maximum of (count common to current document value)/(count ov values in document). Nowadays we use PostgreSQL. Table weight ...
Berkeley DB would be the best choice probably but I can't use it due to licensing issues. Are there any alternatives? ...
What happens to a Redis data store if the data exceeds available ram? ...
This question asks for one in java, what's the best candidate in PHP? BTW,does the following establish: distributed key value store == distributed cache ? ...
Most of the time,we just get the result from database,and then save it in cache server,with an expiration time. When do we need to persistent that key/value pair,what's the significant benifit to do so? ...
I will build a system where I want to reduce single-point-of-failures, and I need a database. Is there any (free) relational database systems that can handle multi-master setups good (i.e where it is easy to add and remove nodes) or is it better to go with a NoSQL-database? As what I have understood, a key-value store will handle this b...
I have setup Riak on a Ubuntu machine, and it seam to work if I do riak ping. Now I would like to use the Riak Java client to store an object, but it doesn't work. I get com.basho.riak.client.response.RiakIORuntimeException when I try to store an object. What am I doing wrong? Is there a way to test if I can access riak from my java cli...
We want to persist some user settings int he GUI part of our code. I used to do Win32 programming exclusively and the typical way this was done was with registry settings. I assume that this should be done with configuration files, but was wondering if there was a library or cross platform wrapper that made key/value pair persistence v...
I have to store e-mail messages for use with our application. I have "metadata" for all messages inside a relational database, but I don't feel comfortable keeping message content (gigabytes and terabytes of email data) inside a database. I'm currently using IMAP as a storage, but I have my doubts if I choose correctly. First of all ther...
I am planning on using CouchDB on a project. But as the querying mechanism involves writing views (which are a lot like indexes on regular RDMBMS's) I was wondering, if the document database keeps getting updated a lot ( a write heavy database) would CouchDB perform well compared to a regular RDBMS? Or do we have to compact/re-index the ...
I am looking for a key/value type datastore with very specific requirements. Anyone know anything that will work? Needs to be a component of some sort. No additional installation needed. The datastore needs to be on the local hard drive. I am using VB.Net for a desktop app running Windows XP through 7 so it needs to callable by that en...
I am interested to know the performance impacts of using MySQL as a key-value database vs. say Redis/MongoDB/CouchDB. I have used both Redis and CouchDB in the past so I'm very familiar with their use cases, and know that it's better to store key/value pairs in say NoSQL vs. MySQL. But here's the situation: the bulk of our applicatio...
I need to write a C++ application that reads and writes large amounts of data (more than the available RAM) but always in a sequential way. In order to keep the data in a future proof and easy to document way I use Protocol Buffer. Protocol buffer however does not handle large amounts of data. My previous solution consisted on creating...