views:

65

answers:

2

We're trying to add some kind of persistence in our app. The app generates about 250 entries per second. Each of these entries belong to one of 2M files. For each file, we want to keep the last 10 entries, so we can look them up later.

The way our client application works :

  • it gets a stream of all the data
  • it fetches the right file (GET)
  • it adds the new content
  • it saves the file back (PUT)

We're looking for an efficient way to store this data that can scale horizontally as the amount of data we're getting is doubling every few weeks.

We initially looked at S3. It works fine, but becomes very expensive very fast (>$1000 monthly just in PUT operations!)

We then gave a shot at Riak. But it seems we can't get more than 60 write/sec on each node, which is very very slow.

Any other solution out there?

A: 

What about Hadoop's HDFS spread over Amazon EC2 instances? I know each instance has a good amount of storage space, and you don't have to pay for put/get, only the inbound transfer.

webdestroya
We're not using Amazon... so yes, we would have to pay for the bandwidth to EC2. But maybe that's an option. Have you already setup an HDFS cluster?
Julien Genestoux
Yes, I have. I setup a cluster to run on about 50 of the Small EC2 machines (which is about 7.5TB of storage)
webdestroya
At this rate S3 is a bargain :) 50 small EC2 is already 3500 monthly without even considering bandwidth! :(
Julien Genestoux
Well, you could also run Hadoop over your own machines. I think any system you choose is going to be somewhat costly (either you have to by/rent your own servers, or you pay for bandwidth)
webdestroya
A: 

There are lots of knobs you can turn in Riak - ask the mailing list if you haven't already and we'll figure out a sane configuration for you. 60 writes/sec is not within the norm.

See: http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

We asked... no answer yet : http://lists.basho.com/pipermail/riak-users_lists.basho.com/2010-May/001170.html Just the fact that nobody even said there that 60 writes/sec was not the norm is a very bad news.
Julien Genestoux
Sorry about that Julien, I didn't connect the dots. Please email me and we can set up a phone call to address your problem: sean AT basho.com