Building a high performance and automatically backupped queue

Question

Please give me some hints on my issue.

I'm building a queue data structure that:

1. has a backup on hard disk at realtime
2. and can restore the backup
3. Can respond to massive enqueue/dequeue request

Thank you!

Answer 1

Is this an exercise your doing. If not, you should probably look at some of the production message queueing technologies (e.g. MSMQ for Windows) which supports persisting the queues on the disk and not just storing them in memory.

In terms of your requirements

1. has a backup on hard disk at realtime

Yes MSMQs can do that.

2. and can restore the backup

And that.

3. Can respond to massive enqueue/dequeue request

And this...

Answer 2

If you can avoid it, don't roll your own. For Java try ActiveMQ.

Answer 3

You're probably looking at something more complex than just a simple library.

Since this is an exercise (probably wanting you to think about the underlying data structures), you might start the simple way by queueing to a mySQL database. Your performance will suck compared to dedicated queueing software, but you can at least get the rest of the infrastructure working.

After that, you'll probably be looking at some form of custom file format and a multi-threaded server on top of it. You might be able to pull it off using something like BDB or SQLite for the I/O layer, thus saving you the trouble of writing the actual disk routines.