Long running ruby process that uses ActiveRecord to store records in a database

Question

I'm trying to write an app using Ruby on Rails and I'm trying to achieve the following:

The app needs to receive UDP messages coming in on a specific port (possibly 1 or more per second) and store them in the database so that the rest of my Rails app can access it.

I was thinking of writing a separate daemon that would receive these messages and shell out to a ruby script on my rails app that will store the message in the database using the right model. The problem with this approach is that the ruby script will be run very often. It would be better performance-wise if I could just have a long-running ruby process that can constantly receive the UDP messages store them in the database.

Is this the right way to do it? Is there something in the Rails framework that can help with this?

Answer 1

You definitely don't want to load the Rails stack for each incoming request -- that would be way too slow; you'll want to use something lower-level to handle the incoming connections. You might look at the internals of Webrick to see a simple server daemon coded in ruby -- or, if you want something more performant, look at Mongrel or Thin.

In general, the whole Rails stack isn't gonna help you a lot here -- most of it is geared towards serving web apps, not saving stuff straight off the wire.

The part of Rails that will probably help you the most is ActiveRecord -- there's a pretty good chance you'll want to use that to store your Model data in the database. In fact, you should be able to include your actual Rails models and use them in your UDP-monitoring process. Check the ActiveRecord docs for examples in connecting to a database outside your Rails project.

Answer 2

I have an application that does something similar to this, i.e receiving lots of messages on a port and persisting them to the database. We addressed a number of issues when evolving the design of our database, including the fact that we must not lose messages even if the database was unavailable for some reason.

For performance reasons and to ensure we did not lose messages we went for a two stage process. We wrote a small handler that listened for messages and then persisted them to a message queue using Apache Active-MQ. We then used the ActiveMessaging plugin within a separate rails application to consume the messages from the queue and persist them to the database. This method makes it easy to scale the listeners and will result in a much higher messaging throughput.

If you were to go this route then you might want to look at the Fuse implementation of Active-MQ which is generally a few versions further on that the Apache version.