views:

42

answers:

2

I've been working on a rails app for a couple of days now that I need an underlying "middle layer" for that connects my rails application to the various services that make up the data.

The basic setup looks like this:

Frontend ("Rails app") -> user requests data to be aggregated -> info goes in database and a JSON request is sent to "middle layer" to retrieve the data from a source of other places, process it, then send it back to the frontend which streams it to the users browser via websockets.

Middle layer -> uses sockets to listen for the frontend making a request. Once request is made, aggregation begins.

Base layer -> load balancing among a scalable network design.

I don't think that this is as efficient as it could be though. I feel like I'll run into concurrency problems or it just being too slow for use. I need to be able to scale it up.

My main question though rests in the area of what language would be more efficient for this to run fast?

+1  A: 

It depends. What are your data sources? Are they on the same machine? Are they databases?

My gut tells me that the language you choose won't play much of a role in how well this kind of application performs, but it it hard to say without knowing the details.

C++ is probably a bad idea for a scalable web-app though. It is very likely that you will end up with something which is slower than what you would have written in Ruby, because you end up worrying about irrelevant details. With regards to concurrency concurrency problems, C++ is definitely not the easiest language in which to write concurrent code.

Without knowing more, I would recommend that you stick with Ruby or some other high-level language and profile to see where the bottlenecks are. If you find out that there is some tight loop which needs to run really fast you can write that part in C, but you probably won't need that.

Jørgen Fogh
Ahh, I forgot to mention that. I'm using third party data sources such as Twitter (parsing tweets for certain words or phrases), RSS feeds (high quantities that will reach several thousand or more when its finished), a web crawler that has yet to be implemented, and other things like that. That brings me to my next dilemma though if I use Ruby. Should I be spawning instances for each request if it needs to search that much info? Or using a few instances of one well managed process?
Anthony Crognale
@Anthony Crognale: You should probably be caching the data instead. The language will *definitely* not be a bottleneck in this case. And no, you should not spawn instances for each request, though you might want to use a thread pool.
Jørgen Fogh
A: 

Stick with Ruby unless you can prove an actual need for C++. Look into something like delayed_job to handle your background tasks.

jdl