Dealing with high number of real-time calls to partner API in Rails

Question

My Rails app needs to make a call to a partner's RESTful API for every request that it receives on a particular controller's action. I need to pass in request-specific parameters (e.g. IP, user-agent, etc.) to the partner API, and return the response I get to the user. Since the call to the partner API is very user-specific, I can't cache the response I get back from the partner API (e.g. in memcached). My requirement is to respond to the user in 1500ms or less.

My Rails app is hitting a wall in terms of requests / second because each mongrel gets blocked until the partner API returns a response. In a recent performance test, I saw a server eke out barely 5 requests/second (running 5 mongrels).

I have a few ideas on what to do:

1) Make the partner API respond faster. Realistically, I can only pull this lever so much - even with really hard work from the partner, average response time is 200ms.

2) Limit the amount of time I wait for the partner API to return. I tried using Rails::Timeout, but it only allows waiting periods in multiples of a second. Is there a way to make the timeout happen in milliseconds instead?

3) Is there a way to queue the calls to the partner API, have them execute asynchronously? In Java, I would have used threads for the same task.

Thanks for your help!

Answer 1

We have used delayed_job and it is working well. If you don't have to display the returned value from the partner apis, delayed_job seems to be a good solution. You can use the send_later feature of delayed_job to make any method call into a delayed_job. You can configure the number of workers in delayed_job as well and figure out at what point you get the right responsiveness.

Bharat

Answer 2

You can tell delayed_job to poll the database every second, so the job is performed quickly. Then use a column in your database to poll for the result.

Example:

1. Add the job to delayed job
2. Redirect the user to the result page
3. Display a progress indicator of some sort (circling arrow maybe)
4. Poll that database column/row/whatever for a result every second through an action

That would give the user feedback, and allow them to wait without thinking it's timing out.