A web app that launches cpu intensive background jobs in the background whenever users call it.
example: youtube.com. people upload videos, it gets converted in background job.
how can you deal with many users simultaneously attempting to launch their own background jobs ? it must be queued correct?
additionally, if the background job fails, the user on web app should be notified. if success, notified.
is there a framework or library in ruby or php that can handle this ?
In Ruby you have several options when it comes to background processing:
If you want rails specific related things, you can watch episods 127, 128, 129 and 171 from RailsCasts.
However if you want to use more modern way(event-driven architecture using messaging) then I recommend ActiveMessaging.
The basics is to create a queue that you put the job in and then have a worker running as a cronjob or daemon. The worker takes jobs off the queue, processes it and puts the result back with a flag that says "done". The frontend will then periodically poll the queue to see if its job has finished. You can use a database for the queue - pick one that supports locking.