Say I have a web application. I have a front end web server for serving static assets and 3 http-speaking application servers. I need to process some background jobs. The natural thing to do is to start up a separate process in the background to the process the job. However, I'm wondering if that's necessary - and why this is the normal approach as opposed to running the job in the application server, and invoking the job by sending a HTTP request, say with curl.
Let's say I have 200MB of memory to play with. The application servers use 50MB each. The background job uses 50MB. The two possible configurations are:
- With background process: 3 app servers, 1 background job = 200MB
- Without background process: 4 app servers = 200MB
The advantage to the latter is that when I have no need to process any jobs, I am able to use my extra application server instance to serve requests.
Given the assumption that my application servers and my background jobs have the same memory footprint (the 50MB each), what are the disadvantages to running the jobs in the application servers, and not having a background process?