We have a server-side component that is currently written in PHP on Apache2 that is essentially a cross domain proxy that uses cURL to make external HTTP requests, does header processing and performs some regular expression replacement on the payloads. It is going to be the most heavily used piece of server-side code for our site.
PHP is fine for our development needs now, but we would like to move to something more efficient. We plan to run on EC2. Considering that we estimate that this proxy will makeup at least 80% of our load, the more efficient we cane make it from a load and network perspective, the lower our costs will be.
We realize that performance testing of our specific use case is going to be the only for sure way to determine the optimal solution, but what should we consider? Right now we are thinking about actually going with a compiled approach with C and FastCGI since cURL is a C library to begin with. However, from reading about it, it seems that Apache communicates with the FastCGI processes through sockets? That wouldn't count as EC2 network traffic would it?
It seems like FastCGI is an excellent solution when connections to resources like databases can be left open; however, in our case, we make an external HTTP request with cURL to a different domain on each request from the browser, so there is no potential to pool or keep anything open.
It seems what would be ideal would be a method to spawn a thread under apache's process that executes the C program. Is writing a custom apache module within reason and the only way to accomplish this? Our code is so brief that we would be willing to go through something to that extent.