Real time embeddable http server library required

Question

Having looked at several available http server libraries I have not yet found what I am looking for and am sure I can't be the first to have this set of requirements.

I need a library which presents an API which is 'pipelined'. Pipelining is used to describe an HTTP feature where multiple HTTP requests can be sent across a TCP link at a time without waiting for a response. I want a similar feature on the library API where my application can receive all of those request without having to send a response (I will respond but want the ability to process multiple requests at a time to reduce the impact of internal latency).

So the web server library will need to support the following flow

1) HTTP Client transmits http request 1

2) HTTP Client transmits http request 2 ...

3) Web Server Library receives request 1 and passes it to My Web Server App

4) My Web Server App receives request 1 and dispatches it to My System

5) Web Server receives request 2 and passes it to My Web Server App

6) My Web Server App receives request 2 and dispatches it to My System

7) My Web Server App receives response to request 1 from My System and passes it to Web Server

8) Web Server transmits HTTP response 1 to HTTP Client

9) My Web Server App receives response to request 2 from My System and passes it to Web Server

10) Web Server transmits HTTP response 2 to HTTP Client

Hopefully this illustrates my requirement. There are two key points to recognise. Responses to the Web Server Library are asynchronous and there may be several HTTP requests passed to My Web Server App with responses outstanding.

Additional requirements are

1. Embeddable into an existing 'C' application
2. Small footprint; I don't need all the functionality available in Apache etc.
3. Efficient; will need to support thousands of requests a second
4. Allows asynchronous responses to requests; their is a small latency to responses and given the required request throughput a synchronous architecture is not going to work for me.
5. Support persistent TCP connections
6. Support use with Server-Push Comet connections
7. Open Source / GPL
8. support for HTTPS
9. Portable across linux, windows; preferably more.

I will be very grateful for any recommendation

Best Regards

Answer 1

You could try libmicrohttp.

Answer 2

What you want is something that supports HTTP pipelining. You should make yourself familiar with that page if you are not already.

Yes, go for libmicrohttp. It has support for SSL etc and work in both Unix and Windows.

However, Christopher is right on the spot in his comment. If you have a startup time for each response, you are not going to gain much by pipelining. However, if you only have a significant response time to the first request, you may win something.

On the other hand, if each response has a startup time, you may gain a lot by not using pipelining, but create a new request for each object. Then each request can have its own thread, sucking up the startup costs in parallel. All responses will then be sent "at once" in the optimum case. libmicrohttp supports this mode of operation in its MHD_USE_THREAD_PER_CONNECTION thread model.

Answer 3

Howard,

Have you taken a look at lighthttpd? It meets all of your requirements except it isn't explicitly an embedded webserver. But it is open source and compiling it in to your application shouldn't be too hard. You can then write a custom plugin to handle your requests.

Answer 4

Can't believe no one has mentioned nginx. I've read large portions of the source-code and it is extremely modular. You could probably get the parts you need working pretty quickly.

Answer 5

uIP or lwip could work for you. I personally use uIP. It's good for a small number of clients and concurrent connections (or as you call it, "pipelining"). However, it's not as scalable or as fast at serving up content as lwip from what I've read. I went with simplicity and small size of uIP instead of the power of lwip, as my app usually only has 1 user.

I've found uIP pretty limited as the concurrent connections increase. However, I'm sure that's a limitation of my MAC receive buffers available and not uIP itself. I think lwip uses significantly more memory in some way to get around this. I just don't have enough ethernet RAM to support a ton of request packets coming in. That said, I can do background ajax polling with about a 15ms latency on a 56mhz processor.

http://www.sics.se/~adam/software.html

I've actually modified uIP in several ways. Adding a DHCP server and supporting multipart POST for file uploads are the big things.) Let me know if you have any questions.