Which programming language should I choose for my high performance webservice?

Question

I am in the process of building a Web Service API for my application. Also, I am planning to expose the Service via both REST and SOAP.

I'm interested in getting some feedback from the community as to which programming language I should choose to implement the service? (I know C#, Java and Ruby - RoR well enough to create the service).

The service is mainly an HTTP POST service. It will need to handle around 2000 concurrent connections as well as be able to handle around 10,000 HTTP POST's per second. (for SOAP we will have a submit method for the clients to call).

The service does not return any response to the client for the POST requests.

Any ideas on which programming language/architecture which should be used?

Answer 1

10,000 requests per second is 25 billion hits per month. That means one of two things:

1. Your application is more popular than MySpace; or
2. You are trying to use this to communicate between two very chatty components that you control, and it's a poor horrible design choice.

The switching hardware alone to distribute that much load across a farm of web front-ends would cost many thousands of dollars.

Start by writing a web service that can handle 50 requests per second (language choice is not hugely relevant). If your application is so busy that you cross that threshold regularly, you can afford to hire someone to work on the scaling problem full-time, and not have to ask for help on a free Q&A site.

Answer 2

You can really use any language through CGI (Common Gateway Interface), so it comes down to performance. Among the languages you list, i expect C# to be fastest. A good comparison for speed among languages is The Language Shootout

If you really need performance you might want to look in the direction of a more performance oriented language like C or D to handle the requests.

It all depends on what kind of computation each request have to perform really.

Answer 3

At 10,000 posts a second, the language is the least of your worries. A much bigger issue would be the design of your server farm and network. I assume you don't plan on running this on a single box?

Answer 4

Update: Its meant to be a fire and forget web service. I guess i will send back a simple HTTP 200/OK response

No this is not intended to run on a single box. Its intended to run on a few boxes (say 3-4).

When the requests are received they are pushed to queue's on other machines, then they are taken and put in a HBase/Voldemort Store.

As i said, its meant to be a "fire and forget" web service

Answer 5

Highly scalable applications, reliable, distributed, and using multicore/multiprocessor systems? Here I immediately think of Erlang/OTP together with Yaws as the web application server. Yaws runs extreme stable and fast under extreme high load. And Erlang/OTP as the platform is designed for concurrency and distribution, together with some mechanisms helping to develope stable software. The costs: concurrency-orientation with a functional programming language is no OOP with Java or C#, the syntax seems weird (but is very straight and powerful once you've adopted it), and the number of third-party libraries is not as huge as for the mainstream languages. But it's worth it.

Hope this helps

mue

Answer 6

At that rate, and since you're breaking HTTP anyway (no response) you might as well develop your own server, or modify an open source server.

Write it all in C or C++ and you'll be blazing about as fast as possible.

Scalability is affected by more than language choice though.

Answer 7

You need a C llike language and to avoid writing a complete server I would suggest CGI (which is what php and the like all run through anyway) Windows servers offer ISAPI plug ins, but these run in the context of the server so memory leaks and GPFs will take down the server. Add to that the inconvenience of stopping/starting the server each time you change somthing, CGI/FastCGI looks better.

Answer 8

I could see getting a billion posts per month out of a single machine. I have a web service written in c# that's currently handling about 3.5 Million posts per day. The web server is running along at 3% CPU utilization. Which means I could push it at least 20 times as hard...

Assuming each of your machines had 4 Xeon Six cores, 32GB of RAM, a fast disk array, and a highly optimized database for writes you could do it. Although, the cost of each server is probably in the $35K to $40K range.

Regardless, your bottleneck would not be with C# or Java. It would be with the database server depending on how large it grows. In my case, it's about 300GB with 10GB being deleted and 10GB being added per day.

Answer 9

Based on my previous experience I can give you the following advice.

1. Pick the language you (and possibly other team members) like most. I would prefer higher level languages because hardware is fast and cheap, but programmers are slow and expensive.
2. Design your services to be absolutely state-free (no sessions!). This makes it easy to add new hardware, as the different instances of your service need not know of each other.
3. Handle your processing asynchronously, as you fortunately need not give the client any response (other than OK). If you do it synchronously your process will block and your request-rate will drop. A good read is this Wikipedia article, and especially (the classic!) The C10K problem.
4. Put the service on many machines. (depending on the speed of your services)
5. Put your database server(s) on other machines than the web-services. Use fast disks!
6. Handle the load by balancing it with something like:
   • Linux Virtual Server, the most performant solution, because it runs in the Kernel. Scales like mad. I used it 2003 with ~500req/sec on a P3/1GHz with 0.1% CPU load. Can be paired to achieve HA. Should handle the 10000req/sec quite nicely on a single machine. Do this after trying something simpler. This can be quite challenging.
   • Pound, very easy to set up but high overhead. Good starting point. Can do SSL.
   • Nginx, easy configuration, very performant. Can do SSL. Can also act as HTTP-Server and may be a performant hosting solution for your services.
   • Perlbal, haven't used it but heard good things.
   • or other reverse proxys.

Answer 10

Lets look at the issues:

IO: this will easily be the greatest bottleneck in your system. Pick a language that provides the best integration with host os, and, provides advanced semantics for non blocking and optionally support for concurrency.

Data: SOAP? XML? You will want to minimize any un-necessary cpu cycles. What's wrong with simply using JSon? (And there is no divine dictate that says a REST architecture based server can't use binary data in the protocol ...)

Content: If any transformation of data (from to text to number, for example) is involved, you will also need to consider which language provides the most efficient mechanisms. As an example, in Java (which is a very strong candidate for you, btw) the String class is a serious CPU hog.

Java and Erlang are very good candidates. C is always an option but concurrent programming is much more difficult.