Suggest Comet server

Question

I am looking for open-source linux-based Comet server. Currently, looking at Hookbox - but as it's in Python, I am concerned about performance. Does Python instead of C++ have big impact on performance here?

Developer estimated it's performance to about 100 of requests per second when having 100 channels... Not really fast :-|

Could you suggest the most actively supported/used comet server which is binary(=faster), easy to integrate and (kinda) lighweight? Also, Java/.NET is no-go here.

Our goal is some 10k+ connections per server, about 2000-5000 messages per second across 10-20k channels.

Answer 1

I have had tremendous results in terms of performance with http://www.ape-project.org/

Answer 2

Have a look at the jave based cometd server that used to be bundled with Jetty web server.

Now available at commetd.org

No idea how well it performs but it certainly functions!

Answer 3

Have you looked into Node.js? It is excellent in terms of performance, and might even make your code easier to implement (if you're writing new code and don't already have a bunch of code written).

If you're looking for a server for existing code that isn't server-side Javascript, just ignore this answer :)

Here is a success story on a company using Node.js for their Comet needs.

Here is a presentation on why Node.js is awesome, and how you would implement Comet with Node.js.

Answer 4

What about Go?

Here's a possible - if slightly "alternative" approach - as

• you seem to be very focused on performance,
• you want something Linux based,
• and don't want a JVM language but probably something more bare-bone,
• and I assume it's for a personal project or something where you are free to pick technologies.

Have a look at Google's Go language, and in particular their webserver example, which presents pretty good performance benchmarks.

They also have a web-application development code-lab for training.

Some interesting discussions on web-frameworks for Go and Comet-support. You'll also find another http-server implementation.

Like I said, maybe not exactly a ready-made solution, but I thought you might be interested in exploring that.

---

Really, no Python?

If you were to reconsider Python, you could have a look at Tornado.

---

Really, no (J)VM-Languages?

If you were to reconsider your position on VM languages, I'd also recommend you have a look at Scala and the Lift web framework, which has great support for comet with a very expressive syntax and good performance (that's used by Foursquare, Twitter, Novell Pulse, so you can bet on its performance)

---

Fine, let's be conventional...

Last but certainly not least, have a look at LightHttpd, which is developed in C and should satisfy all your needs. Look at the LightHttpd page on Wikipedia for a brief.

(but really, have a look at the other ones, especially at Go and Scala/Lift, even if only for educational purposes)

---

If you need more...

Actually, I just noticed someone seems to have already done the legwork. So if you want more details or to do some more research, have a look at the Comet Servers for Single Dealer Platforms article and Comet Daily's Comet Maturity Guide.

---

EDIT: I just realized I din't answer the first part of your question. yes, using Python might have an overhead over a C/C++ implementation, especially if you don't use pre-compiled Python files. And considering what you want (in terms of numbers of connections per seconds and concurrent connections), I think you more importantly need something that can scale than something that would necessarily be "just" fast in terms of execution speed.

The Comet for highly-scalable applications presentation may also interest you.

Answer 5

Does Python instead of C++ have big impact on performance here?

The answer is it depends on the application and the developer's skills. For a number crunching program, the language choice is more important. However you are developing a networking program which is IO bound.

The question you should be asking is, how good is Hookbox at scaling?

The answer is that it is pretty good because it is based on the eventlet asynchronous IO library. Your developer is grossly underestimating it's capabilities.

Further, all of the alternatives mentioned by Haylem are also worth considering. If all you are doing is building a massive chat room or game, you will probably get pretty far with a Python-based solution. I suspect you will encounter a rash of other scalability issues long before Python performance becomes an issue.