Are you using event streaming products?

Question

Maybe you're familiar with the concept of event streaming processing (ESP) ... if you are, I'd love to hear what you're using and what platforms you're using them on. I am an active contributor to the Esper project (http://esper.codehaus.org/) but I'd be interested in hearing what others are using? Anyone using Coral8, Aleri or Streambase? What platforms are you using them on?

Answer 1

at present i am not using any event stream processing (ESP) / complex event processing (CEP) products, but am familiar with the concepts - we evaluated Streambase for a prior (military) project and I still get emails from them every now and then ;-)

I looked into using esper for another real-time system (not a financial trading system, which seems to be the primary application for CEP these days), but thought it was - no offense - overkill. We wanted something lightweight to control an event-driven process in such a way that it could be altered without redeploying the code base, but frankly ran out of time so the evaluation did not get to go 'deep' enough. Will probably revisit this in future.

ESP/CEP is underrated, in my opinion, as a solution platform. It can be difficult to comprehend if you're used to purely transactional thinking.

Answer 2

At a previous job, we looked at Cayuga, Esper, StreamBase, and Coral8. I'm actually interested in seeing if I can make a souped-up RSS reader using one of these engines and a text-processing engine. I'm not sure how it would all fit together, as I haven't had the time to think about it, but I think an RSS item can be consumed as an "event", processed, and then things could happen if similar "events" (similar messages or postings) are received in a time span. Perhaps the identification of hot topics and such.

Answer 3

I work for Aleri (opinions here are my own, though), so my view is a little skewed. Having said that...

We're seeing a number of great use cases, ranging hugely in complexity and data volume. A few examples:

• Market data cleansing. We take stock quotes and "sanitise" them, flagging quotes that are out of expected market bounds
• Algo trading is the canonical CEP example application, and that's kind of the sanitising application above with an extra piece at the end that places an order if a certain pattern of events occurs
• Aggregating order books across trading venues, i.e., feeding in market prices across exchanges and outputting a consolidated order book
• Liquidity and risk management

Some people use it for the low-latency aspect, but many use it because it allows rapid development of many event-driven applications.

I'm seeing a lot of installations on Solaris/x86-64 machines, though we also have SPARC and Linux in production too.

By the way, most vendors use the term CEP, Complex Event Processing, to describe their products. I don't think that was the best TLA to use, but it looks as though we're stuck with it now.

Feel free to ask any further questions. Obviously there are certain things I can't talk about but I'll do what I can.

Answer 4

I was on a team that did a proof-of concept with rulecore. We used it to run a couple of hundreds of rules which were automatically generated from a database. We used a single server with 16 gigs of ram and 4 cores.