Streaming an undetermined number of objects through WCF

views:

answers:

+7 Q:

Streaming an undetermined number of objects through WCF

I have a WCF service that needs to read large (10 to 20 million) numbers of objects from the database.

What I'd like to do, is have the client open up a stream and have the server push data from the database as it's reading.

So the client could just sit in a loop deserializing messages until it gets the EOF message from the server, in the style of the Twitter Streaming API, but with a finite set. The issue I'm having, is how to return the stream and then keep writing to it. Is this possible with WCF?

+2 A:

How about instead of setting up a streaming/blocking service you use something like WS Dual Http. With it you have an async bidirectional callback channel that allows you to request/reply information back and forth between the server and the client. Issues you may see if you want to stream the entire set is a normal fashion is that some of the resources may block other requests (or timeout) while other users try to access the service.

Matthew Whited 2010-08-29 12:41:31

WS Dual Http does not allow streaming at all. But generally you are right. Probably the only way to simulate such communication in WCF is to open bidirectional streamed Net.TCP connection (not sure if this combination is possible). Instead of returning single stream use client callback to send set of data.

Ladislav Mrnka 2010-08-29 12:43:16

I didn't say it did allow streaming. I am trying to say it's probably a better option than streaming. You can do streaming in WCF but it will block I/O for other incomming requests. It's not a good choice when it comes to scaling. Also, most built in serializers will want the entire message downloaded before they started to deserialize the content.

Matthew Whited 2010-08-29 13:21:21

Would I be better off just using an HttpHandler and writing to the stream that way? I need to go over HTTP as this is an external web service. Took a look at your solution Matt and it looks nice, but I had forgot to mention that we might have people using this from outside WCF, so I'm guessing I'd have to go the HttpHandler route? I'm not necessarily tied to WCF, just wondering if it was possible, so if there are other better options, I'm certainly open to them. Thanks for taking the time to answer my question btw.

Jason Ruckman 2010-08-29 13:39:09

The problem is that WCF does not provide response stream to operation. The stream returned from operation is just a "content" of some message. I tryed to trick WCF with some threading scenarios where I returned MemoryStream and tryed to fill it from other thread but as expected it didn't work.

Mentioned HttpHandler is the only way to go.

Ladislav Mrnka 2010-08-30 22:07:54

ansaurus

tags:

views:

answers:

Streaming an undetermined number of objects through WCF

related questions