Reading a repeating streaming data source in C#

Question

Hello Everyone,

Let's say I have a url to a streaming data source (for example a stream of updated weather data) and I know this url works with GET (I verified that it does return steaming data). The stream is compressed with GZIP, each "message" begins with a 1 byte id, a 2 byte part containing the message length, then another 2 byte part with some proprietary info. After that comes the message itself, and then this format is repeated as long as the stream remains open.

I need to execute a block of code every time a complete message is received which will parse the raw bytes into .net types (I can figure out the parsing part I'm sure if I have an array of bytes to work with). I've tried endless ways I've found on the net for similar situations but for some reason cannot get this to work. If someone could also explain how to do the same process using POST instead of GET that would be appreciated as well. Thanks in advance everyone!

P.S. From my own attempts at this it appears that only async reading will work. Which is where I think I was falling down in my attempts.

Bob

Answer 1

What isn't working, exactly? Have you tried getting the response stream with GetResponseStream(), doing a Stream.Read() to a byte[] buffer, and then using BitConverter?

Answer 2

Something like:

public IEnumerator<Message> GetEnumerator()
{
    HttpWebRequest req = (HttpWebRequest) WebRequest.Create(uri);
    req.AutomaticDecompression = DecompressionMethods.GZip;
    Stream s = req.GetResponse().GetResponseStream();
    BinaryReader read = new BinaryReader(s);
    while(true)
    {
      byte id = read.ReadByte();
      short len = (short)((read.ReadByte() << 8) | read.ReadByte());
      short proprietary = (short)((read.ReadByte() << 8) | read.ReadByte());
      byte[] msgBytes = read.ReadBytes(len);
      yield return new Message(msgBytes);
    }
}

Answer 3

I found some better search terms and found this question where the answer contained the information I was missing. I was working with IAsyncResult and the request state object incorrectly.

http://stackoverflow.com/questions/295557/c-downloading-a-url-with-timeout