C# - How to read a continuous stream of XML over HTTP

Question

I am trying to figure out the best way to consume a continuous stream of XML data from a service that is sending the data as a "constant" feed over HTTP.

I was considering using HttpWebRequest/Response but I am not sure how that will behave if the data just continuously streams.

Any thoughts?

Answer 1

I have done this before, not with XML, but with data that needed to be parsed for state changes for an application. HttpWebResponse.GetResponseStream() method worked fine for this. Make sure to call Close() on this stream when you are done. I suggest a finally block.

HttpWebRequest req;

try
{
    req = (HttpWebRequest)WebRequest.Create("http://www.example.com");
    Stream stream = req.GetResponseStream();

    byte[] data = new byte[4096];
    int read;
    while ((read = data.Read(data, 0, data.Length)) > 0)
    {
         Process(data, read);
    }
}
finally
{
    if (req != null)
        req.Close();
}

Or, alternatively:

HttpWebRequest req;

try
{
    req = (HttpWebRequest)WebRequest.Create("http://www.example.com");
    Stream stream = req.GetResponseStream();

    XmlTextReader reader = new XmlTextReader(stream);

    while (reader.Read())
    {
       switch (reader.NodeType) 
        {
         case XmlNodeType.Element:
           Console.Write("<{0}>", reader.Name);
           break;
         case XmlNodeType.Text:
           Console.Write(reader.Value);
           break;
         case XmlNodeType.CDATA:
           Console.Write("<![CDATA[{0}]]>", reader.Value);
           break;
         case XmlNodeType.ProcessingInstruction:
           Console.Write("<?{0} {1}?>", reader.Name, reader.Value);
           break;
         case XmlNodeType.Comment:
           Console.Write("<!--{0}-->", reader.Value);
           break;
         case XmlNodeType.XmlDeclaration:
           Console.Write("<?xml version='1.0'?>");
           break;
         case XmlNodeType.Document:
           break;
         case XmlNodeType.DocumentType:
           Console.Write("<!DOCTYPE {0} [{1}]", reader.Name, reader.Value);
           break;
         case XmlNodeType.EntityReference:
           Console.Write(reader.Name);
           break;
         case XmlNodeType.EndElement:
           Console.Write("</{0}>", reader.Name);
           break;
       }       

    }       
}
finally
{
    if (req != null)
        req.Close();
}

Answer 2

Should be able to do it fairly easily. You'll need to get the response stream by calling Response.GetResponseStream() and then use the async ResponseStream.BeginRead() in a loop.

There is no Timeout setting on the Response but if you're consistently getting sent data then it should be fine.

Answer 3

XmlTextReader takes stream as a constructor and I believe its reading is lazy so that should suffice.

http://msdn.microsoft.com/en-us/library/system.xml.xmltextreader.xmltextreader.aspx