views:

8

answers:

0

I have a continous stream setup and I tried just reading the data into a buffer but since the stream does not allow seeking I cannot determine the size of the stream at a time. I want to capture the xml by grabbing the opening element and continue reading till I have the closing element. I have tried using xml textreader but I get errors about the format of the xml with missing closing stuff etc so I then tried loading it as an xmldocument and then using the getelements by tag name approach but this did not work either...I get an unclosed literal string error.

So if just write the data which I capture by restricting buffer size and write to a file I can see the data there but it is just not in a manner to parse.

   If responseStream.CanRead Then
                mstring = New StringBuilder()

                Do

                    ''Dim xtr As XmlTextReader = New XmlTextReader(responseStream)
                    'Dim ntype As XmlNodeType = xtr.NodeType
                    '                        While xtr.Read()
                    'responseData = xtr.ReadInnerXml()
                    'BlogDiscover.outfile.WriteLine(responseData)
                    'BlogDiscover.outfile.Flush()
                    '                        End While


                    numbytesread = responseStream.Read(bufferread, 0, BUFFER_SIZE)
                    mstring.AppendFormat("{0}", Encoding.UTF8.GetString(bufferread, 0, numbytesread))
                    Dim xdoc As XmlDocument = New XmlDocument
                    xdoc.LoadXml(mstring.ToString)
                    Dim entry As XmlNodeList = xdoc.GetElementsByTagName("entry")
                    For i As Integer = 0 To entry.Count - 1
                        ' this is just a test - need call new thread here to do parsing
                        BlogDiscover.outfile.WriteLine(entry.Item(i).ToString)
                        BlogDiscover.outfile.Flush()
                    Next
                    'BlogDiscover.outfile.WriteLine(mstring.ToString)
                Loop While numbytesread > 0
            Else
                logger.constructLog("Response Stream was empty for some reason", True, True)
                ' reconnect 
                WebsRequest("GET", rurl, page)
            End If