OutOfMemoryException loading xml document winmo 6.1

Question

I am using c# on a windows mobile 6.1 device. compact framework 3.5. I am getting a OutofMemoryException when loading in a large string of XML. The handset has limited memory, but should be more than enough to handle the size of the xml string. The string of xml contains the base64 contents of a 2 MB file. The code will work when the xml string contains files of up to 1.8 MB.
I am completely puzzled as to what to do. Not sure how to change any memory settings.
I have included a condensed copy of the code below. Any help is appreciated.

 Stream newStream = myRequest.GetRequestStream();
                // Send the data.
                newStream.Write(data, 0, data.Length);
                //close the write stream 
                newStream.Close();
                // Get the response. 
                HttpWebResponse response = (HttpWebResponse)myRequest.GetResponse();
                // Get the stream containing content returned by the server.
                Stream dataStream = response.GetResponseStream();
                //Process the return
                //Set the buffer
                byte[] server_response_buffer = new byte[8192];
                int response_count = -1;

                string tempString = null;

                StringBuilder response_sb = new StringBuilder();
                //Loop through the stream until it is all written
                do
                {
                    // Read content into a buffer
                    response_count = dataStream.Read(server_response_buffer, 0, server_response_buffer.Length);
                    // Translate from bytes to ASCII text
                    tempString = Encoding.ASCII.GetString(server_response_buffer, 0, response_count);
                    // Write content to a file from buffer
                    response_sb.Append(tempString);
                }
                while (response_count != 0);
                responseFromServer = response_sb.ToString();
                // Cleanup the streams and the response.
                dataStream.Close();
                response.Close();
            }
            catch {
                MessageBox.Show("There was an error with the communication.");
                comm_error = true;
            }
              if(comm_error == false){
            //Load the xml file into an XML object

                  XmlDocument xdoc = new XmlDocument();
                  xdoc.LoadXml(responseFromServer);
              }

The error occurs on the xdoc.LoadXML line. I have tried writing the stream to a file and then loading the file directly into the xmldocument but it was no better. Completely stumped at this point.

Answer 1

Have you tried loading from a stream instead of from a string(this is different from writing to a stream, because in your example you are still trying to load it all at once into memory with the XmlDocument)?

There are other .NET components for XML files that work with the XML as a stream instead of loading it all at once. The problem is that .LoadXML probably tries to process the entire document at once, loading it in memory. Not only that but you've already loaded it into a string, so it exists in two different forms in memory, further increasing the chance that you do not have enough free contiguous memory available.

What you want is some way to read it piece meal into a stream through an XmlReader so that you can begin reading the XML document piece wise without loading the entire thing into memory. Of course there are limitations to this approach because an XmlReader is forward only and readonly, and whether it will work depends on what you are wanting to do with the XML once it is loaded.

Answer 2

I would recommend that you use the XmlTextReader class instead of the XmlDocument class. I am not sure what your requirements are for reading of the xml, but XmlDocument is very memory intensive as it creates numerous objects and attempts to load the entire xml string. The XmlTextReader class on the other hand simply scans through the xml as you read from it.

Assuming you have the string, this means you would do something like the following

        String xml = "<someXml />";
        using(StringReader textReader = new StringReader(xml)) {
            using(XmlTextReader xmlReader = new XmlTextReader(textReader)) {
                xmlReader.MoveToContent();
                xmlReader.Read();
                // the reader is now pointed at the first element in the document...
            }
        }

Answer 3

I'm unsure why you are reading the xml in the way you are, but it could be very memory inefficient. If the garbage collector hasn't kicked in you could have 3+ copies of the document in memory: in the string builder, in the string and in the XmlDocument.

Much better to do something like:

XmlDocument xDoc = new XmlDocument();
Stream dataStream;
try {
  dataStream = response.GetResponseStream();
  xDoc.Load(dataStream);
} catch {
 MessageBox.Show("There was an error with the communication.");
} finally {
  if(dataStream != null)
    dataStream.Dispose();
}