views:

126

answers:

1

I'm using the System.Security.Cryptography library to encrypt and decrypt xml files. Recently though I've been getting OOM (Out of Memory) exceptions while trying to decrypt a 75MB file. Here's the code I'm using:

    using System.Security.Cryptography.Xml;
    ...
    public static XmlDocument DecryptIntoXmlDoc(string filename)
    {
        //Decrypt the XML
        XmlDocument xmldoc = new XmlDocument();
        EncryptedXml exml = new EncryptedXml(xmldoc);
        TripleDESCryptoServiceProvider ekey = new TripleDESCryptoServiceProvider();
        ASCIIEncoding encoding = new ASCIIEncoding();

        ekey.Key = encoding.GetBytes(GetMasterKey());
        exml.AddKeyNameMapping("ekey", ekey);
        xmldoc.Load(filename);

        // -- THROWS THE OOM ERROR --
        exml.DecryptDocument();

        //Clear exml
        exml = null;

        return xmldoc;
    }

As soon as .DecryptDocument() is called I get the following error:

 Exception of type 'System.OutOfMemoryException' was thrown.

I haven't been able to find anyone else having this problem, but I have read that if the xml tags aren't properly named/nested, the file can be very large when loaded into memory. Would renaming my XML tags to shorter names reduce the size? Is there a way to nest xml tags to reduce the file size?

Is there anything else I could do?

A: 
  • Run your program under the debugger
  • When the OOM exception is thrown, pause and examine the object graph rooted at exml. What's actually occupying most of the memory?

Immutable strings are probably the killer here, since some xml parsers can wind up holding your entire doc in memory multiple times.

JSBangs
I'm new to examining memory problems in .NET, where is the 'object graph'?
Nefariousity
@Nefariousity, it's the tree of objects referenced by a given object. In the VS debugger, you should add `exml` to the watch list, then recursively expand all of its members for a few levels to get an idea of what sort of data it's holding onto.
JSBangs
Ooooh. I gotcha. Well, it looks like the xmldocument is throwing an OOM message before the exml is. Do you think renaming/restructuring the XML would reduce the file size? Could compression help?
Nefariousity
Compression *might* help. `XmlDocument` takes up a lot of memory because every single node in the DOM has to be copied multiple times, which you can reduced by using a less heavily-nested XML structure. The deeper the level of embedding in the XML document, the more memory it consumes.
JSBangs
Actually, I'm not sure that the level of embedding is that significant after all, since experimentation proved me wrong. Anyway.
JSBangs