Hello, we have a client who wants us to store xml table dumps from their database in an xml file (the xml file will contain potentially any combination of text or binary files (binary being base64 encoded) and xml database dumps).
To achieve this, we obviously need to encode the xml special characters inside the database dumps. The client wants to be able to read the resulting xml file, including the contents of database tables where they're text. This rules out using base64 encoding for the database dumps. We decided to go with html encoding (we're using .net so we have HttpUtilities.HtmlEncode).
The problem we're hitting is that because of the potential size of these tables, the resulting encoded xml has to be read back in chunks and can't necessarily be held in a single string in memory. This is causing us problems on decoding when an encoded character gets split across two chunks, as the .net HtmlDecode function only takes a string as a parameter.
What would be the best way to handle this decoding to get around the memory requirements of storing the whole thing as a string? As I see it, the ideal solution would be another version of HtmlDecode that read and writes to streams, but is there a simpler way to do it using the existing HtmlDecode? Or is there (even better) an existing part of the framework that will do this for us?
Thanks for any help.