Adding text to beginning and end of file in C#

Question

I have a process which picks up a series of "xml" files. The reason I put xml in quotes is that that the text in the file does not have a root element which makes in invalid xml. In my processing, I want to correct this and open up each file add a root node to the beginning and end of each file, and then close it up. Here is what I had in mind, but this involves opening the file, reading the entire file, tagging on the nodes, and then writing the entire file out. These files may be more than 20 MB in size.

        foreach (FileInfo file in files)
        {
            //open the file
            StreamReader sr = new StreamReader(file.FullName);

            // add the opening and closing tags
            string text = "<root>" + sr.ReadToEnd() + "<root>";
            sr.Close();

            // now open the same file for writing
            StreamWriter sw = new StreamWriter(file.FullName, false);
            sw.Write(text);
            sw.Close();
        }

Any recommendations?

Answer 1

I can't see any real improvement on this...which is kind of a bummer. Since there's no way to "shift" a file you'll always have to move the bytes in the entire file to inject anything at the top.

You may find some performance benefit by using raw streams rather than the StreamReader which has to actually parse the stream as text.

Answer 2

To avoid holding the whole file in memory, rename the original file, then open it with StreamReader. Then open the original filename with StreamWriter to create a new file.

Write the <root> prefix to the file, then copy data in large-ish chunks from the reader to the writer. When you've transferred all the data, write the closing </root> (note the forward slash if you want it to be XML). Then close both files and delete the renamed original.

char[] buffer = new char[10000];

string renamedFile = file.FullName + ".orig";
File.Move(file.FullName, renamedFile);

using (StreamReader sr = new StreamReader(renamedFile))
using (StreamWriter sw = new StreamWriter(file.FullName, false))
{
    sw.Write("<root>");

    int read;
    while ((read = sr.Read(buffer, 0, buffer.Length)) > 0)
        sw.Write(buffer, 0, read);

    sw.Write("</root>");
}

File.Delete(renamedFile);

Answer 3

If you do not want to do this is C#, it would be easy to handle at the commandline or in a batch file.

ECHO ^<root^> > outfile.xml
TYPE temp.xml >> outfile.xml
ECHO ^</root^> >> outfile.xml

This would assume that you have some existing process for getting the data files that this could be hooked into.

Answer 4

20 MB is not terribly much, but when you read it as a string, it will use about 40 MB of memory. That's not terribly much either, but it's processing that you don't need to do. You can handle it as raw bytes to reduce the memory usage, and to avoid decoding and re-encoding the data:

byte[] start = Encoding.UTF8.GetBytes("<root>");
byte[] ending = Encoding.UTF8.GetBytes("</root>");

byte[] data = File.ReadAllBytes(file.FullName);

int bom = (data[0] == 0xEF) ? 3 : 0;

using (FileStream s = File.Create(file.FullName)) {
   if (bom > 0) {
      s.Write(data, 0, bom);
   }
   s.Write(start, 0, start.Length);
   s.Write(data, bom, data.Length - bom);
   s.Write(ending, 0, ending.Length);
}

If you need to recude the memory usage much more, use a second file as Earwicker suggested.

Edit:
Added code to handle BOM (byte order mark).

Answer 5

Isn't there a way to open the file, insert a node at the very begining of the file then move the file pointer to the end of the file and write another node, without reading the file contents into a string?!