C# StdIn/StdOut Performance

Question

I am launching a process and writing an intruction xml file to the destination process over stdin. There is only 2k of data but it is taking almost half a second to transfer that much data, which seems like way too much to me. What I am measuring is the amount of time it takes the child process to consume the data.

Any suggestions for how to speed this up?

Parent process:

    ProcessStartInfo psi = new ProcessStartInfo(path, args);
    psi.CreateNoWindow = true;
    psi.UseShellExecute = false;
    psi.RedirectStandardInput = true;
    Process p = Process.Start(psi);
    p.StandardInput.Write(stdin);
    p.StandardInput.Close();

Child Process:

    Stream s = Console.OpenStandardInput();
    StreamReader sr = new StreamReader(s);
    return sr.ReadToEnd();

For the record, I did try one change where I wrote out the length of the data first and read that exact number of bytes back in to avoid waiting for the stream to close, but that didn't affect the performance at all.

I am spawning lots of little worker processes so the extra .5 seconds per worker is killing me!

Edit: I appreciate all of the comments, but I am specifically looking for any way to improve the stdin performance. The .5s I mentioned is being measured around the child process code:

    Stopwatch sw = new Stopwatch();
    sw.Reset();
    sw.Start();
    string stdin = ReadStdIn();
    sw.Stop();
    Console.WriteLine("Ellasped time to read stdin: " + sw.EllapsedMilliseconds);

2nd Edit: In this case, spawning threads isn't a possibility. For reasons beyond the scope of this question, I need each working to have it's own memory space (limited to 32 bit 2gb).

Answer 1

Have you tried doing this same thing except not reading/writing the data? Ie. measuring the time it takes your parent program to launch the child, have it prepare to run, run, and exit?

Since your 'child' code looks like C# as well, it may be the overhead of loading your child exe + libraries, starting the .Net runtime, etc. that's taking 0.5s, not the writing/reading of your data.

That said, starting new processes on windows isn't cheap (at least not like it is on most *nix variations). You may want to look at using threads instead of processes if you're that concerned about the timing here.

Answer 2

It may be that launching the child processes is taking so long, not reading from stdin. Remember that the CIL needs to be JITted afresh each time.

If that is the case, consider running ngen install on the EXE that you are spawning. This will dramatically improve its launch time by basically JITting the child EXE once and not every time you run it.