Program Execution Speed Testing

Question

I have a C# app in which I am testing 3 ways to do something with XML.

I read a string of XML get a value out and then add it to a custom collection.

I am testing XMLDocument, XDocument and String manipulation to find the node value, in that order recording a start and end time for each.

However if I say mix the order up I get different speeds reported. Also when I press the button again to perform the test again the one that was slowest initially sometimes takes the least on subsequent tests.

I'm sure its all related to JIT and Garbage collection but what is the truest way to determine what runs the fastest.

I have downloaded EQATEC profiler but I'm lost on what to do with it.

Can someone advise me the best way to determine what methods run the quickest.

Thanks

Answer 1

Try using a profiling tool like ANTS from Red-Gate or dotTrace from JetBrains. The profilers will show you what percentage of time is spent in each method allowing you to identify which is the faster executed method.

Answer 2

Another profiler that might be worth trying is Slimtune.

Answer 3

Isn't it sufficient to each method with a set of sample input while measuring the time with StopWatch?

Answer 4

Learn to use the tools you have:

http://www.eqatec.com/tools/profiler/guide

If you're getting inconsistent results, do the scientific thing and make control cases:

• Either test multiple times in a loop so the first case is averaged out among the rest of the runs
• or just test the first case (restarting the app each time).

Answer 5

You can use Timing class from "Data structures and algorithms using c#" by Michael McMillian.

Roughly it'll be as follows:

TimeSpan startingTime, duration;
GC.Collect();
GC.WaitForPendingFinalizers();
startingTime = Process.GetCurrentProcess().Threads[0].UserProcessorTime;
DoLengthyWork();
duration = Process.GetCurrentProcess.Threads(0).UserProcessorTime.Subtract(startingTime);

Answer 6

Jon Skeet has written a pretty good bechmarking 'framework' for measuring simple operations like this that I think would be pretty well suited to checking each of those methods, which you could find here: http://www.yoda.arachsys.com/csharp/benchmark.html (note, this is just timing the operations, not profiling the heap usage or anything like that).

An important point to keep in mind (Jon notes it on that site above, and Eric Lippert is fond of mentioning it whenever the topic comes up) is that when you run code multiple times, it only gets JIT'd the first time, so if you want to measure the true differences for each method, measure at least twice and compare the second runs - I would imagine this is the result of your shifting orders.