I'm running some performance tests on some .NET code that processes lots of data. I want some tests that ensure the garbage collector isn't influencing my results. How do I temporarily pause the garbage collector?
+10
A:
There is not a way to do this through the BCL APIs.
Turning it off for the profiling of a particular algorithm is also not a great idea because it will yield false results. The garbage collector will run during the execution of your program. Profiling without the GC could hide real problems with your algorithm if it causes lots of garbage collections in the real world.
JaredPar
2009-09-24 17:40:45
So, I totally didn't appreciate how often the GC runs! Now that I understand how often the GC collects I understand why my question is mute. Thank you all!
MrPhil
2009-09-24 19:52:50
+2
A:
The real world will include garbage collection. Since this is a random arrival process (poisson), it might be worthwhile doing it in the monte carlo fashion and doing a few 1000 profiling runs and averaging the results.
whatnick
2009-09-24 17:49:59
The problem is that the garbage collection's behavior in the test environment and in production aren't necessarily the same. I'm also comparing the results to non-managed approaches to the problem and think the data would be valuable for comparison.
MrPhil
2009-09-24 18:01:26
Will it be a fair comparison if you ran in unmanaged mode - i.e. different language altogether ?
whatnick
2009-09-24 18:03:33
I not positive what you are getting at, but I will say this: If one of the non-managed approaches is in a different language is it fair to penalize the managed approach because during the test it was unlucky enough to have the garbage collection routine run? No, I don't think so, because I'm interested in the comparison of the parts and the garbage collector is a system for the whole. The non-managed approach has to worry about memory too, but is able to better optimis (in theroy) in speed senitive areas.
MrPhil
2009-09-24 18:09:55
Since unmanaged mode gives you more predictable behaviour and control that seems to bet the way to go. You can make forced calls to the garbage collector at certain points in managed mode and see how long it takes. There are also GC performance counters that can be used http://blogs.msdn.com/maoni/archive/2004/06/03/148029.aspx.
whatnick
2009-09-24 18:23:43
So, that gives me an idea. I can run the test monte carlo fashion like you suggest but throw out the runs that had a collection occur by checking the GC.CollectionCount before and after. That's seems like a good compromise.
MrPhil
2009-09-24 18:29:45
My idea is useless. The GC runs so often that a test run is never "good."
MrPhil
2009-09-24 19:54:20