opinions on yourkit .NET profiler?

Opinion? Yeah. While you're deciding which profiler to buy or not, give this a try.

ADDED: @Max: Step-by-step instructions: The IDE has a "pause" button. Run your app under the IDE, and while it is being subjectively slow, i.e. while you are waiting for it, hit the "pause" button. Then get a snapshot of the call stack.

To snapshot the call stack, what I do is display it (that's one of the debug windows). In the IDE options you can find options for what to display in the stack view. I turn off the option to display the function arguments, because that makes the lines too long. I'm interested in the line number where the call takes place and the name of the function being called. Then, in the call stack view, you can do a "Select All" and then a "Copy", and then paste it into Notepad. It's a little clumsy, I know, but I used to write them down by hand.

I take a few samples this way. Then I look at them for lines that appear on more than one sample, because those are the time-takers. Some are simply necessary, like "call main", but some are not. Those are the gold nuggets. If I don't find any, I keep taking samples, up to about 20. If I still don't find any (very unusual) by that time the program is pretty well optimized. (A key point is that every time you do this the program gets faster, and in the process the remaining performance problems get relatively bigger & easier to find. I.e. not only does the program get faster by a certain ratio R, but the remaining problems get bigger, percentage-wise, by that same ratio.)*

Another thing I do in this process is ask myself what the program is doing and why in that sample. The "why" is very important, because that is how you tell if a line is actually necessary or could be replaced with something less costly. If I'm not sure why it is there, I single-step it a little, maybe look at the data, or maybe let it return up a few levels (shift-F11) until I understand what it's doing. That's about all there is to it.

Existing profilers could help with this process if they actually perform stack sampling, retain the samples, rank lines by what percent of samples contain them, and then let you study individual samples in detail. Maybe they will at some time, but they don't now. They are hung up on issues like efficiency and measurement.

*Suppose your code spends 90% of its time doing X, and 9% of its time doing Y, both not really necessary. Take a small number of samples, and you will see X, but probably not Y. If you fix X you get a 10x speedup. Do the sampling again (you may have to wrap an outer loop around the program so you can take samples). Now you see Y with certainty because now it takes 9% x 10x = 90%. Fixing it gives you another 10x, for overall 100x speedup.

Disclaimer: my suggestion is based on the assumption that you are serious about performance profiling, and would like to find out the accurate % improvement of your tuning effort.

Based on my experience with performance profilers like ANTS and DotTrace, I think they are useful while all fail at one point or another.

The problem with general purpose performance profilers is they add too much overhead and render the result inaccurate. But they do however, a good job at finding heavy operations. When you use a profiler and find an operation taking 40% of the total process time, does it actually take 40%? Maybe not. But it probably does take a lot of time to run.

So to answer your question, it doesn't matter whether yourkit is "excellent" or "satisfactory", unless it sucks so bad that you hear horrible stories about it. Because it is going to give you a big picture of performance bottlenecks in your app. Giving you an idea where to insert your own time logging instructions to find out the actual performance profile of your application.

PS. I would also suggest you take a read about performance profiling, which would give you an overall understanding of the concept and help understand other people's suggestions.

Somebody gave me a drive-by. That's evidence that it's controversial. Here's another explanation: http://stackoverflow.com/questions/406760/whats-your-most-controversial-programming-opinion/1562802#1562802

Mike Dunlavey 2009-10-14 16:07:20

Your way sounds interesting but to be honest I am not quite sure about how to do this using C# and Visual Studio 2008 (in short: I did not understand it yet). Any "step-by-step" instruction on this?

Max 2009-10-17 13:38:22

@Max: I answered above.

Mike Dunlavey 2009-10-17 15:02:14

Now thats an interesting way of optimization / measuring. Never saw or heard something like this. But it sounds very interesting, I will try it out! Thank you for your thorough explanation!

Max 2009-10-28 10:26:42

@Max. You're welcome. Part of the mind-set of that method is that precise measurement need not be a prerequisite to finding things to optimize. The targets are large and at close range. Good luck.

Mike Dunlavey 2009-10-28 12:40:09

@Bill: I would suggest being serious about performance improvement does not require accurate a priori measurement of the gain to be achieved, because it can be measured accurately a posteriori. Nor is it important that the method have minimal impact on performance (witness ValGrind). What is important is that it *locate* problems quickly and precisely. Profilers do not yet do that (though they could).

Mike Dunlavey 2009-10-27 00:03:27

@Mike: I agree that it is important to find the problem quickly. But I'm sort of lost on your priori/posteriori argument, do you mean that performance improvement is posteriori knowledge rather than priori knowledge? In which case I do agree, and that's what I'm proposing.

Bill Yang 2009-10-27 05:44:45

@Bill: For example, suppose there is a function call instruction that happens to be on the stack exactly 50% of the time (so that is exactly what it costs). Suppose you take 5 random stack samples. You could see it on 0 samples (very unlikely), 1 sample (unlikely) 2 samples (estimating 40%), 3 (estimating 60%), 4 (unlikely), or 5 (very unlikely). So you know precisely where it is, and fixing it will give you up to 50% reduction, even though you didn't know ahead of time exactly how much it would save you. Does that help?

Mike Dunlavey 2009-10-27 16:35:43

... So the samples did not measure the problem exactly, but they did locate it exactly. If you need more precise measurement of the speedup, you can always time it before/after fixing it. 100s of samples were not needed. The speed of taking a sample was irrelevant. You can take a sample and look at it all day before taking another - it will not affect the results.

Mike Dunlavey 2009-10-27 16:41:49

... I am totally mystified that such an unbelievably simple concept is understood by so few.

Mike Dunlavey 2009-10-27 17:00:58

@Bill Yang: Thank you for your answer! Yeah, I am kind of looking for the "horrible stories", none have come so far, so at least thats a good sign. ;)

Max 2009-10-28 10:28:56

@Mike: how do you know the method took 50% of the time with sampling? Also fixing a 50% method does not give you 50% time reduction, unless you removed it.

Bill Yang 2009-10-28 16:33:46

@Bill: You don't know it took 50%, it takes some percent, so you can just suppose for argument it is 50%, or 40, or 20, or whatever. (If you really want to know, take a huge number of samples.) My point is, it has some cost and it is not little so you don't need to know it accurately - you just need to know it is worth fixing.

Mike Dunlavey 2009-10-28 22:54:35

... To your second point, if its cost happens to be 50% (just suppose), and if you can replace it with something that takes a lot less time, then you can save up to 50%. Maybe you only save 40%. That's not too shabby. In experience, much of the time the problem is something where you can basically save the whole cost. For example if you query a DB to get some info, and then later you query it again multiple times, because that was easier to code than saving the data from the first query. Another example, incrementing an iterator instead of just i++.

Mike Dunlavey 2009-10-28 22:59:35

@Mike: I'm not arguing that one must get accurate % (it comes at a cost), hence the disclaimer. It is really useful though, when you need to justify your effort to stakeholders.

Bill Yang 2009-10-29 00:06:03

@Bill: As soon as I see something appear on >1 sample, I know it is worth fixing. If somebody wants to know what it costs, I just take 10 samples, or 20 if it's small, and tell them the % of samples I saw it on, and include the word "roughly". Since I know what the problem is, and usually it is easy to fix, I have no trouble making the sale.

Mike Dunlavey 2009-10-29 00:54:03

... if the % happens to be large, like 60% or 90%, it's easy to make the sale. I just tell them the potential speedup is 2.5x or 10x, because that's what it is. Usually they can't resist. On the other hand, if it's small, like 10%, it's hardly worth fixing unless it's really easy. What usually happens is that, as you fix big problems, the small problems become a larger percentage, so easier to find, so the whole thing takes off like a multi-stage rocket.

Mike Dunlavey 2009-10-29 01:30:21

I realized that this issue is entirely subjective, so how about let's leave it as that?

Bill Yang 2009-10-29 16:00:13

@Bill: If I thought it were subjective I wouldn't have so much to say, but thanks for putting up with my discourse. Check this out: http://stackoverflow.com/questions/406760/whats-your-most-controversial-programming-opinion/1562802#1562802

Mike Dunlavey 2009-10-30 13:43:58

ansaurus

tags:

views:

answers:

opinions on yourkit .NET profiler?

related questions