Is there any small tool that gives me access to the data gathered by the Intel CPU Counters (like L1/L2 cache misses, branch prediction failures ... you know there are hunderts of them on modern Core2 CPU's).
It must work on Windows (while being able to use it with Solaris, FreeBSD, Linux, MacOSX would of course be nice).
My first though was to look up the What every programmer should know about memory article series to see if there were any references to source code used (to see if I could find any examples) although I did not see any right away. Nevertheless I assume you will find those articles interesting in any case.
You didn't write of your are looking for a application or for a library.
For Windows there is Intel VTune. But this not exactly an small tool. For linux I have used oprofile, which works without kernel patches.
On OS X, Shark lets you get data from the PMCs. I'm not sure what's available on Windows other than Intel's tools (VTune, as mentioned by drhirsch).
Try this http://icl.cs.utk.edu/papi/ It is a full library that allows you to read any CPU counters data, works both on Windows and Linux [and other OS]
PAPI is a very promising lead, however, I believe they discontinued support for Windows (and therefore .NET C#) quite a few years ago.
On the windows front, Visual Studio 2010 Premium comes with performance explorer. If you run any project or binary in instrumentation mode, you can get access to hardware events such as instructions retired.
The results can be somewhat mixed and inconsistent depending external factors, but it integrates with Visual Studio nicely and you get detailed counts (avg, maximum, total) on a per method/module level.
Intel V-tune performance analyzer also exposes these natively. I haven't played with this tool yet but it might be a more flexible API than what Visual Studio 2010 exposes.