AMD's ACML is a free download, but it is binary only, not open source, and native code, not .NET.
Performance is generally superior to the Netlib.org code, and generally roughly the same as Intel's MKL -- which is not free IIRC.
The download includes one sample that demonstrates how to bind it to C#. Not any different from calling any other C or C++ library from C#.
Library implements BLAS, LAPACK, FFTs, and RNGs.
http://developer.amd.com/cpu/Libraries/acml/downloads/pages/default.aspx
EDIT TO RESPOND TO COMMENT:
On an Intel CPU, AMD's ACML will perform approximately as well as Intel's MKL, but it depends on the algorithm, matrix sizes, number of cores, memory topology and speed, etc. etc. etc. Your mileage may vary. The only way to tell for sure is to run your own benchmark. In some cases, ACML is faster than MKL even on Itel hardware.
Either one will be significantly faster than any "naive" implementation for large matrixes. Both are architected to use multiple threads on multicore processors, and have hand-tweaked assembly language kernels and a lot of tuning for the cache behaviours on various machines.
For small matrixes, performance is generally a don't-care, since any modern cpu can solve a small matix in just a few milliseconds, even using the simplest code. In that case, you're only using a library to avoid writing and debugging code that has been written hundreds of times already.