CryptoAPI (often abbreviated as "CAPI") is a set of C-based API which applications (under Windows) can use to access cryptographic services. In particular, cryptographic operations which use keys (e.g. RSA encryption) are delegated to pluggable modules called "CSP" (Cryptographic Service Providers). The CSP performs the cryptographic operation itself. Conceptually, only the CSP knows where the key actually is; it can be viewed as the driver for the piece of (virtual or real) hardware which stores the key.
The RSACryptoServiceProvider class is part of .NET and offers an access to a CSP which computes RSA. That class is part of a set of .NET classes which are a kind of .NET counterpart of CryptoAPI.
RSACryptoServiceProvider is what you use in the .NET world; most of the time, making a roundtrip to the C world in order to invoke CryptoAPI to end up using the same CSP looks like a superfluous complication. Also, RSACryptoServiceProvider has the good taste of using the standard (big-endian) representation of RSA signatures and encrypted messages, contrary to CryptoAPI, which "reverses" the bytes.