Re the first point: have you tried the (free) VIsual Studio Express Edition? For a lot of things, this is perfectly capable. You just don't get as many helpers / designers, and no plug-in support (for IDE extensions).
Re the second: excluding some nasty tricks, you can't create a pure native executable from .NET; it relies heavily on the framework being available on the local machine. An assembly is just a package of IL, and can be contained (typically) in either a dll, or bootstrapped into an exe that loads the assemblies entry-point; but in this scenario the exe is just a simple loader plus a regular assembly.
Actually, the CLR is more like the JVM; the "framework" is really just the equiavalent of a BCL. The main MS framework+CLR certainly has some Windows specific optimizations, but other runtimes/frameworks (compact, micro, Silverlight, Mono) will have different optimizations.
Re multi-core - you have full threading support (for doing it yourself) - but the main automated multi-core support will (hopefully) be in .NET 4.0 with the "parallel extensions" work.
Re the last point: should be very familiar indeed. Actually, if you want to do some comparisons, "reflector" (free) can take a compiled assembly and show you the code in either C# or delphi (or a few others).
[update re questions]
IL = Intermediate Language; .NET doesn't compile to native CPU instructions, but to something in-between that becomes CPU instruction at runtime (compiled "Just In Time" (JIT) on a method-by-method basis). This means that the JIT compiler can optimize the same IL for the local machine. You can do this in advance using NGen.
CLR = Common Language Runtime; essentially the VM
BCL = Base Class Library; the set of classes shared my many apps
Re deployment: first, install the .NET framework on the client ;-p
After that - various options. At the simplest level, you can just copy the exe/etc onto the local machine and run. For example, I use "robocopy" to push code to web-servers.
For full local installs of a complex client app, msi is an option (and the full VS IDE will help you with this).
For simple clients, you can use ClickOnce - which packages the app into a signed bundle that provides self-updating etc capabilities and allows you to make statements about what security you need (full trust, etc). Express Edition allows you to author ClickOnce packages. ClickOnce can even be used on locked down clients where the user can't install apps, since the app is isolated and sand-boxed.
Finally, you can run a .NET app off a network share, but there are some security implications: the "Code Access Security" layer won't give a network share "full trust" (although there were some recent changes to this so that mapped (F: etc) shares are trusted). So you'd need to use CASPOL at each client to trust the code. ClickOnce would be easier ;-p