Or I have to configure IIS specifically for this ? Do I have to use Web garden to take advantage of multiple cores ?
IIS will by default take advantage of a multi-core CPU.
[If you want to set processor affinity, you need to explicitly configure this.]
It's actually the other way around. Processor affinity is switched off in IIS6, which means the load will be distributed across all available CPUs. If you want to restrict it, you can use processor affinity to stop this default behaviour.
As you want it to use all available CPUs, the default installation will work as you want.
A web server is multi-threaded (at least, IIS is).
Any multi-threaded program will be distributed over all available cores by the Windows scheduler by default (though "processor affinity" can be tuned in the task manager, for example). Whether you are using ASP.NET or any other application server/language makes no difference here.
Also ASP.Net pages can run asynchronously and they can be multithreaded.