What, if any Add-ins do you have installed?
Edit:
One suggestion I would have then is to systematically disable each of your add-ins and see if performance changes and if it does research the culprit and see if there any updates available.
Uninstall any MS Enterprise Framework addin's you may have.
Delete the VS temp directory (and the Windows one).
Do you use TFS? Perhaps the server is a bit sleepy, that will make it freeze for a few minutes, but is ok afterwards.
You can see exactly what VS is doing at any given moment, if you attach a debugger to the devenv.exe process and hit Break when it hangs. Then load the symbols from Microsoft Symbols server and show the call stack for the VS main thread.
I wrote a very detailed article about how to debug crashes and hangs here: http://blogs.msdn.com/kirillosenkov/archive/2008/12/07/how-to-debug-crashes-and-hangs.aspx
From the call stack it should be obvious what is causing the delay.