Most of the speed hit you are seeing is MSTests firing up the ASP.NET development server every time you want to run even a single test.
I keep as much of my application as possible in separate libraries outside of my ASP.NET application, so that they can be unit tested independently. This avoids the hit you take when it fires up the ASP.NET development server during unit testing.
When mocking controllers, I don't see any reason why you couldn't remove the attributes that tell the testing system to fire up the development server. After all, isn't the whole purpose of mocking to take out the "outside" components?
Similarly, the whole purpose of using repositories for your models is so that you can inject mocking objects for testing. These tests don't need the development server either.
As far as the views go, I wouldn't write unit tests for these. Keep them as thin as possible, and test them manually by visual inspection.
A different set of tests can be included for your models and controllers that include the ASP.NET development server. These tests would be part of your Integration Test suite.