Categorizing MsTest Cases

Question

I'm looking for the easiest way to categorize my tests so I can run them separately.

My organization would be something like the following

• Units Tests
• Integrations Tests
• Systems Test

The reasoning for this is all our testing hasn't evolved to be completely automated yet. Most of our unit tests are completely automated, while integration and system tests aren't. However all tests are still useful to invoke on developers machines.

For simplicity I'd like it if we could just commit our different types of tests with distinctive attributes [IntegrationTest] or [TestCategory("Integration")] so our build server only runs the tests we want.

I'm aware nUnit has test categories but our team likes the Microsoft Stamp of approval on MsTest + IDE integration.

How has your team solved this problem?
Are there extensions that provide this functionality?
Will this be built into .Net 4.0?

Thanks

Answer 1

These are basically different types of tests and thus should be placed in different projects.

There are a lot of good reasons to keep integration tests and unit tests separate, and it will also help in this case.

• Each class library subjected to testing should have one (or more) associated unit test projects.
• Integration tests should go into one (or more) separate test projects so that you can vary them as you need.

You can have different solution files that include some, but exclude other test projectz depending on the type of test suite you wish to run.

Note that this strategy will work with MSTest any other testing framework you may decide to use in the future, which I would consider an extra benefit.

Answer 2

Have you looked at the Test Lists feature? Click Test -> Windows -> Test List Editor to bring up the UI for managing test lists. Right click List of Tests and choose New Test List giving it a name and saving. Afterward you drag tests into the new group creating a subset of All Tests that can be run together.

Another option is to use the filter feature of the Test View tool window to select tests with similar traits. It will match partials, so you can switch to Test Name and enter something like "Asp" to match all tests containing that string and then run the results together by making a selection of all filtered items. The downside is you'd potentially have to rename your tests.

Along the same lines, and closer to what you were thinking, you could right click tests in the Test View window, choose Properties and populate the Description property or as you suggested by decorating tests with the Description attribute. Afterward you use can use the prior approach to filter tests (this time filtering on Description) albeit not the intended use of the field, it could achieve your desired result

Answer 3

Yes, there is a TestCategory attribute in .NET 4.0 Beta 2 (it won't be ruled out in RTM as Beta 2 is feature complete AFAIK)

http://msdn.microsoft.com/en-us/library/ms182489%28VS.100%29.aspx