Parameterized testing tools with automatic code exploration

views:

answers:

+1 Q:

Parameterized testing tools with automatic code exploration

I've checked and played a bit with Microsoft Pex. It's a very handy tool/library even though it's not as straight forward to write parametrized unit tests with it. But when used correctly it can provide much better test code coverage than writing manual unit tests. Not to mention time spent writing those tests either. And it integrates nicely with Visual Studio as well.

I was wondering whether there are any other tools out there that let you write parametrised unit tests and run them so internal algorithm intelligently chooses values to provide full code test coverage?

If some other tool/library was as easy to use as NUnit+Moq I'd be even more pleased. Pex has quite a steep learning curve (so does Moles I suppose).

Are there any plans for NUnit to support parametrised tests with inteligent parameter choosing in the future maybe? Anybody has any inside info about it?

If you're looking for other automatic testing tools for .NET, take a look at FsCheck. It's a port of Haskell's QuickCheck to F#. It's also usable from other .NET languages.

QuickCheck has also been ported to other platforms (ClojureCheck, ScalaCheck)

*Check libraries are not exactly like Pex, but there are similarities.

If you mean parameterized tests for .NET (as in writing tests that take parameters), all major test frameworks in .NET (NUnit, MbUnit, xUnit) support this.

Mauricio Scheffer 2010-10-01 01:06:28

Do they? Can you write a simple NUnit example of a parametrised test? It's the first time I hear of NUnit supporting parametrised unit tests... Does it run test explorations similar to how Pex works?

Robert Koritnik 2010-10-01 01:09:04

Well I've checked FsCheck and it says it tests against randomly generated values, which is even worse than manual tests if you ask me. Semi-complex units will end up very scarcely test code covered... Pex is very clever about choosing values, based on if statement conditions hence covering all paths much easier than trying some random set of finite values.

Robert Koritnik 2010-10-01 01:11:46

@Robert: you seem to be confusing *parameterized* tests with *generated* tests. Also, if it's the first time you hear about *Check libraries, take your time to read through.

Mauricio Scheffer 2010-10-01 01:15:08

Mauricio Scheffer 2010-10-01 01:16:39

No I'm not confusing the two. Pex does provide unit test generation but it can run custom parametrised unit tests as well. But thanks for pointing me to nUnit's documentation. But functionality that NUnit provides is basically only giving you the possibility of test consolidation. Code coverage by tests is still limited to what developers write (and they are usually lazy), compared to what Pex does, that provides certain values that are relevant.

Robert Koritnik 2010-10-01 01:17:26

You may as well check out Pex documentation and see what it does and how it does it. http://research.microsoft.com/en-us/projects/pex/pextutorial.pdf because it's something completely different to what NUnit does. Or FsCheck for that matter.

Robert Koritnik 2010-10-01 01:23:57

@Robert: thanks, I already know Pex.

Mauricio Scheffer 2010-10-01 01:24:37

@Robert: BTW: this is the standard usage of the term "parameterized test": http://xunitpatterns.com/Parameterized%20Test.html

Mauricio Scheffer 2010-10-01 01:28:37

@Robert: I never said that *Check testing is exactly like Pex, but there are similarities.

Mauricio Scheffer 2010-10-01 01:35:01

I've edited my question to not refer to parametrised test only but also to code exploration automation... Otherwise: yes I understand completely what you're saying and you did point me to some other valuable resources, that I think are not on-par with Pex at all. I'm sure you'd agree. Maybe there aren't any other tools with such automation.

Robert Koritnik 2010-10-01 01:46:47

ansaurus

tags:

views:

answers:

Parameterized testing tools with automatic code exploration

related questions