Pex users: what are your Impressions of Pex and Automated Exploratory Testing in general?

Question

Those of you who have used Pex, what do you think its advantages and disadvantages are of Pex as a tool?

Also, what do you think are the advantages and disadvantages of "Automated Exploratory Testing" in general, as a supplement to TDD/Unit Testing?

Answer 1

I think Pex as an exploratory testing tool is really intriguing. In that regard, I see it as something I'd want to hand off to QA to use.

As a TDD tool, it needs some work, as TDD is a design activity. However, I do like the direction that Peli is heading. THere's something to be said for automated assisted design. For example, just because TDD is a design tool, there's no reason I can't have an automated tool point out potential edge cases while I'm designing, right? Build quality in from the start.

Check out this post in which Peli uses Pex in a TDD style workflow. http://blog.dotnetwiki.org/TDDingABinaryHeapWithPexPart1.aspx

Answer 2

Pex lets your write parameterized unit tests. In that sense, it totally fits into the TDD/unit testing flow: write the test, have Pex 'explore' it, find some failing tests, fix the code, and so forth.

The big advantage is that you can express your tests for classes of inputs, not just a couple hard-coded values. This gives more expressiveness for writing tests and also forces to think about the invariant/expectation that your code should fullfill (i.e. it's harder to write assertions).

Answer 3

I'm really pumped about Pex. It will provide tests for edege cases that you won't ever dream up, especially if your team is small and the person writing the methods is the same as the person writing the tests.

It will also provide contractual obligations that your methods will obey.

Answer 4

If you look for literature on writing Theories (google David Saff) - which are a more general way of writing unit tests, and use Pex as a theory explorer I've found a step change in productivity from my experience so far. I've just wrote a blog post detailing my experiences of Pex in TDD, here: http://taumuon-jabuka.blogspot.com/2009/01/theory-driven-development-using_11.html

and as I said - I see it as TDD on steroids! It in no way replaces TDD, but enhances the activity.

Answer 5

Test-first development makes you structure your code for testability. In this respect, Pex finds clever and awkward paths thru your code, helping out beyond simple coverage metrics.

Major forte of Pex with Moles is enabling of tracking side effects when doing Brownfield development: run Pex once and save outputs, then apply code changes, and run Pex again to see what got broken.