How do I get SpecFlow to expect an exception?

Question

I'm using SpecFlow, and I'd like to write a scenario such as the following:

Scenario: Pressing add with an empty stack throws an exception
    Given I have entered nothing into the calculator
    When I press add
    Then it should throw an exception

It's calculator.Add() that's going to throw an exception, so how do I handle this in the method marked [Then]?

Answer 1

Great question. I am neither a bdd or specflow expert, however, my first bit of advice would be to take a step back and assess your scenario.

Do you really want to use the terms "throw" and "exception" in this spec? Keep in mind the idea with bdd is to use a ubiquitous language with the business. Ideally, they should be able to read these scenarios and interpret them.

Consider changing your "then" phrase to include something like this:

Scenario: Pressing add with an empty stack throws an exception
    Given I have entered nothing into the calculator
    When I press add
    Then the user is presented with an error message

The exception is still thrown in the background but the end result is a simple error message.

Scott Bellware touches this concept in this Herding Code podcast: http://herdingcode.com/?p=176

Answer 2

As a newbie to SpecFlow I wan't tell you that this is the way to do it, but one way to do it would be to use the ScenarioContext for storing the exception thrown in the When;

try
{
    calculator.Add(1,1);
}
catch (Exception e)
{
    ScenarioContext.Current.Add("Exception_CalculatorAdd", e);
}

In your Then you could check the thrown exception and to asserts on it;

var exception = ScenarioContext.Current["Exception_CalculatorAdd"];
Assert.That(exception, Is.Not.Null);

With that said; I agree with scoarescoare when he says that you should formulate the scenario in a bit more 'business-friendly' wordings. However, using SpecFlow to drive the implementation of your domain-model, catching exceptions and doing asserts on them can come in handy.

Btw: Check out Rob Conery's screencast over at TekPub for some really good tips on using SpecFlow: http://tekpub.com/view/concepts/5

Answer 3

BDD can be practiced on feature level behavior or/and on unit level behavior.

SpecFlow is a BDD tool that focuses on feature level behavior. Exceptions are not something that you should specify/observe on feature level behavior. Exceptions should be specified/observed on unit-level behavior.

Think of SpecFlow scenarios as a live specification for the non technical stakeholder. You would also not write in the specification that an exception is thrown, but how the system behaves in such a case.

If you do not have any non technical stakeholders, then SpecFlow is the wrong tool for you! Don't waste energy in creating business readable specifications if there is nobody interested in reading them!

There are BDD tools that focus on unit level behavior. In .NET the most popular one is MSpec (http://github.com/machine/machine.specifications). BDD on unit-level can also easily be practices with standard unit-testing frameworks.

That said, you could still check for an exception in SpecFlow.

Here are some more discussion of bdd on unit-level vs. bdd on feature-level: SpecFlow/BDD vs Unit Testing BDD for Acceptance Tests vs. BDD for Unit Tests (or: ATDD vs. TDD)

Also have look at this blog post: Classifying BDD Tools (Unit-Test-Driven vs. Acceptance Test Driven) and a bit of BDD history

Answer 4

Changing the scenario not to have an exception is a probably good way to have the scenario more user oriented. However, if you still need to have it working, please consider the following: 1) Catch an exception (I really recommend catching specific exceptions unless you really need to catch all) in the step that invokes an operation and pass it to the scenario context.

[When("I press add")]
public void WhenIPressAdd()
{
   try
   {
     _calc.Add();
   }
   catch (Exception err)
   {
      ScenarioContext.Current[("Error")] = err;
   }
}

2) Validate that exception is stored in the scenario context

[Then(@"it should throw an exception")]
public void ThenItShouldThrowAnException()
{
      Assert.IsTrue(ScenarioContext.Current.ContainsKey("Error"));
}

P.S. It's very close to one of the existing answers. However, if you try getting value from ScenarioContext using syntax like below

var err = ScenarioContext.Current["Error"]

it will throw another exception in case if "Error" key doesn't exist (and that will fail all scenarios that perform calculations with correct parameters). So ScenarioContext.Current.ContainsKey may be just more appropriate

Answer 5

@jbandi I agree. SpecFlow is more for feature-driven specs.. while for more "technical" stuff, you can try MSpec.

what do you think on using in a same project both MSpec and SpecFlow, the first one for technical stuff, and the latter for more "business" features"?

Nevertheless, I think you could still write your features in SpecFlow in technical terms for unit level tests. In that case, it won't be a business person, but the developer himself.