Skip unit test if csv file not found

Question

Hi,

I have a number of unit tests which rely on the presence of a csv file. They will throw an exception if this file doesn't exist obviously.

Are there any Gallio/MbUnit methods which can conditionally skip a test from running? I'm running Gallio 3.1 and using the CsvData attribute

[Test]
[Timeout(1800)]
[CsvData(FilePath = TestDataFolderPath + "TestData.csv", HasHeader = true)]
public static void CalculateShortfallSingleLifeTest()
{
    .
    .
    .

Thanks

Answer 1

An easy way would be to include an if condition right at the start of the test that would just execute any code in the test if the CSV file can be found.

Of course this has the big drawback that tests would be green although they haven't actually run and asserted anything.

I agree with Grzenio though, if you have unit tests that rely heavily on external conditions, they're not really helping you. In this scenario you will never really know whether the unit test ran successfully or was just skipped, which contradicts what unit tests are actually for.

In my personal opinion, I would just write the test so that they correctly fail when the file is not there. If they fail this is an indicator that the file in question should be available on the machine where the unit tests run. This might need some manual adjustments at times (getting the file to the computer or server in question), but at least you have reliable unit tests.

Answer 2

According to the answer in this question, you'll need to make a new TestDecoratorAttribute that calls Assert.Inconclusive if the file is missing.

Assert.Inconclusive is very appropriate for your situation because you aren't saying that the test passed or failed; you're just saying that it couldn't be executed in the current state.

Answer 3

What you have here is not a unit test. A unit test tests a single unit of code (it may be large though), and does not depend on external environmental factors, like files or network connections.

Since you are depending on a file here, what you have is an integration test. You're testing whether your code safely integrates with something outside of the control of the code, in this case, the file system.

If this is indeed an integration test, you should change the test so that you're testing the thing that you actually want tested.

If you're still considering this as a unit test, for instance you're attempting to test CSV parsing, then I would refactor the code so that you can mock/stub/fake out the actual reading of the CSV file contents. This way you can more easily provide test data to the CSV parser, and not depend on any external files.

For instance, have you considered that:

• An AntiVirus package might not give you immediate access to the file
• A typical programmer tool, like TortoiseSvn, integrates shell overlays into Explorer that sometimes hold on to files for too long and doesn't always give access to a file to a program (you deleted the file, and try to overwrite it with a new one? sure, just let me get through the deletion first, but there is a program holding on to the file so it might take a while...)
• The file might not actually be there (why is that?)
• You might not have read-access to the path
• You might have the wrong file contents (leftover from an earlier debugging session?)

Once you start involving external systems like file systems, network connections, etc. there's so many things that can go wrong that what you have is basically a brittle test.

My advice: Figure out what you're trying to test (file system? CSV parser?), and remove dependencies that are conflicting with that goal.

Answer 4

In Gallio/MbUnit v3.2 the abstract ContentAttribute and its concrete derived types (such as [CsvData] have a new optional parameter that allows to change the default outcome of a test in case of an error occured while opening or reading the file data source (ref. issue 681). The syntax is the following:

[Test]
[CsvData(..., OutcomeOnFileError = OutcomeOnFileError.Inconclusive)]
public void MyTestMethod()
{
  // ...
}