Is it possible to test the return value of Haskell I/O functions?

Question

Haskell is a pure functional language, which means Haskell functions have no side affects. I/O is implemented using monads that represent chunks of I/O computation.

Is it possible to test the return value of Haskell I/O functions?

Let's say we have a simple 'hello world' program:

main :: IO ()
main = putStr "Hello world!"

Is it possible for me to create a test harness that can run main and check that the I/O monad it returns the correct 'value'? Or does the fact that monads are supposed to be opaque blocks of computation prevent me from doing this?

Note, I'm not trying to compare the return values of I/O actions. I want to compare the return value of I/O functions - the I/O monad itself.

Since in Haskell I/O is returned rather than executed, I was hoping to examine the chunk of I/O computation returned by an I/O function and see whether or not it was correct. I thought this could allow I/O functions to be unit tested in a way they cannot in imperative languages where I/O is a side-effect.

Answer 1

I'm sorry to tell you that you can not do this.

unsafePerformIO basically let's you accomplish this. But I would strongly prefer that you do not use it.

Foreign.unsafePerformIO :: IO a -> a

:/

Answer 2

Within the IO monad you can test the return values of IO functions. To test return values outside of the IO monad is unsafe: this means it can be done, but only at risk of breaking your program. For experts only.

It is worth noting that in the example you show, the value of main has type IO (), which means "I am an IO action which, when performed, does some I/O and then returns a value of type ()." Type () is pronounced "unit", and there are only two values of this type: the empty tuple (also written () and pronounced "unit") and "bottom", which is Haskell's name for a computation that does not terminate or otherwise goes wrong.

It is worth pointing out that testing return values of IO functions from within the IO monad is perfectly easy and normal, and that the idiomatic way to do it is by using do notation.

Answer 3

I like this answer to a similar question on SO and the comments to it. Basically, IO will normally produce some change which may be noticed from the outside world; your testing will need to have to do with whether that change seems correct. (E.g. the correct directory structure was produced etc.)

Basically, this means 'behavioural testing', which in complex cases may be quite a pain. This is part of the reason why you should keep the IO-specific part of your code to a minimum and move as much of the logic as possible to pure (therefore super easily testable) functions.

Then again, you could use an assert function:

actual_assert :: String -> Bool -> IO ()
actual_assert _   True  = return ()
actual_assert msg False = error $ "failed assertion: " ++ msg

faux_assert :: String -> Bool -> IO ()
faux_assert _ _ = return ()

assert = if debug_on then actual_assert else faux_assert

(You might want to define debug_on in a separate module constructed just before the build by a build script. Also, this is very likely to be provided in a more polished form by a package on Hackage, if not a standard library... If someone knows of such a tool, please edit this post / comment so I can edit.)

I think GHC will be smart enough to skip any faux assertions it finds entirely, wheras actual assertions will definitely crash your programme upon failure.

This is, IMO, very unlikely to suffice -- you'll still need to do behavioural testing in complex scenarios -- but I guess it could help check that the basic assumptions the code is making are correct.

Answer 4

The way I would do this would be to create my own IO monad which contained the actions that I wanted to model. The I would run the monadic computations I want to compare within my monad and compare the effects they had.

Let's take an example. Suppose I want to model printing stuff. Then I can model my IO monad like this:

data IO a where
  Return  :: a -> IO a
  Bind    :: IO a -> (a -> IO b) -> IO b
  PutChar :: Char -> IO ()

instance Monad IO where
  return a = Return a
  Return a  >>= f = f a
  Bind m k  >>= f = Bind m (k >=> f)
  PutChar c >>= f = Bind (PutChar c) f

putChar c = PutChar c

runIO :: IO a -> (a,String)
runIO (Return a) = (a,"")
runIO (Bind m f) = (b,s1++s2)
  where (a,s1) = runIO m
        (b,s2) = runIO (f a)
runIO (PutChar c) = ((),[c])

Here's how I would compare the effects:

compareIO :: IO a -> IO b -> Bool
compareIO ioA ioB = outA == outB
  where ioA = runIO ioA ioB

There are things that this kind of model doesn't handle. Input, for instance, is tricky. But I hope that it will fit your usecase. I should also mention that there are more clever and efficient ways of modelling effects in this way. I've chosen this particular way because I think it's the easiest one to understand.

For more information I can recommend the paper "Beauty in the Beast: A Functional Semantics for the Awkward Squad" which can be found on this page along with some other relevant papers.

Answer 5

You can test some monadic code with QuickCheck 2. It's been a long time since I read the paper, so I don't remember if it applies to IO actions or to what kinds of monadic computations it can be applied. Also, it may be that you find it hard to express your unit tests as QuickCheck properties. Still, as a very satisfied user of QuickCheck, I'll say it's a lot better than doing nothing or than hacking around with unsafePerformIO.