Seems like it would be a good way to introduce some people to unit testing.
+3
A:
Well for one thing, the documentation for doctest talks about "interactive Python sessions". There's no equivalent of that in C#... so how would the output be represented? How would you perform all the necessary setup?
I dare say such a thing would be possible, but personally I think that at least for C#, it's clearer to have unit tests as unit tests, where you have all the benefits of the fact that you're writing code rather than comments. The code can be checked for syntactic correctness at compile-time, you have IntelliSense, syntax highlighting, debugger support etc.
If you're writing code, why not represent that as code? Admittedly it's reasonably common to include sample code in XML documentation, but that's rarely in the form of tests - and without an equivalent of an "interactive session" it would require an artificial construct to represent the output in a testable form.
I'm not saying this is a bad feature in Python - just that it's one which I don't believe maps over to C# particularly well. Languages have their own styles, and not every feature in language X will make sense in language Y.
Jon Skeet
2010-10-19 05:26:44
C# has several interactive shells, one of the best being Mono's [C# REPL](http://www.mono-project.com/CsharpRepl). But I think you might be missing the point of doctest. It is not a substitute for unit-testing the main *code*. Rather, it's way of (unit-)testing the *documentation*. I.E. it verifies that the sample code in the documentation does what it says is does. This sample code is supposed to be in a form that helps clarify to a reader, not in the form of tests.
Matthew Flaschen
2010-10-19 05:35:54
`doctest` also has a secondary use for general testing of the main code. But as the documentation warns, "filling your docstrings with obscure test cases makes for bad documentation", so it recommends distinguishing between these two purposes.
Matthew Flaschen
2010-10-19 05:44:59
@Matthew: Just because there are implementations of interactive shells doesn't mean that C# itself has a defined interactive shell behaviour. I would be loathe to see implementation-specific syntax in comments. As for whether I misunderstood the question: the OP suggested that this would be a good way of introducing someone to unit tests. I think a better way of introducing unit testing is to write some *actual unit tests*.
Jon Skeet
2010-10-19 05:58:55
@Jon, I agree that the portability concern is valid. However, there are already multiple implementation-specific documentation syntaxes (NDoc, Sandcastle, etc.), and of course people use libraries that aren't part of the standard. Looking at the question again, you may well be right that the OP wanted a C# `doctest` for general unit-testing (as opposed to unit-testing the docs). He could explain better what his intended use case is.
Matthew Flaschen
2010-10-19 06:09:36
@Matthew: NDoc and Sandcastle both build documentation from standard XML document comments. The output is stylised, but the input is the same - you can use the same source for both. There may be specific extensions, but you don't have to use them. Without *any* standard REPL format, I don't think doctest is in a similar position. I do see the benefit of being able to test documentation examples, but I suspect the added complexity *for C#* wouldn't be worth it (whereas Python gives a more natural fit).
Jon Skeet
2010-10-19 06:16:29
I'm not really hearing why this would be a bad thing to have. I don't see how not having an interactive shell means this would be useless. No one is saying that doctests are a replacement for unittests, as they aren't in python. For purely functional methods that does some sort of calculation, this would be valuable.
zumalifeguard
2010-10-19 07:07:18
I said that this would be a good way to introduce someone to unit testing because that's how I heard Python doctests be positioned. I realize that doctests are not replacement for unittests, and the purpose of doctests is to validate the documentation. However, for those people who simply don't write unit tests, seeing that they can have some level of "testing" by simply adding sample inputs in the documentation will get them thinking along those lines.
zumalifeguard
2010-10-19 07:09:49
@zumalifeguard: Without a standard format for an interactive shell, there's no standard format for doctest to look for. If people want to learn unit testing, it's very easy to show them a class containing unit tests - *that's* how to get people thinking about testing, rather than showing them a different technique which isn't really designed for unit testing your production code, and which is more awkward for that purpose than testing in the more conventional way.
Jon Skeet
2010-10-19 08:20:23
Jon, I've read other places that starting with doctests in Python is a good way to introduce Python to the concept of unit testing. It doesn't matter that the mechanics are different. What's important to learn is that programmers should go beyond just testing their code manually, and actually contribute to the verification process.
zumalifeguard
2010-10-20 00:57:32
I also have another agenda -- I think that programmers should think more about writing data-driven unit tests. If they are going to write 10 unit tests which only differ in the parameters that being passed in, that's a lot of code duplication. I believe that's going to be the next level of sophistication with unit testing: to write unit tests such that they ultimately are functional in nature -- inputs, algorithm, outputs, and then drive those inputs using a data file. I don't think we're there yet, but we will be.
zumalifeguard
2010-10-20 01:00:07
@zumalifeguard: There doesn't *have* to be a lot of duplication for data-driven tests. There are unit test frameworks which allow data to be specified for multiple runs via attributes, or several unit tests can call one helper method. Or run the same helper method several times from one test, loading the data from a file. None of this requires doctests. I would argue that just because you've *read* that doctests are a good way to introduce unit testing doesn't mean it's true, and *also* doesn't mean it would suit C#. I still don't see any reason why tests in docs are better than out of docs.
Jon Skeet
2010-10-20 07:36:25
There are some problems with the way we write unit tests today. Because we don't want the test code to be in the same assembly as the production code, we have to put the test code in an entirely different source file. I could make the argument that there's an advantage of to placing some testing scenarios close to the code that's actually being tested. Today, there's no way to do that.
zumalifeguard
2010-10-20 19:17:07
In envision a tool which will allow you to place tests closer to the code that's being tested, but special annotions signal the compiler not to place the test code in the production assembly. There's pros and cons to this method. I don't think it's one-size-fits-all. In today's world, test code is in separate modules because we don't want it to be "built" in the same module as the production code. Some have gotten used to this, and some may rightfully argue that they prefer that separation.
zumalifeguard
2010-10-20 19:19:25
But I'll make the argument that as much as possible, everything associated with a production function, including how it is verified, should be in located close to the function itself. This is similar to the Code Contracts feature in .NET 4. It's "Design by contract". The contract to the inputs and outputs of the function are within that function. Some of those contracts are executed at runtime. Some are at compile time. The contracts executed at compile time do not get compiled into the released assembly, but the contracts are *written* inline with the production code.
zumalifeguard
2010-10-20 19:21:39
A complex CodeContract is only a few steps away to a doc-test, which is only a few steps away from a unit test.. Ultimately, these are all a way to verify that the code does what it's supposed to do, catching deviations from the design.
zumalifeguard
2010-10-20 19:22:39
@zumalifeguard: I wouldn't want everything to be very close together, because sometimes the test code can be overwhelming compared with the production code. However, given the quantity of comments you've written, I suggest that a blog post would be a more appropriate place for you to express your opinion. You asked a question, and I gave my honest opinion. You clearly disagree, and that's fine - but this sort of ongoing discussion isn't really what StackOverflow is good for.
Jon Skeet
2010-10-20 19:28:43