A programming language designed to be testable

Question

Does anyone know of a programming language that is testable by design or at least exhibit very good properties in terms of testability?

For instance, a programming language designed so that unit testing is a non optional part of the coding process, or, better yet, a programming language where the program is more or less inferred from the unit tests.

Or if you prefer twisting the question a bit, what imperative programming is bad practice or unnecessary to ensure testability?

Edit:

What about object oriented programming? things like dependency injection and mocking libraries really helped TDD, wouldn't these practices benefit from being part the language design? I've been looking around and languages with rich meta models tend to permit very DSL like APIs to be written in the target language. These libraries are being built on top of that, why not pull these things into the language itself?

Edit 2:

If find that that blog post referenced by James Black has a really good point to it.

Not long ago, I reflected on my testing habits for the past ten years, and I made a couple of interesting observations:

• I feel the need to write tests for the code that I write very often.
• Just as often, that need is thwarted by environmental constraints, so I end up not writing these tests.

Edit 3:

Do you agree with the statement that dynamic programming languages make writing tests easier?

Answer 1

A purely functional language such as Haskell might be a good candidate, as in such a language functions do not have side effects and will always product the same results given the same input.

Answer 2

Not that I'm aware of, but the closest I've come to this is probably Eiffel. It also plays well with .NET

Answer 3

I've never seen anything where testing is non-optional, built-in or inferred, but the most 'testy' language I've seen is Ruby - it comes with unit testing tools out of the box, it's easy to test and mock because the types are dynamic, and the community is generally well up on testing practices (with many TDD & BDD tools available). Although a strange lack of code coverage tools I notice.

Answer 4

Maude is probably an extreme example: every program is a theory in algebraic rewrite logic, and can be directly subject to constructive proofs.

Answer 5

A language that features design by contract such as Eiffel (or iContract in Java) might be a good idea. That way you can test the pre and post conditions as well as class invariants fairly easily.

Besides that, TDD is largely language-neutral.

Answer 6

The Spring Framework centres around dependency injection and creation of easily testable Plain Old Java Objects (POJOs)

Answer 7

Google is working on noop as a language (OOP, Java-based) created to produce code that is always testable.

Noop is a new language that will run on the Java Virtual Machine, and in source form will look similar to Java. The goal is to build dependency injection and testability into the language from the beginning, rather than rely on third-party libraries as other languages do.

The language also explicitly forbids certain constructs that make it harder to test code, like for instance statics.

Answer 8

Fortress seems to be designed to have more unit testing features built in, but, according to this blog D may also be designed so that unit tests can be part of your class.

http://beust.com/weblog2/archives/000522.html

UPDATE:

The blog referenced above deals with the question for this post, as the title is:

Should programming languages support unit testing natively?

So there are comments about other languages regarding testing.

Answer 9

Dynamic-typed languages (Python, Ruby, etc) have seen have some of the largest and best test suites I have ever seen. I have more experience with Python, so I shall concentrate on it, but almost every attribute of Python and Ruby are equivalent with respect to its testability, and the propensity of open source projects using it to include test suites.

There are many concrete examples of real world projects that are large, and are used by a lot of people, and have large large suites of tests, are practical examples of good results in testing software, and the ones I am most familiar with are written in Python. However, perhaps the SQLite people (C/C++) deserve some credit for building one of the most thorough test-suites in the whole open source world, too.

Python test suites are (like all python application code) generally shorter, easier to write, and easier to read than any other language I have used. And yet, while I do not think C/C++ in particular is a very easy language to write tests in, and that it was more likely the passion and technical skill of the developers on the SQLite project, and the need to be able to trust their work, that lead to writing a large, thorough, effective test-suite in C++, and nothing to do with the C++ language itself.

So, Python helps you out by making writing your apps easier, giving you time to write tests, that's all. But where C++ lacks in features to help you write tests, and takes you longer to build the same thing in C++, still some developers supply the skill and dedication and make it happen anyways, at whatever cost of time and effort.

While certain features in a language make formal assurances of code-coverage in popular compiled and dynamic languages, very limited in scope, I believe the right path is to fix your formalisms, not change your language. The problem is not that the real world fails to meet my formal requirements for testing, but that the limits of the achievable in the real world need to be addressed by changing the way we build our formal definition of how tested the code is (code coverage, for example). In other words, the leaky abstractions in our testing formalisms are the thing failing us, not the real existing technology (C/C++).

Testing is definitely an OPTIONAL part of the process, like everything in Python, very little is actually forced upon you. However, I would argue, that any language that found a way to FORCE you to write tests rather than ENCOURAGING such good behavior is in fact, an example of trying to legislate coding practices, that is bound to backfire.

I do not frankly understand how a language can be designed in a way that forces you to test, without turning the language into a completely useless academic trivet. Like many others, I find the usability of a language to do real work at a real job much more important than ivory tower notions of formalism, and provable correctness.

On the other hand, I find that the best way to free up a developer to write a lot of tests, and be able to test their code effectively, is to remove the accidental complexity from development. I believe that Python, and Delphi, which are the two languages I am most proficient in, do more to ease this burden than other languages widely used in commercial and open-source development do. Because of the extra time I didn't have to spend finding out how many pages of mumble-mumble it would take to write my main application code, I now find I have a half-day to test for correctness, scalability, and and even do some cross-platform testing.

Ask not what you can do for your formalisms, but what your formalisms can do to come back down to earth.

Answer 10

As I played around with the concept of Zero Button Testing, a forced-testing language / IDE kind of emerged naturally. It's just a toy language, but the idea is there within it: any line that isn't covered by tests shows up in yellow, and any functions with any untested lines could be considered un-runnable (although the project never went that far).

The idea here is that the tests are built directly into the procedure they're testing; it works at this kind of scale, but I am in doubt as to the ultimate scalability of the approach.

Answer 11

I think Ruby is where it's at.

Tools such as cucumber - Where you write user stories in almost natural language that business people can understand and then actually get them passing is awesome as a 'TDD tool'.

It's not so much about writing 'tests' but rather starting with what the customer wants and having tools which use that as your specification to build to.

As well as rspec, shoula, machinist, faker and the rest make Ruby a real consideration if this is where you interest lies.

I would also say the ability to create your own DSL puts the dynamic languages in front in this realm.

Answer 12

What programming languages are testable by design?

I'm having a hard time with this question because I'm not sure what it means for a language to be testable by design. Testing is usually about tools and static analysis, not about language design.

But to the extent that a language can be testable, I'm going to assume that this means the language includes some sort of independent specification that can be tested against the code, and moreover that these specifications have been designed with testing in mind, or at least not only for theorem proving.

By these criteria, I know of three:

• Eiffel, where the "design by contract" encourages you to put preconditions and postconditions in your code, and they are tested. I consider Eiffel a successfully deployed language.

• Racket (formerly PLT Scheme), where some interesting research is going on in contracts and testing; the interesting bits are that when a test fails, the language tells you not only that a contract was violated, but it identifies what part of the code should be blamed. For more information google for Robby Findler and for "blame". Racket is a research language but is an extension of Scheme, which is pretty widely used for a niche language.

• Anna, an extension of Ada done by David Luckham and his students, which supported some testing. As far as I know, Anna never made it beyond the research stage.

In addition to these languages there are myriad languages that have had assertions for purposes of theorem proving. Many of these languages had assertions that were not testable in any efficient way (universal and existential quantifiers will do that to you).