Looking for a better JavaScript unit test tool

Question

Let's consider the state of JavaScript unit tests and testing tools.

JsUnit

We already use it for some of our js code.

Pros

• can be invoked from an ant build file
• launches browser to run the tests

• Eclipse plug-in

  Cons

• launches browser to run the tests

• Does not support js file to write the unit test code: it has to be embedded inside an html file
  • it has not be updated for a few years

Note: There is a JsUnit (2).

RhinoUnit

Pros

• ant driven
• supports js file

• very simple to use

  Cons

• Simulation of JavaScript engine: not advanced enough to support our code: I tried to run test code working with JsUnit: I encountered issue when loading our common JavaScript files

crosscheck

Pros

• Can invoked from ant build file

• Simulates real browser behaviour

  Cons

• Simulation of JavaScript engine from a limited number of browser versions

• No activity for 2 years: it does not support Firefox versions 2.x nor 3.x

jsspec

Pros

• Runs on actual browser

  Cons

• JavaScript only framework: cannot be called from ant build file

jspec

Pros

• Runs on actual browser

  Cons

• Does not seem to support our code: I tried to run test code running with js unit: I encountered issue when loading our common JavaScript files

• JavaScript only framework: cannot be called from ant build file,

Screw.unit

Pros

• Runs on actual browser

  Cons

• JavaScript only framework: cannot be called from ant build file

Note: I did not try it but it is very similar to jsspec and jspec.

It looks like JsUnit is the only choice we have. Please note it is already used in other components. It is not perfect though because it does not provide an easy way to apply the TDD process for the following reasons: - It does not provide a simple and integrated way to run JavaScript unit test - It forces you to write the unit tests in a html file instead of a .js file, - It forces you to have a local installation of the JsUnit framework in order to avoid absolute hard coded path to reference js unit files.

As a consequence, you have to switch back and forth from you IDE and all the browsers we want to support while "TDDing" in JavaScript. It is feasible but I do not think it is very effective (I may be wrong here).

Also I still need to look at a proper JavaScript editor or Eclipse plug-in allowing easy referencing and refactoring. There is the JSTD one which is part of WTP, however I have not played with it enough in order to have a good idea of it.

As a conclusion, I do not think we have the tools to be fully "TDD" compliant. We can be TOD (Test Oriented Development), but I have not found any tool that allows us to work with JavaScript as efficiently as with Java. Once again, the ideal solution would be to have something identical to JUnit.

What JavaScript unit testing tools do you use?

Recent Entries:

• Mark Levison's Blog:
• Agile/Scrum Smells:
• Agile Games for Making Retrospectives Interesting:

Answer 1

You have "runs on actual browser" as a pro, but in my experience that is a con because it is slow. But what makes it valuable is the lack of sufficient JS emulation from the non-browser alternatives. It could be that if your JS is complex enough that only an in browser test will suffice, but there are a couple more options to consider:

HtmlUnit: "It has fairly good JavaScript support (which is constantly improving) and is able to work even with quite complex AJAX libraries, simulating either Firefox or Internet Explorer depending on the configuration you want to use." If its emulation is good enough for your use then it will be much faster than driving a browser.

But maybe HtmlUnit has good enough JS support but you don't like Java? Then maybe:

Celerity: Watir API running on JRuby backed by HtmlUnit.

or similarly

Schnell: another JRuby wrapper of HtmlUnit.

Of course if HtmlUnit isn't good enough and you have to drive a browser then you might consider Watir to drive your JS.

Answer 2

Another name to throw into the ring would be Selenium, which (using Selenium RC) can be driven (for example) from Java, which allows for nice integration with JUnit/Eclipse/Ant. The actual JavaScript code to be tested runs in a real web browser (yes, that is slow), all of the major ones are being supported.

Answer 3

MochiKit has a testing framework called SimpleTest that seems to have caught on. Here's a blog post from the original author.

Answer 4

You might also be interested in the unit testing framework that is part of qooxdoo, an open source RIA framework similar to Dojo, ExtJS, etc. but with quite a comprehensive tool chain.

Try the online version of the testrunner. Hint: hit the gray arrow at the top left (should be made more obvious). It's a "play" button that runs the selected tests.

To find out more about the JS classes that let you define your unit tests, see the online API viewer.

For automated UI testing (based on Selenium RC), check out the Simulator project.

Answer 5

Do you recall the issues with JSpec that you had?

Answer 6

You should have a look at env.js. See my blog for an example how to write unit tests with env.js.

Answer 7

A different approach, but crosses boundaries well is WATIN, WATIR, and WATIJ. Instead of testing javascript, they test the page its on, including specific interactions. Harnessed in the unit test framework (junit, nunit) the results can be run along with your other unit tests including from your build process.

http://watin.sourceforge.net/

Answer 8

YUI has a testing framework as well. This video from Yahoo! Theater is a nice introduction, although there are a lot of basics about TDD up front.

This framework is generic and can be run against any JavaScript or JS library.

Answer 9

Check out the recent screencast covering recent changes to JSpec, you will like what you see ;) very sexy, powerful, extendable test framework.

http://vision-media.ca/resources/javascript/jspec-032-javascript-testing-framework

Answer 10

TestSwarm

John Resig (jQuery) have created a tool for distributed JavaScript testing, TestSwarm. Mainly for open source JavaScript projects, but TestSwarm is open source, so you can set up a server yourself for corporate testing. Although this might require that you to do some modifications.

pros:

• Supports all major browsers/operating systems
• Run on multiple clients at once
• Don't need to run server/clients on development computer (no need for IE)
• Automatic run tests on all clients when you commit something (or whenever you modify the script to run the tests)
• Show history of test results pr commit
• Supports multiple JavaScript test frameworks
• Have test results for OS and browser versions

cons:

• Can not break your build through ant/maven
• Don't notice the test fail before commit
• No IDEplug-in

http://ejohn.org/blog/javascript-testing-does-not-scale/

TestSwarm architecture:

JsTestDriver

Some people at Google have also started on a distributed JavaScript tool, JsTestDriver. It is similar to TestSwarm, that it has a server, and clients connected. But it also has support for running tests from command line and plugins for Eclipse and IntelliJ!

pros:

• Supports all major browsers/operating systems
• Run on multiple clients at once
• Don't need to run server/clients on development computer (no need for IE)
• Run tests from command line (jar) (can be integrated in ant/maven)
• Eclipse plugin
• IntelliJ plugin
• Supports multiple JavaScript test frameworks

cons:

• Doesn't show results for os or browser version. Only browser names. It does however print out the version in the test results.
• No history of previous test results

Overview of how JsTestDriver works at runtime:

Eclipse plugin screenshot:

Short intro video: http://www.youtube.com/watch?v=V4wYrR6t5gE

YUI Yeti

Yahoo now has their own JavaScript test run server, Yeti. Built with node.js. It should be able to run your existing YUI-tests as they are, and works in multiple browsers. Since it runs from the command-line I guess it's mostly similar to JsTestDriver.

Announcement from August 25th 2010: http://www.yuiblog.com/blog/2010/08/25/introducing-yeti-the-yui-easy-testing-interface/

Project home: http://yuilibrary.com/projects/yeti/

Sinon

Another great tool is sinon.js by Christian Johansen, the author of Test-Driven JavaScript Development. Best described by himself:

Standalone test spies, stubs and mocks for JavaScript. No dependencies, works with any unit testing framework.

http://tddjs.com/

Answer 11

The JavaScript section of the Wikipedia entry, List of Unit Testing Frameworks, provides a list of available choices. It indicates whether they work client-side, server-side, or both.

Answer 12

I have tried JsTestDriver and it looks pretty good but unfortunately does not work with prototype framework. It is already reported issue http://code.google.com/p/js-test-driver/issues/detail?id=32&q=prototype

Answer 13

It's been mentioned (@Aaron Digulla) but you should take a look at envjs (http://www.envjs.com) It's basically a DOM written in Javascript that can be run in Rhino (and i believe SpiderMonkey) so you get to run all your tests on your CI server... and it's open source.

http://ejohn.org/blog/bringing-the-browser-to-the-server/

Answer 14

I've tried YUITest and Qunit. I find Qunit much simpler and easy to use.