views:

3676

answers:

9

I'm trying to incorporate some JavaScript unit testing into my automated build process. Currently JSUnit works well with JUnit, but it seems to be abandonware and lacks good support for AJAX, debugging, and timeouts.

Has anyone had any luck automating (with ANT) a unit testing library such as YUI test, JQuery's QUnit, or jQUnit (http://code.google.com/p/jqunit/)?

Note: I use a custom built AJAX library, so the problem with Dojo's DOH is that it requires you to use their own AJAX function calls and event handlers to work with any AJAX unit testing.

+2  A: 

Look into YUITest

Hank Gay
+7  A: 

There are many javascript unit test framework out there (jsUnit, scriptaculous, ...) but jsUnit is the only one I know that may be used with an automated build.

If you are doing 'true' unit test you should not need AJAX support. For example, if you are using an RPC ajax framework such as DWR, you can easily write a mock function :

   function mockFunction(someArg, callback) {
      var result = ...; // some treatments   
      setTimeout(
function() { callback(result); }, 300 // some fake latency
); }

And yes, JsUnit do handle timeouts : Simulating Time in jsUnit Tests

Alexandre Victoor
A: 

Another JS testing framework that can be run with Ant is CrossCheck. There's an example of running CrossCheck via Ant in the build file for the project.

CrossCheck attempts, with limited success, to emulate a browser, including mock-style implementations of XMLHttpRequest and timeout/interval.

It does not currently handle loading javascript from a web page, though. You have to specify the javascript files that you want to load and test. If you keep all of your JS separated from your HTML, it might work for you.

Jason Wadsworth
+8  A: 

I'm just about to start doing Javascript TDD on a new project I am working on. My current plan is to use qunit to do the unit testing. While developing the tests can be run by simply refreshing the test page in a browser.

For continuous integration (and ensuring the tests run in all browsers), I will use Selenium to automatically load the test harness in each browser, and read the result. These tests will be run on every checkin to source control.

I am also going to use JSCoverage to get code coverage analysis of the tests. This will also be automated with Selenium.

I'm currently in the middle of setting this up. I'll update this answer with more exact details once I have the setup hammered out.


Testing tools:

Karl
yes, please share it. thanks
melaos
Did you ever get this set up? How did it go?
El Yobo
I did get this set up, but using slightly different technologies. I used JS Test Driver to run the unit tests in each browser (rather than Selenium), using the QUnit Adapter (http://code.google.com/p/js-test-driver/wiki/QUnitAdapter).Another at my current employer I am using the Jasmine test framework, and running the tests in Jasmine Node (http://github.com/mhevery/jasmine-node) which avoids the delay of using a browser.I use my own little ruby project (http://github.com/karl/loris) to run the tests on every change.
Karl
+2  A: 

I recently read an article by Bruno using JsUnit and creating a JsMock framework on top of that... very interesting. I'm thinking of using his work to start unit testing my Javascript code.

Mock Javascript or How to unit test Javascript outside the Browser environment

Elijah Manor
A: 

For .Net using NUnit or MbUnit you could use WatiN to run a JSUnit Runner and check the results... like this guy:

http://adamesterline.com/2007/05/15/integrating-jsunit-with-nunit-using-watin/

Paul Shannon
+5  A: 

Im a big fan of js-test-driver

It works well in a CI environment and is able to capture actual browsers for cross-browser testing.

cuberoot
I like it because of the CI integration but think its best plus point is that it works with YUITest and QUnit!
AutomatedTester
+1  A: 

I am in agreement that jsunit is kind of dying on the vine. We just finished up replacing it with YUI Test.

Similar to the example using qUnit, we are running the tests using Selenium. We are running this test independently from our other selenium tests simply because it does not have the dependencies that the normal UI regression tests have (e.g. deploying the app to a server).

To start out, we have a base javascript file that is included in all of our test html files. This handles setting up the YUI instance, the test runner, the YUI.Test.Suite object as well as the Test.Case. It has a methods that can be accessed via Selenium to run the test suite, check to see if the test runner is still running (results are not available until after it's done), and get the test results (we chose JSON format)

var yui_instance; //the YUI instance
var runner;  //The YAHOO.Test.Runner
var Assert; //an instance of YAHOO.Test.Assert to save coding
var testSuite; //The YAHOO.Test.Suite that will get run.

/**
 * Sets the required value for the name property on the given template, creates
 * and returns a new YUI Test.Case object.
 * 
 * @param template the template object containing all of the tests
 */
function setupTestCase(template) {
    template.name = "jsTestCase";
    var test_case = new yui_instance.Test.Case(template);
    return test_case;
}

/**
 * Sets up the test suite with a single test case using the given 
 * template.
 * 
 * @param template the template object containing all of the tests
 */
function setupTestSuite(template) {
    var test_case = setupTestCase(template);
    testSuite = new yui_instance.Test.Suite("Bond JS Test Suite");
    testSuite.add(test_case);
}

/**
 * Runs the YAHOO.Test.Suite
 */
function runTestSuite() {
    runner = yui_instance.Test.Runner;
    Assert = yui_instance.Assert;

    runner.clear();
    runner.add(testSuite);
    runner.run();
}

/**
 * Used to see if the YAHOO.Test.Runner is still running.  The
 * test results are not available until it is done running.
 */
function isRunning() {
    return runner.isRunning();
}

/**
 * Gets the results from the YAHOO.Test.Runner
 */
function getTestResults() {
    return runner.getResults(yui_instance.Test.Format.JSON);
}

As for the selenium side of things, we used a parameterized test. We run our tests in both IE and FireFox in the data method, parsing the test results into a list of Object arrays with each array containing the browser name, the test file name, the test name, the result (pass, fail or ignore) and the message.

The actual test just asserts the test result. If it is not equal to "pass" then it fails the test with the message returned from the YUI Test result.

    @Parameters
public static List<Object[]> data() throws Exception {
 yui_test_codebase = "file:///c://myapppath/yui/tests";

 List<Object[]> testResults = new ArrayList<Object[]>();

 pageNames = new ArrayList<String>();
 pageNames.add("yuiTest1.html");
 pageNames.add("yuiTest2.html");

 testResults.addAll(runJSTestsInBrowser(IE_NOPROXY));
 testResults.addAll(runJSTestsInBrowser(FIREFOX));
 return testResults;
}

/**
 * Creates a selenium instance for the given browser, and runs each
 * YUI Test page.
 * 
 * @param aBrowser
 * @return
 */
private static List<Object[]> runJSTestsInBrowser(Browser aBrowser) {
 String yui_test_codebase = "file:///c://myapppath/yui/tests/";
 String browser_bot = "this.browserbot.getCurrentWindow()"
 List<Object[]> testResults = new ArrayList<Object[]>();
 selenium = new DefaultSelenium(APPLICATION_SERVER, REMOTE_CONTROL_PORT, aBrowser.getCommand(), yui_test_codebase);
 try {
  selenium.start();

  /*
   * Run the test here
   */
  for (String page_name : pageNames) {
   selenium.open(yui_test_codebase + page_name);
   //Wait for the YAHOO instance to be available
   selenium.waitForCondition(browser_bot + ".yui_instance != undefined", "10000");
   selenium.getEval("dom=runYUITestSuite(" + browser_bot + ")");

   //Output from the tests is not available until 
   //the YAHOO.Test.Runner is done running the suite
   selenium.waitForCondition("!" + browser_bot + ".isRunning()", "10000");
   String output = selenium.getEval("dom=getYUITestResults(" + browser_bot + ")");

   JSONObject results = JSONObject.fromObject(output);
   JSONObject test_case = results.getJSONObject("jsTestCase");
   JSONArray testCasePropertyNames = test_case.names();
   Iterator itr = testCasePropertyNames.iterator();

   /*
    * From the output, build an array with the following:
    *  Test file
    *  Test name
    *  status (result)
    *  message
    */
   while(itr.hasNext()) {
    String name = (String)itr.next();
    if(name.startsWith("test")) {
     JSONObject testResult = test_case.getJSONObject(name);
     String test_name = testResult.getString("name");
     String test_result = testResult.getString("result");
     String test_message = testResult.getString("message");
     Object[] testResultObject = {aBrowser.getCommand(), page_name, test_name, test_result, test_message};
     testResults.add(testResultObject);
    }
   }

  }
 } finally {
  //if an exception is thrown, this will guarantee that the selenium instance
  //is shut down properly
  selenium.stop();
  selenium = null;
 }
 return testResults;
}
/**
 * Inspects each test result and fails if the testResult was not "pass"
 */
@Test
public void inspectTestResults() {
 if(!this.testResult.equalsIgnoreCase("pass")) {
  fail(String.format(MESSAGE_FORMAT, this.browser, this.pageName, this.testName, this.message));
 }
}

I hope this is helpful.

Josh
A: 

I just got Hudson CI to run JasmineBDD (headless), at least for pure javascript unit testing.

(Hudson running Java via shell, running Envjs, running JasmineBDD.)

I haven't got it to play nice with a big library yet, though, like prototype.

Ingvald