views:

1131

answers:

5

I have a webapp build plan running on a Continuous Integration system (Atlassian Bamboo 2.5). I need to incorporate QUnit-based JavaScript unit tests into the build plan so that on each build, the Javascript tests would be run and Bamboo would interpret the test results.

Preferably I would like to be able to make the build process "standalone" so that no connections to external servers would be required. Good ideas on how to accomplish this? The CI system running the build process is on an Ubuntu Linux server.

+9  A: 

As I managed to come up with a solution myself, I thought it would be a good idea to share it. The approach might not be flawless, but it's the first one that seemed to work. Feel free to post improvements and suggestions.

What I did in a nutshell:

  • Launch an instance of Xvfb, a virtual framebuffer
  • Using JsTestDriver:
    • launch an instance of Firefox into the virtual framebuffer (headlessly)
    • capture the Firefox instance and run the test suite
    • generate JUnit-compliant test results .XML
  • Use Bamboo to inspect the results file to pass or fail the build

I will next go through the more detailed phases. This is what my my directory structure ended up looking like:

lib/
    JsTestDriver.jar
test/
    qunit/
            equiv.js
            QUnitAdapter.js
    jsTestDriver.conf
    run_js_tests.sh
    tests.js
test-reports/
build.xml

On the build server:

  • Install Xvfb (apt-get install Xvfb)
  • Install Firefox (apt-get install firefox)

Into your application to be built:

server: http://localhost:4224

load:
# Load QUnit adapters (may be omitted if QUnit is not used)
  - qunit/equiv.js
  - qunit/QUnitAdapter.js   

# Tests themselves (you'll want to add more files)
  - tests.js

Create a script file for running the unit tests and generating test results (example in Bash, run_js_tests.sh):

#!/bin/bash
# directory to write output XML (if this doesn't exist, the results will not be generated!)
OUTPUT_DIR="../test-reports"
mkdir $OUTPUT_DIR

XVFB=`which Xvfb`
if [ "$?" -eq 1 ];
then
    echo "Xvfb not found."
    exit 1
fi

FIREFOX=`which firefox`
if [ "$?" -eq 1 ];
then
    echo "Firefox not found."
    exit 1
fi

$XVFB :99 -ac &    # launch virtual framebuffer into the background
PID_XVFB="$!"      # take the process ID
export DISPLAY=:99 # set display to use that of the xvfb

# run the tests
java -jar ../lib/JsTestDriver.jar --config jsTestDriver.conf --port 4224 --browser $FIREFOX --tests all --testOutput $OUTPUT_DIR

kill $PID_XVFB     # shut down xvfb (firefox will shut down cleanly by JsTestDriver)
echo "Done."

Create an Ant target that calls the script:

<target name="test">        
    <exec executable="cmd" osfamily="windows">
        <!-- This might contain something different in a Windows environment -->
    </exec>

    <exec executable="/bin/bash" dir="test" osfamily="unix">
        <arg value="run_js_tests.sh" />
    </exec>
</target>   

Finally, tell the Bamboo build plan to both invoke the test target and look for JUnit test results. Here the default "**/test-reports/*.xml" will do fine.

miek
A: 

You may be able to use rhino, the headless browser, to run your unit tests on your CI machine. Of course, the disadvantage here is that it won't find bugs specific to browser X... but it does beat installing 2-3 OSes on your CI box, to cover all the main platforms...

But yes, this kind of sucks... but it might work just well enough in a CI scenario.

RyanWilcox
I was actually wondering if Rhino or HtmlUnit could be used together with JsTestDriver. Couldn't figure it out yet, however...
miek
+2  A: 

As an alternative, you could also try TestSwarm. I've got it up and running using QUnit to run my JS tests.

http://github.com/jeresig/testswarm http://testswarm.com/

Dom
A: 

I'm trying to integrate qunit, js-test-driver and htmlunit all into a maven plugin, and for some unknown reason HtmlUnit produces a deadlock in js-test-driver. Yep, I'm breaking my head trying to understand WHY, but I hope to find a solution.

You can take a look at the plugin. Until I could make js-test-driver to work together with HtmlUnit, the plugin uses env-js for creating the browser environmet.

Plugin page: http://code.google.com/p/qunit-maven-plugin/

Regards,

Maciek.

maciek
A: 

For anyone interested in running their Jasmine BDD specs headlessly in maven, you might be interested in the jasmine-maven-plugin I maintain:

http://github.com/searls/jasmine-maven-plugin

Justin Searls