How can I effectively test a scripting engine?

Question

I have been working on an ECMAScript implementation and I am currently working on polishing up the project. As a part of this, I have been writing tests like the following:

[TestMethod]
public void ArrayReduceTest()
{
    var engine = new Engine();
    var request = new ExecScriptRequest(@"
        var a = [1, 2, 3, 4, 5];
        a.reduce(function(p, c, i, o) {
            return p + c;
        });
    ");
    var response = (ExecScriptResponse)engine.PostWithReply(request);
    Assert.AreEqual((double)response.Data, 15D);
} 

The problem is that there are so many points of failure in this test and similar tests that it almost doesn't seem worth it. It almost seems like my effort would be better spent reducing coupling between modules. To write a true unit test I would have to assume something like this:

[TestMethod]
public void CommentTest()
{
    const string toParse = "/*First Line\r\nSecond Line*/";
    var analyzer = new LexicalAnalyzer(toParse);
    {
        Assert.IsInstanceOfType(analyzer.Next(), typeof(MultiLineComment));
        Assert.AreEqual(analyzer.Current.Value, "First Line\r\nSecond Line");
    }
}

Doing this would require me to write thousands of tests which once again does not seem worth it.

Answer 1

Just spitballing here, but what if you stored your tests in a file/database/etc... (as Doug points out - the ability to store in version control makes files the best choice!) Each entry could have a Name, a Script, and the Expected Output.

Then you could just write a small app that executes each script using your engine and compares the output to the expected output, and notifies you of success/failure based on the result.

This wouldn't save you from needing to test the pain points in the script engine itself, but it might be easier to maintain something like this than to write a unit test for each way that you can envision using the scripting runtime.