Fuzzy/approximate checking of solutions from algorithms

Question

We have people who run code for simulations, testing etc. on some supercomputers that we have. What would be nice is, if as part of a build process we can check that not only that the code compiles but that the ouput matches some pattern which will indicate we are getting meaningful results.

i.e. the researcher may know that the value of x must be within some bounds. If not, then a logical error has been made in the code (assuming it compiles and their is no compile time error).

Are there any pre-written packages for this kind of thing. The code is written in FORTRAN, C, C++ etc.

Any specific or general advice would be appreciated.

Answer 1

I expect most unit testing frameworks could do this; supply a toy test data set and see that the answer is sane in various different ways.

Answer 2

A good way to ensure that the resulting value of any computation (whether final or intermediate) meets certain constraints, is to use an object oriented programming language like C++, and define data-types that internally enforce the conditions that you are checking for. You can then use those data-types as the return value of any computation to ensure that said conditions are met for the value returned.

Let's look at a simple example. Assume that you have a member function inside of an Airplane class as a part of a flight control system that estimates the mass of the airplane instance as a function of the number passengers and the amount of fuel that plane has at that moment. One way to declare the Airplane class and an airplaneMass() member function is the following:

class Airplane {
public:

    ...

    int airplaneMass() const; // note the plain int return type

    ...

private:
    ...
};

However, a better way to implement the above, would be to define a type AirplaneMass that can be used as the function's return type instead of int. AirplaneMass can internally ensure (in it's constructor and any overloaded operators) that the value it encapsulates meets certain constraints. An example implementation of the AirplaneMass datatype could be the following:

class AirplaneMass {
public:

    // AirplaneMass constructor
    AirplaneMass(int m) {
        if (m < MIN || m > MAX) {
            // throw exception or log constraint violation
        }

        // if the value of m meets the constraints,
        // assign it to the internal value.
        mass_ = m;
    }

    ...

    /* range checking should also be done in the implementation
       of overloaded operators. For instance, you may want to
       make sure that the resultant of the ++ operation for
       any instance of AirplaneMass also lies within the
       specified constraints. */

private:
    int mass_;
};

Thereafter, you can redeclare class Airplane and its airplaneMass() member function as follows:

class Airplane {
public:

    ...

    AirplaneMass airplaneMass() const;
    // note the more specific AirplaneMass return type

    ...

private:
    ...
};

The above will ensure that the value returned by airplaneMass() is between MIN and MAX. Otherwise, an exception will be thrown, or the error condition will be logged.

Answer 3

I had to do that for conversions this month. I don't know if that might help you, but it appeared quite simple a solution to me.

First, I defined a tolerance level. (Java-ish example code...)

private static final double TOLERANCE = 0.000000000001D;

Then I defined a new "areEqual" method which checks if the difference between both values is lower than the tolerance level or not.

private static boolean areEqual(double a, double b) {
    return (abs(a - b) < TOLERANCE);
}

If I get a false somewhere, it means the check has probably failed. I can adjust the tolerance to see if it's just a precision problem or really a bad result. Works quite well in my situation.