views:

36

answers:

2

Here's a skeleton Makefile just to make it easier to describe the problem:

all_tests : unit_tests other_tests_1 other_tests_2 ... other_tests_N

unit_tests : set1_summary.txt set2_summary.txt ... setN_summary.txt

%_summary.txt : %_details.txt
    perl createSummary.pl --in $^ -out $@

%_details.txt : test_harness
    ./test_harness --test-set $*

So I have a test runner that produces a file with detailed results, and then a filtering mechanism to create a summary file.

Now, the test runner application returns an error code if any of the items in the test set fails, which will correctly abort the "all_tests" target and never invoke the other_test targets. However, I would like to run the details -> summary transformation unconditionally, since that is relevant even for a failed test run.

I've tried some different variants but the only method I could get to work was wrapping the whole command chain into a Perl script, storing away the result of the first command and using that as return value for the whole script.

But that does not feel like a very neat solution, especially since the "actual" command set is a bit more complex than what this skeleton shows. Do you know any purely GNU Make-based method to accomplish this?

A: 

You could have a special rule that invokes Make recursively twice, like this:

.PHONY: test
test :
    make all_tests ; make summary

The only downside is that the exit status of the top make process will no longer indicate success/failure of tests, but you could even fix that if you wanted to by a little extra scripting using the $? shell variable.

Gintautas Miliauskas
I think `make all_tests; STATUS=$?; make summary; exit $STATUS` might do what you want.
Jack Kelly
+1  A: 

You could tell make that it's okay for the test_harness to return non-zero, by prepending a - to the command (see the manual). If you do that, make will not consider the "details" target to have failed, so will go on to do the "summary" step.

Presumably createSummary.pl can tell if tests have failed from looking at the generated "details" files. If you want to generate a make error when a test fails, you could have createSummary.pl return nonzero if it detects any failure.

slowdog