GNU Makefile: multiple outputs from single rule + preventing intermediate files from being deleted

Question

This is sort of a continuation of question from here. The problem is that there is a rule generating multiple outputs from a single input, and the command is time-consuming so we would prefer to avoid recomputation. Now there is an additional twist, that we want to keep files from being deleted as intermediate files, and rules involve wildcards to allow for parameters.

---

The solution suggested was that we set up the following rule:

file-a.out: program file.in
    ./program file.in file-a.out file-b.out file-c.out

file-b.out: file-a.out
    @

file-c.out: file-b.out
    @

Then, calling make file-c.out creates both and we avoid issues with running make in parallel with -j switch. All fine so far.

---

The problem is the following. Because the above solution sets up a chain in the DAG, make considers it differently; the files file-a.out and file-b.out are treated as intermediate files, and they by default get deleted as unnecessary as soon as file-c.out is ready.

A way of avoiding that was mentioned somewhere here, and consists of adding file-a.out and file-b.out as dependencies of a target .SECONDARY, which keeps them from being deleted. Unfortunately, this does not solve my case because my rules use wildcard patters; specifically, my rules look more like this:

file-a-%.out: program file.in
    ./program $* file.in file-a-$*.out file-b-$*.out file-c-$*.out

file-b-%.out: file-a-%.out
    @

file-c-%.out: file-b-%.out
    @

so that one can pass a parameter that gets included in the file name, for example by running

make file-c-12.out

The solution that make documentation suggests is to add these as implicit rules to the list of dependencies of .PRECIOUS, thus keeping these files from being deleted.

---

The solution with .PRECIOUS works, but it also prevents these files from being deleted when a rule fails and files are incomplete. Is there any other way to make this work?

A hack to solve this is to define a target .SECONDARY with no prerequisites, i.e.,

.SECONDARY:

which informs make that all files should be treated as secondary and thus not removed, unless when make gets interrupted or a rule fails. Unfortunately, this does not allow for selecting a subset of rules with wildcards to work this way, so I consider this only a hack (even though it's useful).

Answer 1

If, instead of a single file.in, your outputs were generated from a prerequisite file that included the stem, i.e.

file-a.out: program file-%.in
    ./program file-$*.in file-a-$*.out file-b-$*.out file-c-$*.out

you can then build a list of all possible target matches:

inputs = $(wildcard file-*.in)
secondaries = $(patsubst file-%.in,file-a-%.out,$(inputs)) \
    $(patsubst file-%.in,file-b-%.out,$(inputs))

Similarly if the stem comes from a finite set:

batchnos = 17 18 19 20 
batchnos = $(shell seq 17 20)
secondaries = $(patsubst %,file-a-%.out,$(batchnos)) $(patsubst %,file-b-%.out,$(batchnos))

Then just add these as prereqs to the .SECONDARY target

.SECONDARY: $(secondaries)

Answer 2

The Simplest Thing

file-a-%.out file-b-%.out file-c-%.out: program file.in
    ./program $* file.in file-a-$*.out file-b-$*.out file-c-$*.out

will do exactly what you want.

(Pattern rules with multiple targets are different than the normal rule with multiple targets that you were asking about here. See the bison example in the make manual.)