Clean up my makefile fragment?

Question

I'm using GNU Make to run scientific data analysis, where we have a bunch of Matlab scripts that ponder some input data and then spit out several files. It does this in a fairly complex way, so I've had to do some nasty Makefile tricks when trying to describe its function to Make. Here's an example:

seg_cams_nozero := cam1 cam2 cam3
seg_per_camera := $(shell echo {,dyn_}{hands,obj{1,2,3,4,5}}.mat)
# the complete list of things we want
segmentation_outputs := $(foreach cam,$(seg_cams_nozero),$(foreach product,$(seg_per_camera),derived/cont_$(cam)_$(product)))

# how to make some product, independent of what camera
define seg_per_product
derived/cont_cam%_$$(product): /path/to/input/file_%.mat
        run_a_script $$*
endef

$(foreach product,$(seg_per_camera),$(eval $(seg_per_product)))

segmentation: $(segmentation_outputs)

So, this is basically horrible because I haven't figured out how to use pattern rules effectively in this situation, and because I have to generate a ton of filenames using the shell.

How would you write something like this? Would you pre-generate the filenames and put them in an included Makefile? Figure out a great way to use pattern rules? Is there any way to do it without $(eval ...)?

Answer 1

There are several ways to do this, none of them perfect (until someone puts regex handling into Make).

First, I notice that one command makes all of the targets for a given camera, but it's run for each target, which is a waste. So let's start with the targets for cam1

cam1_outputs := $(addprefix derived/cont_cam1_, $(seg_per_camera))

and make the first one the "primary one" (and remove it from the list). This will be a prerequisite for the rest, and will be the only one that actually requires that the script be run. (There may be slightly more elegant ways using more advanced methods, but this'll do for now.)

cam1_primary := $(firstword $(cam1_outputs))
cam1_outputs := $(filter-out $(cam1_primary), $(cam1_outputs))

$(cam1_outputs): $(cam1_primary)

$(cam1_primary): /path/to/input/file_1.mat 
    run_a_script 1

Now to extend this to the other two cameras. We can rewrite the "primary" rule as a pattern rule:

$(cam1_primary) $(cam2_primary) $(cam3_primary): derived/cont_cam%_hands.mat: /path/to/input/file_%.mat
    run_a_script $*

The rest we could just spell out for all three cameras, but that would mean a lot of redundant code. We could define a template and eval it, but I like to avoid that if possible. So we'll just use a little trick:

cam2_primary := $(subst cam1,cam2,$(cam1_primary))
# ...and the same for the rest...

(That's a little redundant, but not too terrible.)

Put it all together and we get:

# Mention this first so it'll be the default target
segmentation:

seg_cams_nozero := cam1 cam2 cam3 

# seg_per_camera := $(shell echo {,dyn_}{hands,obj{1,2,3,4,5}}.mat) 
# Let's do this without the shell:
seg_per_camera := hands $(addprefix obj, 1 2 3 4 5)
seg_per_camera += $(addprefix dyn_, $(seg_per_camera))
seg_per_camera := $(addsuffix .mat, $(seg_per_camera))

cam1_outputs := $(addprefix derived/cont_cam1_, $(seg_per_camera))

# Now's a good time for this.
segmentation_outputs := $(cam1_outputs)
segmentation_outputs += $(subst cam1,cam2,$(cam1_outputs))
segmentation_outputs += $(subst cam1,cam3,$(cam1_outputs))

cam1_primary := $(firstword $(cam1_outputs))
cam1_outputs := $(filter-out $(cam1_primary), $(cam1_outputs)) 
$(cam1_outputs): $(cam1_primary)

cam2_primary := $(subst cam1,cam2,$(cam1_primary))
cam2_outputs := $(subst cam1,cam2,$(cam1_outputs))
$(cam2_outputs): $(cam2_primary)

cam3_primary := $(subst cam1,cam3,$(cam1_primary))
cam3_outputs := $(subst cam1,cam3,$(cam1_outputs))
$(cam3_outputs): $(cam3_primary)

$(cam1_primary) $(cam2_primary) $(cam3_primary): derived/cont_cam%_hands.mat: /path/to/input/file_%.mat
    run_a_script $*

segmentation: $(segmentation_outputs)