makefile with directory tree creation suitable for parallel (-j ) build

Question

My project needs temporary directories which are created during the build using mkdir -p similarly to this:

all: dirtree $(OBJFILES)

dirtree: 
  @mkdir -p $(BUILD)/temp_directory

But this approach cannot be used with the -j switch, because first of the OBJFILES get compiled before the mkdir target is made.

Is there a standard way to do this?

Answer 1

You could have the rules for building the object files call mkdir -p as their first action.

Answer 2

I'm not sure I fully understand your question. However, I can say this: if your build breaks when you add parallelism, then it's an indication that you haven't defined the dependencies correctly. Ask yourself, "Do the directories need to exist before the object files are generated?" If the answer is "yes", then the directories should be listed as prerequisites of the object files. In other words:

${OBJFILES}: dirtree

And yes, that is pretty much the standard way to do this :)

Answer 3

The standard way is to create directories by the very first command of the rule that generates files in the directory. For example, if you have target $(BUILD)/file.o, then just put the proper mkdir command in the commands section:

$(BUILD)/temp_directory/%.c: ...
        @mkdir -p $(@D)
        generate-c-file > $@

The object files that depend on .c files don't need that directory creation, because if the .c file is generated, the directory is there as well.

Directories are seldom a separate entity that should be tracked via special dependency, so not having any of them as a target is reasonable.

And it is -j-compatible as well: the OS will take care of two processing creating the same directory--via internal filesystem locks.