How did you learn the GNU make tools?

Question

I've been a professional web developer for about five years now, and have compiled many many things for servers. I've also written many simple C programs (one or two files).

The main thing which has held me back from creating more complex applications is the fear of leraning to the GNU build tools. The documentation is huge and a bit intimidating.

Does anyone have a good resource to learn these?

Answer 1

there's a few free tech books that cover this

http://www.freetechbooks.com/gnu-linux-f7.html

Answer 2

The first step is simply to realize there's nothing to fear. While the documentation for make can be intimidating, it's all lots of cool features that you don't have to rely on. A functional Makefile can be just a couple of lines long, or in simple cases, even a single line. Once you've started writing simple ones, it becomes easier to slowly learn a feature at a time as they become useful.

A basic makefile is just a set of rules. Each rule has a name, a list of requirements that must be up to date as a prerequisite for the rule, and a set of commands to run for that rule. When you run make, you can give it a list of rules to build as command line arguments. If you don't give it any, it will build the "all" rule, if it exists. A simple Makefile for a project with one C source file could be:

all:
    gcc foo.c -o foo

(note that the indention is important, and it has to be a single tab character). The file can be named anything you want, but by default make looks for a file named Makefile. If you create a file by that name with the example contents, then run make in that directory, the gcc command in the file will be executed to build your program. For even a simple program with a single source file, this already saves some typing when you compile. If you have some extra compiler flags you always want to pass, now you only have to put them in the Makefile, not try to remember to add them every time you compile.

For a slightly more complex example, let's try a project with two source files:

all:
    gcc -c foo.c
    gcc -c bar.c
    gcc foo.o bar.o -o app

This will work just fine, but it has a few drawbacks. As we start adding more source files, adding a gcc line for each one, plus adding each output object to the final line starts to feel clumsy. Also, this recompiles every source file every time we build. In a simple example, this probably isn't a big deal, but as things get larger, recompiling everything every time gets slow, and probably unnecessary. If I've only edited foo.c since the last time I compiled, there's no need to recompile bar.c again; I just need to recompile foo.c then relink the object files. This is where rule prerequisites come in handy. Rather than compiling each source file in our rule, we can make a separate rule for each oject file, then tell the final rule that those object files are required:

foo.o:
    gcc -c foo.c

bar.o:
    gcc -c bar.c

all: foo.o bar.o
    gcc foo.o bar.o -o app

We're half way there, but make will still recompile everything every time. It has no way to know if foo.o and bar.o are up to date, so it will build them every time, before building the application in our all rule. So we need to tell make what files foo.o and bar.o depend on:

foo.o: foo.c
    gcc -c foo.c

bar.o: bar.c
    gcc -c bar.c

all: foo.o bar.o
    gcc foo.o bar.o -o app

Now make knows that foo.o depends on foo.c. Since there isn't a rule for how to build foo.c, it assumes that's one of your source files. Now, when you run make, and it sees that all requires foo.o, it will look at the timestamps on foo.o and foo.c. If foo.c is not newer than foo.o, it knows that the source file hasn't changed, so it won't bother rebuilding foo.o.

We're improving, but there are still a few things that could be better. Right now we'll still link all the object files every time we run make, even if none of the object files were recompiled. We also still have to type those redundant rules for every C file, and we have to list all our object files twice, once in the requirements for all, and once in the actual build line. Typing the redundant rules turns out to be an easy fix. It's so common to build a .o file from a .c file (or a number of other common source code extensions) that make already knows how to handle it. So we can actually leave out the foo.o and bar.o rules. If a rule requires foo.o, and you have a file named foo.c in that directory, make will figure out the rest. We can also use variables to keep from having to retype things. And making a rule for our executable, which all then requires, allows make to check if it needs to be rebuilt. That gives us:

OBJS=foo.o bar.o

app: $(OBJS)
    gcc $(OBJS) -o app

all: app

That will pretty much do what we want. The only problem is that we no longer have a way to pass the flags we want to the compiler. Also, make assumes we use the cc command to compile C files, but we want to use gcc. No problem. The built in rules for C files actually use two variables, CC and CFLAGS, for our compiler of choice and any options to pass to it. We simply set those variables at the beginning of the file, and we're good to go:

CC=gcc
CFLAGS=-Wall -g
OBJS=foo.o bar.o

app: $(OBJS)
    $(CC) $(CFLAGS) $(OBJS) -o app

all: app

And we have a nice Makefile. When we run make, it builds our project. It doesn't ever do any extra work. When we add new source files to our project, we just have to add the name of the object file to one line in our Makefile, and it does the rest.

Now, there's obviously a lot more to writing make files, and a lot more they can do, or the documentation wouldn't be so big. But this is all it takes to write makefiles for simple projects, and you can then learn piecemeal from there, as you find yourself wanting new things.

Answer 3

Mostly by editing broken Makefiles. :)

The other answers here are quite good, but you can write Makefile.am templates for Automake without having to have a deep knowledge of how to write Makefiles (arguably that's the point). I'm using Automake for my current project, and I haven't had to do much makefile editing (or even reading) to get stuff to work.

Answer 4

I learnt from the OReilly book, which I found really helpful.

Answer 5

Reading Recursive Make Considered Harmful was a huge help in understanding how make is supposed to work. The rest of the GNU make tools (autoconf, automake, and so on) seem like overkill for projects that target one OS. Further, once you know make inside and out, the other tools functionality become more obvious.

Answer 6

Grab a random small project and see how they use it. To be honest I've always let an IDE (Netbeans, Eclipse, etc) generate the stuff for me.

Answer 7

Start by understanding GNU Make. The online documentation is very useful and no-nonsense.

Meditate on Recursive Make Considered Harmful in order to grok how make is intended to be used.

Then if you start having to check the availability of libraries before starting the compilation, learn Autoconf.

Autoconf will take you a long way. It's reasonably straightforward once you wrap your mind around M4 quoting rules. With autoconf you can detect annoying situations like obsolete broken version of libraries you depend on.

I never needed to go any further. I tried several time to wrap my head around Automake, but it still caused my brain to melt, even after I had years of experience with GNU Make and Autoconf.

There are other funky things like libtool and pkg-config that can be nice bonus, but it's not something you need to spend time on before you have a clearly identified need.

If your software is internationalized, you need to use gettext too. But i18n is a whole domain, and you will do it wrong if you are not mentored. Fortunately, the i18n community is full of extremely helpful people.

Answer 8

While learning 'make' is loads of fun, recently I've been tinkering with Scons and it truly makes using make a thing of the past (no pun intended, too many uses of make). Much easier, cross-platform, able to do just about anything Make can do and more. Depends on Python.

Example: Compile all files in a directory into a final binary (SConstruct file)

# Will correctly use g++
Program('binaryname', Glob('*.cpp'))

Example: Build a shared library

# Build a new shared library, libtest.so from all .c files
# If you were windows you'd get test.dll
SharedLibrary('test', Glob('*.c'))

Magic!

Answer 9

I like SCons. I really do. There is only one problem, and its a major one. SCons does not scale beyond small-medium sized projects. When the number of source files approach go up in the 1000s, the incremental build performance goes down the drain. SCons can spend minutes just figuring out if anything needs to be rebuilt. This has been discussed over and over again on the SCons mailing list, but unfortunately there does not seem to be a solution anywhere near.

Autoconf is really a pain to use, and I'm not complaining about the M4 syntax. Google on "autoconf hell" for further reading. If you're running in a Unix environment (Cygwin included), you can solve many issues inside your makefile instead of resorting to Autoconf tools.

Answer 10

man pages.

Answer 11

As Thomee pointed out, there's nothing to fear. Makefiles are not strictly related to GNU build tools, you can automate any task you want with makefiles.

Whenever you want to rebuild (recompile, regenerate, reformat, reconfigure, ...) just a part of your system depending on which source file has changed, makefiles are your friends.

Things get nasty when you want to automate tasks (e.g. programs recompilation) taking into account the specific platform you're are on. Then you have to invent methods to detect which platform you are on, whether it supports certains compilers, where some libraries are, etc etc.

I learned first by looking at others makefiles (you learn a lot of tricks and hacks this way!) than using man and other online documentation.

Just as a side note, if you plan to move away from make, have a look at jam (http://www.perforce.com/jam/jam.html) or, the one that I use, ftjam (http://www.freetype.org/jam/), I've just discovered it after years of using make and I've been amazed of its simplicity and power.