Which open-source project can be used a 'showcase' for mid-size project MAKEFILE management?

Question

hi, guys

now i need to design/organize the source-code structure and makefiles for the next project. This is a software implemented largely by C++ and supposed to be used normally on Linux. It will include following components:

projecthome/3rd_party_lib_boost/ hdr and src
projecthome/3rd_party_lib_from_vendor/ hdr and src

# libraries that will use 3rd party libs
projecthome/lib_base_struct/ hdr and src
projecthome/lib_utilites/ hdr and src

# applications that will generate binaries, they depend on the above libs, these applications will be developed on after another, while the number of libraries are likely to be fixed.
projecthome/app_1/ hdr and src
projecthome/app_2/ hdr and src

# shell scripts that run the above binaries
projecthome/sh

# python scripts that analyze the logs
projecthome/py

# the configuration files that need to configure the binaries
projecthome/config

# how to build? this is the most difficult party now I need to address
projecthome/build

Now I need a way to organize the above files, and most importantly, the makefiles.

This is the first time that I design such an 'architecture' by myself. So I come here for advices.

I think the most convenient way is to download an open-source project's source and use it as a model. Can any one recommend a mid-size project who has similar structure as above?

oh, my project is not very large, i think it should have 10k-20k lines of c++ code

another thing is that, I hope the above components will not depend too much on each other, because at least one application will be sub-contracted to people outside of the company, I don't want him/her checkout the whole projecthome directory to compile.

Can anyone give me a clue ? thanks a lot!

Answer 1

Here are some proposals regarding the file system layout. We use a similar approach for our own big-sized multi-platform builds.

Generally I would try to group the directory hierarchy by "logical means", not by means of vendor or open-source.

the src directory contains the sources (which was called "build" in your example before). You should also contain some samples there if you are shipping libraries.

regarding 3rd party libs I would not differentiate between vendor and non-vendor 3rd party libs.

the tools directory should should contain your helpers. I wouldn't be to picky if it is for building, or for analyzing only the logfiles.

the tmp_build directory should be used for building only, so that you can quickly remove it in case that a complete re-build is necessary.

 projecthome
 |
 |---_src                # your own libs and applications and examples
 |   |-- config
 |   |-- app
 |   |   |-- app1
 |   |   `-- app2
 |   |-- lib
 |   |   |-- base_struct
 |   |   `-- utilites
 |   `-- samples
 |       |-- usage1
 |       `-- usage2
 |
 |-- 3rd_party           # all vendor and non-vendor 3rd party libs
 |   |-- boost
 |   |-- vendor1
 |   `-- vendor2
 |
 |-- build_tools             # all of your tools. Even it is only used for log analyze
 |   |-- build_helper_1
 |   |-- build_helper_2
 |   |-- py
 |   `-- sh
 |
 `-- tmp_build               # *temporary* build directory. Initially empty

Edit: it might also be a good convention to start the most important directories with an undersore (e.g. _src). This has the advantage that on most OS these directories are listed first (Explorer, or ls command).

Edit: the Makefiles in the "final directories" below the _src directory should all have the same distance to _src. E.g.:

_src/app/app1/Makefile -> distance to _src is 2
_src/app/app2/Makefile -> distance to _src is 2
_src/lib/utilities/Makefile -> distance to _src is 2
_src/lib/base_struct/Makefile -> distance to _src is 2
_src/samples/usage1/Makefile -> distance to _src is 2
_src/samples/usage2/Makefile -> distance to _src is 2

The reason for this "same distance" rule is that it allows you to resolve inter-directory dependencies in a generic way using relative-paths, e.g. with the following Makefile snippet for src/app/app1/Makefile:

PROJECT_HOME?=$(shell cd ../../.. && pwd)

Answer 2

Postfix and Subversion use Makefiles.

Answer 3

I wish I could upvote the comment suggesting to avoid make/autotools more than once. They're a really antiquated and unwieldy set of tools that I suspect are no longer fully understood by more than half a dozen creatures in the universe. Both cmake and scons are much better choices indeed (I personally prefer the latter).

Oh, and avoid svn:externals - that road leads straight to hell ;)

Answer 4

Rather than using an open source project as a template, I would define your own structure and build scripts so that you take full control and adapt it to your needs.

For the structure, choose a logical structure that separates generic libraries (to be shared between different applications), third party libraries and applications. For example:

.
|-- app
|   |-- app1
|   |   `-- src
|   `-- app2
|       `-- src
|-- common_build
|-- lib
|   |-- lib1
|   |   `-- src
|   `-- lib2
|       `-- src
`-- third_party
    `-- boost

For the build tool, have a look at some advanced build tools such as:

• Boost Build
• Scons
• Cmake
• ... etc

The advantage of these tools is that they handle complexity that you would have to implement yourself with Make.

I have experience with Boost Build. It is used to build the Boost C++ libraries (although note that they are currently experimenting with Cmake). Boost Build automatically handles library dependencies, building of different toolchains, building of different variants (debug, release, static/dynamic linking, etc). It is a very powerful tool.