I'm curious to find out if and when C++ meta templates are a good design choice for systems small to large. I understand that they increase your build time in order to speed up your execution time. However, I've heard that the meta template code is inherently hard to understand by many developers, which could be a problem for a large group of people working on a system with a very large code base (millions of lines of code). Where do you think C++ meta templates are useful (or not)?
Metaprogramming is just another tool in a (C++) programmers' toolbox - it has many great applications, but like anything can be mis- or over- used. I think it's got a bad reputation in terms of 'hard to use', and I think this mainly comes from the fact that it's a significant addition to the language and so takes a while to learn.
As an example of real-world use; I've used template metaprogramming to implement Compile-time asserts and shim libraries in the past; implementing these without templates would either have been impossible or required significantly more code than I had to write.
In the case of the shim library it could have been implemented in a classic object-orientated fashion which could have been designed to have a similar (low) level of code duplication as the templated implementation; however it's runtime performance would have been significantly worse.
If you want to see some good examples of how it can be used, I suggest you read Modern C++ Design by Andrei Alexandrescu (there is a sample chapter on the publisher's website) - IMO this is one of the best books on the subject.
I did face a situation where I had to tackle a not-so-big system which heavily used template metaprogramming (specifically static polymorphismus, SFINAE and maybe other techniques). And I can tell you that this will make it harder for the developers. If template metaprogramming is used a lot, every developer must be familiar with the techniques otherwise they won't be able to work productively.
On the other hand, some uses of templates are quite easy to understand and use. For example generic containers (vector), smart pointers, ...
Template metaprogramming doesn't make your code "inherently hard to understand". It moves the complexity around. The underlying metaprogramming code may be a pain to understand, but at the same time, it typically simplifies client code.
If its effect was just to make code harder to read, it wouldn't be used. The reason it is used from time to time is exactly that it simplifies the code as a whole.
Of course, programmers who are not familiar with metaprogramming will have trouble reading or maintaining the code, but isn't that just an argument against working with programmers who don't know their stuff?
Programmers who don't know about a for-loop wil find that hard to read too.
Fairly simple meta-programming is used throughout the standard library, in the form of traits. The standard library seems to be pretty well received, so I think we can say that meta programming is useful there.
As always, you need to balance the pros and the cons. If performance is your concern (there are other uses for template metaprogramming too), you should first demonstrate that there are significant and important performance gains to be had by using template metaprogramming techniques. (It's even quite possible to make a program that runs slower by hindering the compiler with excessive use of templates, so always measure!)
C++ templates were not originally designed for metaprogramming so even relatively simple problems solved with metaprogramming can produce code that is difficult to understand, especially to people who are not familiar with common template metaprogramming techniques (since they are usually not very intutitive). Also depending on your particular problem you might consider code generation vs. template metaprogramming.
This depends on who you're working with, and what they like and are familiar with. In the absence of any information I'd suggest the following concrete list of uncontroversial (and not very meta...) template 'things' for a new project:
- handwritten smart pointers for whatever sorts of things your app uses
- STL containers
- static assert
And, for advanced users only:
- traits used for some obvious standard application (script language bindings and serialization spring to mind)
This is perhaps a bit conservative but it seems to be easy to convince everybody of the value of these things. If well put together, they don't bloat compile times and mostly work pretty well with commonly-found code browsing facilities. And most of the templates shouldn't take much explaining, even to that large subset of C++ programmers who don't really understand templates all that well.
(Regarding boost, and any other template libraries that it has yet to merge with, it seems to be pretty adventurous by the standards of many (most?) C++ programmers at the moment. So it seems to me prudent to avoid it for now.)