[C++] What is metaprogramming?

Question

With reference to this question, could anybody please explain and post example code of metaprogramming? I googled the term up, but I found no examples to convince me that it can be of any practical use.

On the same note, is Qt's Meta Object System a form of metaprogramming?

jrh

Answer 1

Here's a common example:

  template <int N>
  struct fact {
      enum { value = N * fact<N-1>::value };
  };

  template <>
  struct fact<1> {
      enum { value = 1 };
  }; 

  std::cout << "5! = " << fact<5>::value << std::endl;

You're basically using templates to calculate a factorial.

A more practical example I saw recently was an object model based on DB tables that used template classes to model foreign key relationships in the underlying tables.

Answer 2

The following example is lifted from the excellent book C++ Templates - The complete guide.

#include <iostream>
using namespace std;

template <int N> struct Pow3 {
   enum { pow = 3 * Pow3<N-1>::pow };
}

template <> struct Pow3<0> {
   enum { pow = 1 };
}

int main() {
   cout << "3 to the 7 is " << Pow<7>::pow << "\n";
}

The point of this code is that the recursive calculation of the 7th power of 3 takes place at compile time rather than run time. It is thus extremely efficient in terms of runtime performance, at the expense of slower compilation.

Is this useful? In this example, probably not. But there are problems where performing calculations at compile time can be an advantage.

Answer 3

The concept comes entirely from the name Meta- means to abstract from the thing it is prefixed on.
In more 'conversational style' to do something with the thing rather than the thing itself.

In this regard metaprogramming is essentially writing code, which writes (or causes to be written) more code.

The C++ template system is meta programming since it doesn't simply do textual substitution (as the c preprocessor does) but has a (complex and inefficient) means of interacting with the code structure it parses to output code that is far more complex. In this regard the template preprocessing in C++ is Turing complete. This is not a requirement to say that something is metaprogramming but is almost certainly sufficient to be counted as such.

Code generation tools which are parametrizable may be considered metaprogramming if their template logic is sufficiently complex.

The closer a system gets to working with the abstract syntax tree that represents the language (as opposed to the textual form we represent it in) the more likely it is to be considered metaprogramming.

From looking at the QT MetaObjects code I would not (from a cursory inspection) call it meta programming in the sense usually reserved for things like the C++ template system or Lisp macros. It appears to simply be a form of code generation which injects some functionality into existing classes at the compile stage (it can be viewed as a precursor to the sort of Aspect Oriented Programming style currently in vogue or the prototype based object systems in languages like JavaScripts

As example of the sort of extreme lengths you can take this in C++ there is Boost MPL whose tutorial shows you how to get:

Dimensioned types (Units of Measure)

quantity<float,length> l( 1.0f );
quantity<float,mass> m( 2.0f );
m = l;    // compile-time type error

Higher Order Metafunctions

twice(f, x) := f(f(x))

template <class F, class X>
struct twice
  : apply1<F, typename apply1<F,X>::type>
{};

struct add_pointer_f
{
    template <class T>
    struct apply : boost::add_pointer<T> {};
};

Now we can use twice with add_pointer_f to build pointers-to-pointers:

BOOST_STATIC_ASSERT((
    boost::is_same<
         twice<add_pointer_f, int>::type
       , int**
    >::value
));

Answer 4

Another example: in this case the metaprogramming tecnique is used to get an arbitrary-precision value of PI at compile-time using the Gauss-Legendre algorithm.

Why should I use something like that in real world? For example to avoid repeating computations, to obtain smaller executables, to tune up code for maximizing performance on a specific architecture, ...

Personally I love metaprogramming because I hate repeating stuff and because I can tune up constants exploiting architecture limits.

I hope you like that.

Just my 2 cents.

/**
 *  FILE     : MetaPI.cpp
 *  COMPILE  : g++ -Wall -Winline -pedantic -O1 MetaPI.cpp -o MetaPI
 *  CHECK    : g++ -Wall -Winline -pedantic -O1 -S -c MetaPI.cpp [read file MetaPI.s]
 *  PURPOSE  : simple example template metaprogramming to compute the
 *             value of PI using [1,2].
 *
 *  TESTED ON:
 *  - Windows XP, x86 32-bit, G++ 4.3.3
 *
 *  REFERENCES:
 *  [1]: http://en.wikipedia.org/wiki/Gauss%E2%80%93Legendre_algorithm
 *  [2]: http://www.geocities.com/hjsmithh/Pi/Gauss_L.html
 *  [3]: http://ubiety.uwaterloo.ca/~tveldhui/papers/Template-Metaprograms/meta-art.html
 *
 *  NOTE: to make assembly code more human-readable, we'll avoid using
 *        C++ standard includes/libraries. Instead we'll use C's ones.
 */

#include <cmath>
#include <cstdio>

template <int maxIterations>
inline static double compute(double &a, double &b, double &t, double &p)
{
    double y = a;
    a = (a + b) / 2;
    b = sqrt(b * y);
    t = t - p * ((y - a) * (y - a));
    p = 2 * p;

    return compute<maxIterations - 1>(a, b, t, p);
}

// template specialization: used to stop the template instantiation
// recursion and to return the final value (pi) computed by Gauss-Legendre algorithm
template <>
inline double compute<0>(double &a, double &b, double &t, double &p)
{
    return ((a + b) * (a + b)) / (4 * t);
}

template <int maxIterations>
inline static double compute()
{
    double a = 1;
    double b = (double)1 / sqrt(2.0);
    double t = (double)1 / 4;
    double p = 1;

    return compute<maxIterations>(a, b, t, p); // call the overloaded function
}

int main(int argc, char **argv)
{
    printf("\nTEMPLATE METAPROGRAMMING EXAMPLE:\n");
    printf("Compile-time PI computation based on\n");
    printf("Gauss-Legendre algorithm (C++)\n\n");

    printf("Pi=%.16f\n\n", compute<5>());

    return 0;
}

Answer 5

Although it's large (2000loc) I made a reflexive class system within c++ that is compiler independant and includes object marshalling and metadata but has no storage overhead or access time penalties. It's hardcore metaprogramming, and being used in a very big online game for mapping game objects for network transmission and database-mapping (ORM).

Anyways it takes a while to compile, about 5 minutes, but has the benefit of being as fast as hand tuned code for each object. So it saves lots of money by reducing significant CPU time on our servers (CPU usage is 5% of what it used to be).

Answer 6

Most of the examples so far have operated on values (computing digits of pi, the factorial of N or similar), and those are pretty much textbook examples, but they're not generally very useful. It's just hard to imagine a situation where you really need the compiler to comput the 17th digit of pi. Either you hardcode it yourself, or you compute it at runtime.

An example that might be more relevant to the real world could be this:

Let's say we have an array class where the size is a template parameter(so this would declare an array of 10 integers: array<int, 10>)

Now we might want to concatenate two arrays, and we can use a bit of metaprogramming to compute the resulting array size.

template <typename T, int lhs_size, int rhs_size>
array<T, lhs_size + rhs_size> concat(const array<T, lhs_size>& lhs, const array<T, rhs_size>& rhs){

  array<T, lhs_size + rhs_size> result;
  // copy values from lhs and rhs to result
  return result;

}

A very simple example, but at least the types have some kind of real-world relevance. This function generates an array of the correct size, it does so at compile-time, and with full type safety. And it is computing something that we couldn't easily have done either by hardcoding the values (we might want to concatenate a lot of arrays with different sizes), or at runtime (because then we'd lose the type information)

More commonly, though, you tend to use metaprogramming for types, rather than values.

A good example might be found in the standard library. Each container type defines its own iterator type, but plain old pointers can also be used as iterators. Technically an iterator is required to expose a number of typedef members, such as value_type, and pointers obviously don't do that. So we use a bit of metaprogramming to say "oh, but if the iterator type turns out to be a pointer, its value_type should use this definition instead."

There are two things to note about this. The first is that we're manipulating types, not values We're not saying "the factorial of N is so and so", but rather, "the value_type of a type T is defined as..."

The second thing is that it is used to facilitate generic programming. (Iterators wouldn't be a very generic concept if it didn't work for the simplest of all examples, a pointer into an array. So we use a bit of metaprogramming to fill in the details required for a pointer to be considered a valid iterator).

This is a fairly common use case for metaprogramming. Sure, you can use it for a wide range of other purposes (Expression templates are another commonly used example, intended to optimize expensive calculations, and Boost.Spirit is an example of going completely overboard and allowing you to define your own parser at compile-time), but probably the most common use is to smooth over these little bumps and corner cases that would otherwise require special handling and make generic programming impossible.

Answer 7

It's hard to say what C++ meta-programming is. More and more I feel it is much like introducing 'types' as variables, in the way functional programming has it. It renders declarative programming possible in C++.

It's way easier to show examples.

One of my favorites is a 'trick' (or pattern:) ) to flatte multiply nested switch/case blocks:

#include <iostream>
using namespace std;

enum CCountry { Belgium, Japan };
enum CEra     { ancient, medieval, future };

// nested switch
void historic( CCountry country, CEra era ) {
  switch( country ) {
        case( Belgium ):
          switch( era ) {
            case( ancient ): cout << "Ambiorix"; break;
            case( medieval ): cout << "Keizer Karel"; break;
          }
          break;
        case( Japan ):
          switch( era ) {
            case( future ): cout << "another Ruby?"; break;
            case( medieval ): cout << "Musashi Mijamoto"; break;
          }
          break;
  }
}


// the flattened, metaprogramming way
// define the conversion from 'runtime arguments' to compile-time arguments (if needed...)
// or use just as is.
template< CCountry country, CEra era > void thistoric();


template<> void thistoric<Belgium, ancient> () { cout << "Ambiorix"; }
template<> void thistoric<Belgium, medieval>() { cout << "Keizer Karel"; }
template<> void thistoric<Belgium, future  >() { cout << "Beer, lots of it"; }

template<> void thistoric<Japan, ancient> () { cout << "wikipedia"; }
template<> void thistoric<Japan, medieval>() { cout << "Musashi"; }
template<> void thistoric<Japan, future  >() { cout << "another Ruby?"; }


// optional: conversion from runtime to compile-time
//
template< CCountry country > struct SelectCountry {
  static void select( CEra era ) {
    switch (era) {
          case( medieval ): thistoric<country, medieval>(); break;
          case( ancient  ): thistoric<country, ancient >(); break;
          case( future   ): thistoric<country, future  >(); break;

    }
  }
};

void Thistoric ( CCountry country, CEra era ) {
    switch( country ) {
          case( Belgium ): SelectCountry<Belgium>::select( era ); break;
          case( Japan   ): SelectCountry<Japan  >::select( era ); break;
    }
  } 



int main() {   
  historic( Belgium, medieval ); // plain, nested switch
  thistoric<Belgium,medieval>(); // direct compile time switch
  Thistoric( Belgium, medieval );// flattened nested switch
  return 0;
}

Answer 8

QtMetaObject basically implements reflection (Reflection) and IS one of the major forms of metaprogramming, quite powerful actually. It is similar to Java's reflection and it's also commonly used in dynamic languages (Python, Ruby, PHP...). It's more readable than templates, but both have their pros and cons.

Answer 9

The only time I needed to use Boost.MPL in my day job was when I needed to convert boost::variant to and from QVariant.

Since boost::variant has an O(1) visitation mechanism, the boost::variant to QVariant direction is near-trivial.

However, QVariant doesn't have a visitation mechanism, so in order to convert it into a boost::variant, you need to iterate over the mpl::list of types that the specific boost::variant instantiation can hold, and for each type ask the QVariant whether it contains that type, and if so, extract the value and return it in a boost::variant. It's quite fun, you should try it :)