Source-to-source compiler framework wanted

Question

Dear all,

I used to use OpenC++ (http://opencxx.sourceforge.net/opencxx/html/overview.html) to perform code generation like:

Source:

class MyKeyword A {
  public:
    void myMethod(inarg double x, inarg const std::vector<int>& y, outarg double& z);
};

Generated:

class A {
  public:
    void myMethod(const string& x, double& y);
    // generated method below:
    void _myMehtod(const string& serializedInput, string& serializedOutput) {
      double x;
      std::vector<int> y;
      // deserialized x and y from serializedInput
      double z;
      myMethod(x, y, z);
    }
};

This kind of code generation directly matches the use case in the tutorial of OpenC++ (http://www.csg.is.titech.ac.jp/~chiba/opencxx/tutorial.pdf) by writing a meta-level program for handling "MyKeyword", "inarg" and "outarg" and performing the code generation. However, OpenC++ is sort of out-of-date and inactive now, and my code generator can only work on g++ 3.2 and it triggers error on parsing header files of g++ of higher version.

I have looked at VivaCore, but it does not provide the infra-structure for compiling meta-level program. I'm also looking at LLVM, but I cannot find documentation that tutor me on working out my source-to-source compilation usage. I'm also aware of the ROSE compiler framework, but I'm not sure whether it suits my usage, and whether its proprietary C++ front-end binary can be used in a commercial product, and whether a Windows version is available.

Any comments and pointers to specific tutorial/paper/documentation are much appreciated.

Answer 1

Are you aware of the practice of template meta-programming? If you haven't used it before, it's the application of the C++ preprocessor to create oddish meta-programs that feel more like LISP than C++. The idea is the same as above -- having a pre-compile step that generates repeated code based on certain inputs. However, it is all executed at compile time (whereas it looks like OpenC++ does several things at run time).

Considering it looks like you're willing to learn a new one regardless, would you be willing to use that as a replacement "language"?

Boost provides a library which uses this technique to provide easy serialization, like what you showed above. From the tutorial in its manual:

/////////////////////////////////////////////////////////////
// gps coordinate
//
// illustrates serialization for a simple type
//
class gps_position
{
private:
    friend class boost::serialization::access;
    // When the class Archive corresponds to an output archive, the
    // & operator is defined similar to <<.  Likewise, when the class Archive
    // is a type of input archive the & operator is defined similar to >>.
    template<class Archive>
    void serialize(Archive & ar, const unsigned int version)
    {
        ar & degrees;
        ar & minutes;
        ar & seconds;
    }
    int degrees;
    int minutes;
    float seconds;
public:
    gps_position(){};
    gps_position(int d, int m, float s) :
        degrees(d), minutes(m), seconds(s)
    {}
};

int main() {
    // create and open a character archive for output
    std::ofstream ofs("filename");

    // create class instance
    const gps_position g(35, 59, 24.567f);

    // save data to archive
    {
        boost::archive::text_oarchive oa(ofs);
        // write class instance to archive
        oa << g;
        // archive and stream closed when destructors are called
    }

    // ... some time later restore the class instance to its orginal state
    gps_position newg;
    {
        // create and open an archive for input
        std::ifstream ifs("filename");
        boost::archive::text_iarchive ia(ifs);
        // read class state from archive
        ia >> newg;
        // archive and stream closed when destructors are called
    }
    return 0;
}

Answer 2

I do not know of any ready-to-use solution, but you could build your own with a relatively little effort. One possible option is Elsa C++ parser, a bit out of date, but easy to use and quite extendible. Another option is to tamper with XML ASTs produced by Clang++. I used both approaches in different scenarios.

Answer 3

You might consider the DMS Software Reengineering Toolkit. DMS is a general foundation for parsing source text in arbitrary languages to compiler data structures (ASTs, symbol tables, control flow graphs, data flow graphs depending on how far you take it).

DMS is a general purpose Source-to-source program transformation system. You can apply source-to-source pattern-directed transformations, or write procedural transformations (much like OpenC++), and then regenerate compilable source text corresponding to the transformed program.

DMS is parameterized by explicit langauge definitions, and handles C, C#, COBOL, Java, Python, javascript, Fortran.

It has a full C++ Front End that handles many real dialects of C++ (ANSI, GNU, MS), with full name and type resolution. DMS with the C++ front end can carry out transformations controlled by "metaprograms" within and across multiple compilation units. It has been used in anger to do radical reorganizations of C++ software systems, including massive rearchitecting of mission avionics software (see papers at website), finally used in UAVs.

DMS runs on Windows.