Source-to-source compiler framework wanted

Question

I used to use OpenC++ (http://opencxx.sourceforge.net/opencxx/html/overview.html) to perform code generation like:

Source:

class MyKeyword A {
  public:
    void myMethod(inarg double x, inarg const std::vector<int>& y, outarg double& z);
};

Generated:

class A {
  public:
    void myMethod(const string& x, double& y);
    // generated method below:
    void _myMehtod(const string& serializedInput, string& serializedOutput) {
      double x;
      std::vector<int> y;
      // deserialized x and y from serializedInput
      double z;
      myMethod(x, y, z);
    }
};

This kind of code generation directly matches the use case in the tutorial of OpenC++ (http://www.csg.is.titech.ac.jp/~chiba/opencxx/tutorial.pdf) by writing a meta-level program for handling "MyKeyword", "inarg" and "outarg" and performing the code generation. However, OpenC++ is sort of out-of-date and inactive now, and my code generator can only work on g++ 3.2 and it triggers error on parsing header files of g++ of higher version.

I have looked at VivaCore, but it does not provide the infra-structure for compiling meta-level program. I'm also looking at LLVM, but I cannot find documentation that tutor me on working out my source-to-source compilation usage. I'm also aware of the ROSE compiler framework, but I'm not sure whether it suits my usage, and whether its proprietary C++ front-end binary can be used in a commercial product, and whether a Windows version is available.

Any comments and pointers to specific tutorial/paper/documentation are much appreciated.

Deerstalker

You can't use the Rose framework in a commercial context without getting your own license to the EDG front end. Dunno if Rose has a Windows version. – Liar

Answer 1

I do not know of any ready-to-use solution, but you could build your own with a relatively little effort. One possible option is Elsa C++ parser, a bit out of date, but easy to use and quite extendible. Another option is to tamper with XML ASTs produced by Clang++. I used both approaches in different scenarios.

Libau

Thanks all very much for providing answers to the problem. Elas and Clang++ are very useful information. – Deerstalker

Clang is what I would strongly recommend. LLVM is too far down the compiler path for your needs, but Clang is in the right spot. – Petrology

The Elsa site appears to not have had any updates since about 2005; it claims to try to parse C++ 2003. – Liar

@IraBaxter, yes, Elsa is deprecated, as well as Clang XML serialisation. Now one should use libclang instead, it covers all the possible use cases previously served by the other tools. – Libau

Answer 2

Are you aware of the practice of template meta-programming? If you haven't used it before, it's the application of the C++ preprocessor to create oddish meta-programs that feel more like LISP than C++. The idea is the same as above -- having a pre-compile step that generates repeated code based on certain inputs. However, it is all executed at compile time (whereas it looks like OpenC++ does several things at run time).

Considering it looks like you're willing to learn a new one regardless, would you be willing to use that as a replacement "language"?

Boost provides a library which uses this technique to provide easy serialization, like what you showed above. From the tutorial in its manual:

/////////////////////////////////////////////////////////////
// gps coordinate
//
// illustrates serialization for a simple type
//
class gps_position
{
private:
    friend class boost::serialization::access;
    // When the class Archive corresponds to an output archive, the
    // & operator is defined similar to <<.  Likewise, when the class Archive
    // is a type of input archive the & operator is defined similar to >>.
    template<class Archive>
    void serialize(Archive & ar, const unsigned int version)
    {
        ar & degrees;
        ar & minutes;
        ar & seconds;
    }
    int degrees;
    int minutes;
    float seconds;
public:
    gps_position(){};
    gps_position(int d, int m, float s) :
        degrees(d), minutes(m), seconds(s)
    {}
};

int main() {
    // create and open a character archive for output
    std::ofstream ofs("filename");

    // create class instance
    const gps_position g(35, 59, 24.567f);

    // save data to archive
    {
        boost::archive::text_oarchive oa(ofs);
        // write class instance to archive
        oa << g;
        // archive and stream closed when destructors are called
    }

    // ... some time later restore the class instance to its orginal state
    gps_position newg;
    {
        // create and open an archive for input
        std::ifstream ifs("filename");
        boost::archive::text_iarchive ia(ifs);
        // read class state from archive
        ia >> newg;
        // archive and stream closed when destructors are called
    }
    return 0;
}

Vinaigrette

Thanks for the idea on using preprocessor. However, some legacy and compatibility issue limit me from changing the current usage too much. – Deerstalker

Answer 3

You might consider our DMS Software Reengineering Toolkit. DMS is a general foundation for parsing source text in arbitrary languages to compiler data structures (ASTs, symbol tables, control flow graphs, data flow graphs depending on how far you take it).

DMS is a general purpose Source-to-source program transformation system. You can apply source-to-source pattern-directed transformations, or write procedural transformations (much like OpenC++), and then regenerate compilable source text corresponding to the transformed program.

DMS is parameterized by explicit language definitions, and handles C, C#, COBOL, Java, Python, javascript, Fortran.

It has a full C++ Front End that handles many real dialects of C++ (ANSI, GNU, MS), with full name and type resolution. DMS with the C++ front end can carry out transformations controlled by "metaprograms" within and across multiple compilation units. It has been used in anger to do radical reorganizations of C++ software systems, including massive rearchitecting of mission avionics software (see papers at website), finally used in UAVs.

DMS runs on Windows, and transparently on Linux under Wine using sh scripts.

EDIT 2/3/2011: DMS seems run fine under Wine on Linux and Solaris, too. Testing underway for DMS on Wine under OSX.

EDIT 3/1/2011: DMS seems to run under Wine for OSX, too.

EDIT 2/21/2013: The C++ front end now handles ANSI C++11, as well as MS and GNU versions of C++11.

EDIT 2/24/2015: Now handles C++14 in ANSI, MS and GNU flavors.

EDIT 1/16/2019: Now handles C++17 in ANSI, MS and GNU flavors.

Liar