ANTLR:Translate the modified AST to java source code by stringtemplate

Question

I use the grammar Java.g from the ANTLR wiki produces a lexer and parser for Java source files.Then use the following code to generate an abstract syntax tree (AST).

    ANTLRInputStream input = new ANTLRInputStream(new FileInputStream(fileName));
    JavaLexer lexer = new JavaLexer(input);     // create lexer
    // create a buffer of tokens pulled from the lexer
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    JavaParser parser = new JavaParser(tokens); // create parser
    JavaParser.javaSource_return r = parser.javaSource();   // parse rule 'javaSource'
    /*RuleReturnScope result = parser.compilationUnit();
    CommonTree t = (CommonTree) result.getTree();*/
    // WALK TREE
    // get the tree from the return structure for rule prog
    CommonTree t = (CommonTree)r.getTree();

Then modify the AST. For example,replace "File file = new File(filepath, fileType);" to "S3Object _file = new S3Object(_fileName);" by modify the AST node. After this,I want to translate this AST to java source code.I modify the JavaTreeParser.g and write a stringtemplate and use the following method to get the java source code:

    FileReader groupFileR = new FileReader("src/com/googlecode/zcg/templates/JavaTemplate.stg");
    StringTemplateGroup templates = new StringTemplateGroup(groupFileR);
    groupFileR.close();
    // create a stream of tree nodes from AST built by parser
    CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);
    // tell it where it can find the token objects
    nodes.setTokenStream(tokens);
    JavaTreeParser walker = new JavaTreeParser(nodes); // create the tree Walker
    walker.setTemplateLib(templates); // where to find templates
    // invoke rule prog, passing in information from parser
    JavaTreeParser.javaSource_return r2 = walker.javaSource();

    // EMIT BYTE CODES
    // get template from return values struct
    StringTemplate output = (StringTemplate)r2.getTemplate(); 
    System.out.println(output.toString()); // render full template

If I don't modify the AST,it will get the java source code correctly,but after I modify the AST,it doesn't get the right java source code(the AST was modified correctly).For example,if I input the following souce code,and translate to AST,then modify "File file = new File(filepath, fileType);" to "S3Object _file = new S3Object(_fileName);":

public void methodname(String address){
    String filepath = "file";
    int fileType = 3;       
    File file = new File(filepath, fileType);
}

the result will be the following:

public void methodname( String address)
  { 
     String filepath="file";
     int fileType=3;
     methodname (Stringaddress){Stringfilepath;//it's not  what I wanted
  }

Am I doing it wrong? Is there a more proper way for me to solve this problem?

Fascia

Answer 1

unfortunately I cannot recommend doing source to source translation by rewriting the abstract syntax trees; try using the parse trees. If I remember ANTLR 3 can also generate those easily. Ter

Mcneal

Thanks. I need to get the type of the variable from the tree and modify the tree, but the parse tree is much more complicate than the abstract syntax tree. Does parse tree can complete such task? – Fascia

parse trees are much bigger but make it easier to do source to source translation. ANTLR 4.2 will help a lot. see master branch for tree pattern stuff. – Mcneal