Linking Xtext with StringTemplate code generator

Question

In my current project, I am trying to link the DSL specification written in xtext and code generator written in StringTemplate.

for instance, the syntax of my DSL specification is as follows. I am entering this information through nice editor facilities provided by xText.

structs:
    TempStruct
        tempValue : double;
            unitOfMeasurement : String;

abilities :
    sensors:    
        TemperatureSensor
            generate tempMeasurement : TempStruct;
            attribute responseFormat : String;  

The grammar of above mentioned DSL specification is as follows:

       VocSpec:

          'structs' ':'
          (structs += Struct)+

          'abilities' ':'
           ('sensors' ':' (sensors += Sensor)+ )+ 
         ;

      Sensor:
          name = ID
          ((attributes += Attributes ) |
          (sources += Sources))* 
          ;

     Sources:
          'generate' name=ID ':' type = Type ';' 
           ;

     Attributes:
         'attribute' name=ID ':' type = Type ';' 
           ; 

    Struct:
          name = ID
          (fields += Field)+ 
         ;

    Field:
         name=ID ':' type += Type ';' 
        ;

The xText generates semantic model corresponding to above mentioned Specification. In our example, xText generates Semantic model, which contains files such as struct.java, Field.java, Attribute.java, Sensor.java, etc.

I can clearly see that this semantic model can be linked with the StringTemplate file. The StringTemplate file takes object of the class. For instance, StringTemplate file takes TemperatureSensor (instance of Sensor) as Input and generate Java code.

My question is how can I instantiate the semantic model ( generated by xText ) and What do I need to do to link with StringTemplate files ?

Bentwood

Answer 1

Iff you want to generate the code with StringTemplate from within Eclipse:

Locate the generator stub in the runtime project of your DSL. There should be a class that implements the IGenerator interface. The method #doGenerator will be invoked with a resource and an instance of the IFileSystemAccess. The resource is an EMF concepts - basically an abstraction over the physical location of your objects. It offer getContents which in turn will provide access to a list of instances of VocSpec (if the grammar snippet is complete). These instances can be passed to your string template thing which will produce the output. The output should be written by means of IFileSystemAccess#generateFile

If you want to do that as a standalone process, you should follow the steps in the Xtext FAQ. They explain how to load the EMF resource. Afterwards you can do pretty much the same as in the Eclipse-based solution. That is, implement the IGenerator and pass the result to the IFileSystemAccess.

To give you a short example, this is what should be done to get started in a few minutes:

First you should enable the following code snippet in the 'GenerateMyDsl.mwe2' workflow file and run the workflow.

fragment = generator.GeneratorFragment {
    generateMwe = false
    generateJavaMain = true
}

You'll find a new artifact in the runtime project's package with the suffix .generator. Namely the 'Main.java' file.

The second step is to implement the generator. The following snippet can be used in the 'MyDslGenerator.xtend' class:

package org.xtext.example.mydsl.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IGenerator
import org.eclipse.xtext.generator.IFileSystemAccess
import org.antlr.stringtemplate.StringTemplate
import org.antlr.stringtemplate.language.DefaultTemplateLexer
import org.xtext.example.mydsl.myDsl.Model

class MyDslGenerator implements IGenerator {

    override void doGenerate(Resource resource, IFileSystemAccess fsa) {
        val hello = new StringTemplate("Generated with StringTemplate, $greeting.name$!", typeof(DefaultTemplateLexer))
        val model = resource.contents.head as Model
        hello.setAttribute("greeting", model.greetings.head)
        fsa.generateFile("Sample.txt", hello.toString())
    }
}

The Java equivalent would be something along these lines:

package org.xtext.example.mydsl.generator;

import org.antlr.stringtemplate.StringTemplate;
import org.antlr.stringtemplate.language.DefaultTemplateLexer;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.xtext.generator.IFileSystemAccess;
import org.eclipse.xtext.generator.IGenerator;
import org.xtext.example.mydsl.myDsl.Model;

public class StringTemplateGenerator implements IGenerator {

    public void doGenerate(Resource input, IFileSystemAccess fsa) {
        StringTemplate hello = new StringTemplate("Generated with StringTemplate, $greeting.name$!", DefaultTemplateLexer.class);
        Model model = (Model) input.getContents().get(0);
        hello.setAttribute("greeting", model.getGreetings().get(0));
        fsa.generateFile("Sample.txt", hello.toString());
    }

}

Next up one has to change the content of the stub 'Main.java' to reflect the location of the input file and the expected output path.

package org.xtext.example.mydsl.generator;

import java.util.List;

import org.eclipse.emf.common.util.URI;
import org.eclipse.emf.ecore.resource.Resource;
import org.eclipse.emf.ecore.resource.ResourceSet;
import org.eclipse.xtext.generator.IGenerator;
import org.eclipse.xtext.generator.JavaIoFileSystemAccess;
import org.eclipse.xtext.util.CancelIndicator;
import org.eclipse.xtext.validation.CheckMode;
import org.eclipse.xtext.validation.IResourceValidator;
import org.eclipse.xtext.validation.Issue;

import com.google.inject.Inject;
import com.google.inject.Injector;
import com.google.inject.Provider;

public class Main {

    public static void main(String[] args) {
        Injector injector = new MyDslStandaloneSetupGenerated().createInjectorAndDoEMFRegistration();
        Main main = injector.getInstance(Main.class);
        main.runGenerator("input/Sample.mydsl");
    }

    @Inject 
    private Provider<ResourceSet> resourceSetProvider;

    @Inject
    private IResourceValidator validator;

    @Inject
    private IGenerator generator;

    @Inject 
    private JavaIoFileSystemAccess fileAccess;

    protected void runGenerator(String string) {
        // load the resource
        ResourceSet set = resourceSetProvider.get();
        Resource resource = set.getResource(URI.createURI(string), true);

        // validate the resource
        List<Issue> list = validator.validate(resource, CheckMode.ALL, CancelIndicator.NullImpl);
        if (!list.isEmpty()) {
            for (Issue issue : list) {
                System.err.println(issue);
            }
            return;
        }

        // configure and start the generator
        fileAccess.setOutputPath("output/");
        generator.doGenerate(resource, fileAccess);

        System.out.println("Code generation finished.");
    }
}

The input file is located in the runtime project in a newly created folder 'input'. The content of the file 'Sample.mydsl' is

Hello Pankesh!

Now you can run the main class and after a quick refresh in Eclipse, you find the new 'output' folder in my runtime project with a single file 'Sample.txt':

Generated with StringTemplate, Pankesh!

Btw: the Xtext documentation contains a tutorial on how to generate code with Xtend - it is nice than StringTemplate because it integrates seemlessly with Eclipse and existing Java utilities:

package org.xtext.example.mydsl.generator

import org.eclipse.emf.ecore.resource.Resource
import org.eclipse.xtext.generator.IFileSystemAccess
import org.eclipse.xtext.generator.IGenerator
import org.xtext.example.mydsl.myDsl.Model

class MyDslGenerator implements IGenerator {

    override void doGenerate(Resource resource, IFileSystemAccess fsa) {
        val model = resource.contents.head as Model
        fsa.generateFile("Sample.txt", '''
            Generated with Xtend, «model.greetings.head»!
        ''')
    }
}

Solarism

Thanks Sebastian. I would love to write my code generator with xtend. But, the problem is that our code generator is written with String Template files. Thanks for your advice. I will try your approach. If nothing work out, I will use editor support of xtext and give them ANTLR grammar to parse. – Bentwood

Dear Sebastian, could you please elaborate your explanation with an example ? – Bentwood

Dear Sebastian, kindly clarify your sentence "It offer getContents which in turn will provide access to a list of instances of VocSpec (if the grammar snippet is complete)". I have written above grammar. you mean above grammar is not complete for generating instances ? – Bentwood

Pankesh, thanks for the suggestion. I updated the answer and added many code snippets. – Solarism