Pros/cons of different language workbench tools such as Xtext and MPS?

Question

Does anyone have experience working with language workbench tools such as Xtext, Spoofax, and JetBrains' MPS? I'm looking to try one out and am having a hard time finding a good comparison of the different tools. What are the pros and cons of each?

I'm looking to build DSLs that generate python code, so I'm especially interested to hear from people who've used one of these tools with python (all three seem pretty Java-focused... why is that?). The DLSs are primarily for my own use, so I care less about building a really pretty IDE than I do about it being KISS to define the syntax and write the code generator. The ability to type-check / do static analysis of the DLSs would be pretty cool too.

I'm a little afraid of getting far down a path, hitting a wall, and realizing that all my code is in a format that can't be ported to anything else -- is that a risk with these tools? MPS in particular seems a little scary since as I understand it you don't really generate text-based syntaxes but rather build specialized editors for ASTs.

Callisthenics

I'm going to start exploring Spoofax b/c it seems to be backed by a pretty rich program transformation language (Stratego), which I'd guess is really the meat-and-potatoes of building a DSL... – Callisthenics

So at this point I've narrowed it down to Stratego / Spoofax vs building my own parser / interpreter in Python (via ANTLR) or similar. My sense is that the main trade-off is that Stratego offers a better paradigm for manipulating my ASTs, but that the write-build-test cycle won't be as clean because I need to output Python code then execute the code as opposed to keeping the entire application in Python. – Callisthenics

Answer 1

Markus Voelter does a pretty good job comparing those three in se-radio and Software ArchitekTOUR podcasts. The basic idea is, that Xtext is most used, therefore most stable and documented, and it is based on popular Eclipse platform and modeling ecosystem - EMF which surrounds it. On the other hand it is parser based and uses ANTLR internally, which means the kind of grammars you can define is limited and languages cannot be combined easily. Spoofax is an academic product with least adoption of those three. It is also parser based, but uses its own parser generator internally which allows language combinations. Jetbrains MPS is projection based, which gives much freedom to language designer and allows combinations of languages. *t also has solid support. Drawback might be the learning curve. None of these tools is strictly Java focused as target language for code generators. Xtext uses Xpand templates, which are plain text. I don't really know how code generation in Spoofax works. MPS has its base language, which is said to be subset of Java, but there are different alternatives. I personally use Xtext because of its simplicity and maturity, but those strong limitations given by its design make it not a very future proof choice.

Ama

Thanks, that's the kind of comparison I was looking for. I've gone a little farther down the Spoofax path since I asked the question, and the code generation piece of it is pretty compelling: it gives you a powerful functional language to manipulate and transform your ASTs in. So right now that's outweighing the superior community / support of the other tools, at least for my project. – Callisthenics

Answer 2

I have chosen XText in the same case two weeks ago, but I don't know anything about Spoofax. My first impression - Xtext is very simple and productive. I have made my first realife(but very simple) project in 30 minutes, I have generated a graphviz dot graph and html report. I don't like MPS because I prefer plain text source and destination files.

Fachanan

Answer 3

There are other systems for doing this kind of thing. If your goal is building tools, you don't necessarily have to look to an IDE with an integrated tool; sometimes you can find better tools that have focused on utility rather than IDE integration

Consider any of the pure program transformation tools:

• TXL (practical, single paradigm)
• Stratego (Spoofax before it was transplanted into Eclipse)
• Rascal (research, very nicely designed in many ways)
• DMS Software Reengineering Toolkit (happens to be mine; commercial; used to do heavy duty DSL/conventional langauge analysis and transformation including on C++)

These all provide good mechanisms for defining DSLs and transforming them.

What really matters is the support machinery for carrying out "life after parsing".

Exercise

Answer 4

I 've experimented for a couple of days with Xtext and while the tool looks promising I was eventually put off by the tight integration with the Eclipse ecosystem and the pain one has to go through just to solve what should be given hassle-free out of the box: a headless run of the code generator you implemented. See here for some of the minutiae one has to go through (and it's not even properly documented on the Xtext web site but rather on a blog, meaning its an ad-hoc patch that could very well break on the next release).

Will take another look in half a year to see if there has been any improvement on this front.

Horsemanship

Answer 5

Take a look at the Markus Völter's book. It does a very comprehensive comparison of these 3 technologies. http://dslbook.org

XText is very well maintained but this doesn't mean it's problem-less. Getting type-system, scoping and generation running isn't as easy as advertised.

Spoofax is scannerless, (simplifying grammar composition). Not that well documented, but seems complete.

MPS is projectional. A pro for language composition and con for editing. Supports multiple editors for an AST and will soon even support a nice diagram editor. Base language documentation isn't that good. Typesystem, scoping, checking is very well handled. Model to model transformations are done by the solver. My colleagues using it complain about model to text languages. (My opinion M2M wasn't that intuitive either.)

Years ago Microsoft had the OSLO project. MGrammar and especially Quadrant were very promising. It was possible to represent your model in table, form, text or diagram view. But suddenly they've cancelled the project (and perhaps shot the people working on it)

Eliza

Answer 6

Perhaps today the best place to compare different language workbenches is http://www.languageworkbenches.net/ and there http://www.languageworkbenches.net/past-editions/ shows how a set of Language Workbenches implement a similar kind of task: a dsl for a particular domain.

Update 2022: as links were broken and newer articles on the topic are written see the site referred above at: https://web.archive.org/web/20160324201529/http://www.languageworkbenches.net/

References to article reviewing language workbenches include: 1) State of the art: https://link.springer.com/chapter/10.1007/978-3-319-02654-1_11 and 2) Empirical evaluation: https://hal.archives-ouvertes.fr/file/index/docid/706841/filename/Evaluation_of_Modeling_Tools_Adaptation.pdf

Within

Links updated to web.archive.org/web/20160324201529/http://… and references to article reviewing language workbenches is added: 1) State of the art: link.springer.com/chapter/10.1007/978-3-319-02654-1_11 and 2) Empirical evaluation: hal.archives-ouvertes.fr/file/index/docid/706841/filename/… – Within