Textual Domain-Specific language (DSL) development with Microsoft Visual Studio

Question

I did some searches on developing a DSL in visual studio. At the beginning, I found out there is a Visualization and Modeling SDK for VS2010. It has a tool called DSL tool. But it seems that it is only for graphical DSL development.

Then I saw some posts saying "Oslo" is a tool for developing textual DSL, which "was" a Microsoft product - Microsoft no longer supports the tool. http://blogs.msdn.com/b/modelcitizen/archive/2010/09/22/update-on-sql-server-modeling-ctp-repository-modeling-services-quot-quadrant-quot-and-quot-m-quot.aspx

Therefore, I would like to know if I want to develop a textual DSL, what tool is the best? What do you think if I implement a DSL parser making use of F# powerpack with FSLex and FSYacc?

Seal

Update: Finally I used an open source tool - Irony (irony.codeplex.com) - for the implementation of the parser, but it seems that the author is not going to further develop or maintain the tool. There is no DSL workbench like Xtext (xtext.org) in Eclipse for visual studio right now, but somebody is developing such kind of tool (metasharp.codeplex.com). – Seal

Answer 1

I am currently developing several external text-based DSLs using FsLex/FsYacc. I was using a hand parser, but I find the FsLex/FsYacc much easier to maintain in the design stage.

FsLex/FsYacc are not as sophisticated as ANTLR, but since most DSLs are fairly simple, FsLex/FsYacc are a perfectly sound choice for use within Visual Studio. And keeping DSLs simple is a good thing, since they are intended to be restricted and simple to learn.

I find Martin Fowler's book to be a good resource, less for the examples and details than as an encyclopedia of DSL ideas. His discussion of useability and other design aspects of DSLs is also worth reading. As Toumas indicated, it does not cover either F# or functional languages. Mr. Fowler writes that he lacked the experience in those subjects to bring the book to market in a timely way.

Having praised FsLex/FsYacc, I do still wish someone would write a good ANTLR back-end for F#. :)

-Neil

Sudduth

Answer 2

I am a fan of embedded DSLs, a la

http://lorgonblog.wordpress.com/2010/04/15/using-vs2010-to-edit-f-source-code-and-a-little-logo-edsl/

http://lorgonblog.wordpress.com/2010/04/16/fun-with-turtle-graphics-in-f/

where you just use leverage F# syntax with some good function names and possibly other syntax cleverness (lists, workflows, ...) to get code that "looks like maybe it is another language" but is actually just F#.

But yes, for external DSLs, you just need a grammar/parser/etc tool chain, and either FsLex/FsYacc, or maybe ANTLR or FParsec are various choices. (I don't have enough experience with any of these to know trade-offs among them.)

Nationwide

Answer 3

Since having made my earlier post, I have also bought and read parts of Terence Parr’s book “Language Implementation Patterns.” It is excellent, though quite a bit more technical than Martin Fowler’s book (with some additional material it could be a “Dragon Book” for the new millennium). The examples are strongly based in Java and ANTLR, but the text is the main thing, so the book is useful regardless of one’s language development environment.

Interestingly, there is little overlap between the two books. Martin Fowler’s book does a good job of covering the design and implementation of basic DSLs, such as those used for specification and configuration, while Terence Parr’s book is more technical and covers the realm extending all the way up through more sophisticated languages and byte-code machines. I recommend both if you can budget for them, otherwise, either is an excellent choice within its given domain.

Sudduth

Answer 4

Martin Fowler has a new book about DSL:s. Sadly, it won't discuss much about Microsoft's tooling nor functional languages.

Microsoft no longer support the graphical tool "Quadrant", but MGrammar is still supported and integrated to SQL server, right? MGrammar is the "DSL making language".

Still, I would say that functional languages (read: F#) are the way to go.

This book has a simple example of how to make a DSL with F#: http://www.manning.com/petricek/ and also Google finds many other good references about this topic.

Roar

I no longer see "SQL Server Modeling Services" mentioned in Books Online. Can you provide a link to Microsoft documentation on it (not blog posts). – Yonita

Sorry, I haven't seen books about SQL Server modeling. Just blogs like Douglas Purdy and Bart de Smet. – Roar

Answer 5

Try MBase, but it only worth using if your DSL is complicated enough to require an efficient compiler and a PEG grammar. Otherwise FsYacc is more than enough.

Postrider

Answer 6

Our DMS Software Reengineering Toolkit is designed to handle arbitrary DSLs (I happen to be the architect).

Most people think if you have a parser you have enough, and it is technically true, in the same sense that if you have transistors you can build a computer.

In my experience you want a lot more than just a parser: you need ways to build symbol tables so that your generator knows what the meaning of a particular identifier is, means to analyze the specification, ways to easily encode your translation and to apply optimizations to generated results.

DMS provides all these capabilities to support building DSLs. And in that sense, it goes much beyond F#.

Privett

@Ira: still at it, I see. How about editing your answer to say, "I develop and sell a commercial product (DMS Software Reengineering Toolkit) that is designed to handle arbitrary DSLs"? Don't beat around the bush: tell us that it's your product, that you make money from it, then please do proceed to tell us how it will solve the problem posed by the question. You'll get upvotes from me on answers like that (I believe I've upvote you twice, BTW). – Yonita

meta.stackexchange.com/questions/57497/…. What, it isn't clear that I'm associated with this? (Normally I follow the recommended policy and say "Our....", I varied slightly this time). And, OP asks, "What tool is the best?". OK, I'll stick with suggested means. – Privett

@Ira: I remember that discussion. I don't care. I just gave you my criteria. I'm wondering what's wrong with you that you won't simply and plainly say that you're selling your product and that your product solves the problem and "here's how". Maybe it's just your manner of speech, but you remind me of a man attempting to avoid the issue. Since the issue is so simple, you really raise my hackles. – Yonita

@Ira: here's a thought. For this question only, why not edit your answer to be explicit? See if the world comes to an end. I bet it doesn't. If you sell a good product that solves the problem, then I think that being straightforward will actually sell more product. – Yonita

@John: I remember the discussion too, and I know you don't care that a policy suggestion was made that you didn't like. Are we going to do this all over again? – Privett

@Ira: I also want to encourage you (again) to detail and give examples of how your products solve these problems. This is for my benefit as well - I have repeatedly come close to looking into your products, and more examples are more likely to get me to do that. Where I'm working now, we could for sure use some reengineering! – Yonita

@John: Now, here maybe I do have a stye problem. The response above has a pithy description of what you need, and link to a much longer, more detailed explanation, which I chose not to repeat because I think that explanation is fine. Did you follow the link? – Privett

@John: I often try to produce more detailed answers; see #5323593 – Privett

@John: And if you do have a "reengineering" question, you could always post it. Who knows who might answer? – Privett

@Ira: I did follow the link, and read it. I'll be reading it again later to see how it applies to my current work situations. I'm not bothered by style differences in how you show that your product solves the problem - it happens that I would be more explicit about it. I'm more concerned with the fact that it appears that you're shy about the fact that your product is commercial. I'd proudly say so, then explicitly show how the product solves the problem. It bothers me that you don't just say up front and proudly, "this is my product, and I hope you buy it" or something like that. – Yonita

@Ira: I have now upvoted three of your answers. And when I get a specific reengineering question, I'll be sure to post it. – Yonita