How do I get started designing and implementing a script interface for my .NET application?

Question

How do I get started designing and implementing a script interface for my .NET application?

There is VSTA (the .NET equivalent of VBA for COM), but as far as I understand I would have to pay a license fee for every installation of my application. It is an open source the application so this will not work.

There is also e.g. the embedding of interpreters (IronPython?), but I don't understand how this would allow exposing an "object model" (see below) to external (or internal) scripts.

Sub-questions:

• What is the scripting interface story in .NET? Is it somehow trivial to do this in .NET?
• E.g. can some .NET objects in my application and their contained objects be declared to be accessible from the outside at runtime?
• How can external scripts access my application (through the object model)?

---

Background:

I have once designed and implemented a fairly involved script interface for a Macintosh application for acquisition and analysis of data from a mass spectrometer (Mac OS, System 7) and later a COM interface for a Windows application.

Both were designed with an "object model" and classes (that can have properties). These are overloaded words, but in a scripting interface context object model is essentially a containment hierarchy of objects of specific classes. Classes have properties, lists of contained objects and are not only data but can have verbs as well (actions/methods). E.g. in the Macintosh case the defined application object can contain an acquisition object that has properties for voltages used in the instrument and a fireLater verb - all as seen from the external script.

Note that in both cases the classes/objects in the programming language used to implement the application had nothing to do with the scripting object model. For the Macintosh case, the mechanisms used to implement the scripting interface was defined by Apple. There were also some standards defined by Apple on how to design the object model. For instance, standardized names for certain common properties in classes.

Or like in the COM interfaces exposed in Microsoft Office applications, where the application object can be used to add to its list of documents (with the side effect of creating the GUI representation of a document).

External scripts can create new objects in a container and navigate through the content of the hierarchy at any given time. In the Macintosh, case scripts could be written in e.g. AppleScript or Frontier.

On the Macintosh, the implementation of a scripting interface was very complicated. Support for it in Metroworks' C++ class library (the name escapes me right now) made it much simpler.

Lowpitched

Possible duplicate of stackoverflow.com/questions/137933 – Stapleton

I don't think it is a duplicate. The primary question is about the scripting interface story on .NET (I have failed to find any) and how to design and implement object models (in the scripting interface sense of the term). Embedding may be part of an answer, but it is not the question. "scripting interface" is not the same as "embedding a scripting language" even though the word "scripting" appears in both. – Lowpitched

It would help if you explained in more depth what you think the difference is. Are you seeking to put a scripting language in your program that can manipulate (some of?) the objects of the program? – Applause

Good Google-fu: "Making .NET applications scriptable". Some results: Scripting for .NET (Alex Angelopoulos), Using .NET to make your application scriptable (5 parts, directly addresses the "object model") and Make your .NET application support scripting - a practical approach (Roy Osherove). – Lowpitched

Related: Adding scripting functionality to .NET applications – Lowpitched

Answer 1

[EDIT: As covered at length in the comments in this, assuming you have a significant need to enable internal scripting where you are hosting snippets or functions someone gives you to customise your app, as opposed to a purely external scenario where one is providing a facaade to allow people to dig stuff out of your app on a more rigid predefined basis]

IronRuby and IronPython are very neat and appropriate for this (but as the other answer says, PowerShell may be appropriate if you have a more infrastructure-type thing).

EDIT: Other ideas for enabling internal scripting are

• using Windows Workflow Foundation (exposing activities to it and/or hosting instances of workflows)
• using Spring.NET's Expression Language (which is terse, easy to doc and learn but surprisingly powerful)

EDIT 2 June 2011: IronJS may also be a suitable candidate, there's a Hanselminutes that talks it thru.

Vizzone

How would an external IronPython script be able to control my application? What is the mechanism? I understand that a script could be embedded inside my application, be executed and have access to all of (or perhaps a subset of) the objects in my application at runtime. But what about access from another process or even over a network? (this worked with Macintosh/AppleScript and Windows COM). – Lowpitched

@Peter Mortensen: Thanks for following up. Before we start, let me confess to only scannign your question - good job you followed up as without the discussion to follow this answer will be very misleadiong. Will read now. I had thought there was an element of you running and/or extracting / customising /processing stuff within your application. – Vizzone

If you're trying to support scenarios where your app exposes stuff that people outside want to pull out (but not push in and/or customise in complex ways including e.g. custom rules) including over a network then exposing stuff to PowerShell either as either .NET objects or cmdlets [in snapins or modules] is the way to go. If you're exposing a tree of stuff then a PowerShell provider may be appropriate). Now having spouted generalities, I'm off to read the question in full! – Vizzone

After re-reading, not sure I misread it all that badly first time. Botom line is that C# is a good lang to expose static stuff to external people and PowerShell cmdlets (which one would implemnet in C# or PS) and/or raw PowerShell are a good way of consuming such an interface (including using PowerShell remoting to do the cross-box stuff). – Vizzone

@Ruben Bartelink (to the 1st comment): no problem, take your time. That is what the bounty is for. I researched the scripting interface story on .NET some time ago and also found the embedding of scripts. But no references to scripting interfaces or "object models". Perhaps I have misunderstood the whole thing, but I do have several years of experience designing and implementing complex scripting interfaces for real-world applications based on "object models". The interface for Mac was used by a script to implement a Fuzzy Logic based control of a laser during data acquisition. Paper published – Lowpitched

@Peter mortensen: If you want to enable internal customisation/scripting by running arbitrary stuff supplied by third parties in the context of your app, then hosting PS code via the PS hosting APIs is viable and offers an element of dynamic/interpreted code. If you are doing both internal an external, then PS is a good choice as it handles both cases. But IPy/IR is the one that is optimised for internal scripting. PS as a lang is optimal for external scripting but can support internal. – Vizzone

When exposing to external users :- For exposing static stuff from your app to a mix of PS local/remote clients, expose C# classes and some wrapper cmdlets. For exposing static stuff from your app to a mix of PS/C#/dynamic clients one would use C# with no cmdlets. For exposing stuff from your app to dynamic clients such as IR/IPy ONLY you could use C# OR IPy/IR. – Vizzone

When inviting people into your .NET app to customise things (ONLY), first choice is IPy/IR, and PS works. – Vizzone

PS is the optimal choice for internal scripting a) when most of your user community will already i) need to understand PS to understand your external and hence are guaranteed to need/understand PS OR b) are already guaranteed to understand PS – Vizzone

Another thing is the fact that using PS for internal scripting builds a dep on PS being installed on the box running your app (safe for W7 and 2008R2 and later but cant be i) assumed or ii) embedded into an installer for anything earlier). For IR/IPy, you just copy local / xcopy deploy the DLR DLL. Also, the script execution policy for PS may cause problems if locked down with Group Policy (not 100% on details of that though). On the plus side for PS, PS hosting offers a way to constrain what scripts can do by whitelisting stuff - dont think IR/IPy can do that. – Vizzone

@Peter Mortenson in resp to your comment I just saw now (switching out of write-only mode!). You need to expose an object model (real C# classes) for PS or static clients for them to be able to externally poke at your app. For scripts you host within you, if your scripting lang is static (C# or PS) then you also need to do that - you could pass the script instances of objects you create that are not public. If you are hosting IR or IPy then the stuff you expose can be internal dynamic classes, which may or may not be appropriate. – Vizzone

@Ruben Bartelink: thanks. This is exactly the kind of considerations/information I am looking for. I think I will try to mock something up to see how it actually works. – Lowpitched

Hey, thanks for the bounty - you have great taste :P I/the world'd be interested in seeing a summary edit at the end of your post which summarises very briefly what you decided to do in terms of the discussion here - i.e., are you doing internal + external? Did you decide there was benefit in supporting or leveraging dynamic languages? What did you decide in terms of how dependencies influence your decisionmaking? (I've participated in 4 bty'd questions and won 3 but 2 have been excruciating experiences- for me this question is a model of what the bty system can be - thxfr restoring my faith!) – Vizzone

Answer 2

Take a look at PowerShell.

It allows you to program simple Cmdlets that are scriptable. It also supports hierachical containers (by default you have the file system, the Registry, certificates store, etc.).

Autum

But where would I define the object model for my application? Or is it somehow called into existence automatically? – Lowpitched

The object model is a set of cmdlets. As parameters (and return value) they can accept .NET objects, so you're able to 'pipe' objects from cmdlet to cmdlet. There is no object model in sense of DOM. – Autum

I don't understand how PowerShell can get access to the internal state of a running .NET application. Are all the .NET objects in a running .NET application exposed to the outside by default? I am confused. – Lowpitched

@Peter: I'm not sure about getting internal state of an application, but you can call .NET classes from PowerShell. Have a look at this page: msdn.microsoft.com/en-us/magazine/cc163430.aspx ...While it's a bit long-winded, it does demonstrate what is possible with PowerShell. – Scrutable

@Peter Mortensen: You'd have public classes in your app that get created (via the equiavalent of new) in PS which would need to then talk to your running app via either a) loading the same data your app does - a parallell instance of a subset of your app that would run it in the PS AppDomain, i.e., you expose a subset of your app in the same way that you'd ofer a command-line version of a GUI app or b) taking to your actual app in its AppDomain via .NET Remoting etc., i.e., you implement a class which is a proxy into your app - i.e., same as how you'd expose a admin interface to a WCF host – Vizzone

Answer 3

I would suggest you consider the Dynamic Language Runtime as a strategy for implementing a scripting interface. Details:

• CodePlex DLR site
• MSDN .NET 4 DLR Overview

The DLR currently supports IronPython and IronRuby, plus a number of other language implementations can be found on CodePlex. If you are interested in creating a language specific to your application Bitwise Magazine has a good primer article.

Dino Viehland's PDC09 session Using Dynamic Languages to Build Scriptable Applications is worth watching. You can find the demo code here.

Vasileior

Answer 4

The Lua scripting language is free, used in a large number of commercial applications, and is easily embeddable into a .NET application using the freely-available LuaInterface library, which permits you to expose types and methods from your application that scripts in your embedded interpreter can leverage.

A tutorial on how to embed Lua into a C# application can be found here.

Edit: It's probably also worth noting that Lua was designed from the ground up to be an embedded scripting language, and as a result, the interpreter is highly customizable. The host application can restrict almost any aspect of the interpretation capabilities as part of the security model; e.g. allowing or preventing scripts from making network connections or writing to files, etc.

Also, you asked about external scripts. Making your program available to out-of-process scripts is done in the same way you make it available to out-of-process applications: by exposing a standardized automation interface through some sort of communication protocol. On Windows, for same-machine cross-process communication, that will most commonly be COM, but it could also be WCF, TCP remoting, RPC or any number of other communications standards. What you choose to do depends heavily on how your application is built and what kind of external automation you intend for it to do.

Heptarchy

You forgot to mention that Lua is used widely in the gaming industry and most importantly, it is FAST! It was a serious contended for my project but was rejected for the pure fact that CS-Script allowed use to use VB. If speed were a concern I would opted for Lua. – Homeroom

Answer 5

For a free, easy to implement .NET scripting language, have a look at C#.

See my answer to a similar question.

As for exposing data, seeing how it's a .NET language, you just need to add your program into the assemblies to link into and make the relevant classes public.

Stapleton

@Grant Peters: Could the object model then consist of a set of facade .NET classes that exposes the required properties and methods? ("Facade" possibly in the GoF sense.) – Lowpitched

I wouldn't characterize C# as a scripting language, at least not according to how it is normally used. For one thing, it has a separate compilation step. – Applause

@Donal, quite a few scripting languages these days have a seperate compilation step as well (in fact, I can't think of any that don't, though many do use JIT compiling, but then cache the compiled code). – Stapleton

@Peter, through c# and .Net, they can have direct access to your classes. You don't need to do anything besides make them public and pass instances to functions in the "script" (or, if using globals/statics, the script can access those directly). – Stapleton

Yes, but they typically hide the fact that they're doing this from view; the compilation is just a part of the implementation. (I know of languages which do compilation but which never save the object code as the compiler is fast enough to beat typical HDD speeds.) – Applause

Answer 6

I used CS-Script to create something like you want. In my case I defined an interface in my application. A script then just needed to implement this interface so that it could be run from the application. My application was about processing scanned images and therefore my interface looked something like this:

public interface ICustomModule
{
    void ProcessBatch(IBatch batch)
}

This way my script could access the object model that I defined in my application (in my case through IBatch). The good thing was that during development I could use a normal Class Library project for the script: IntelliSense, debugging... I do not remember the exact details but I basically had a switch in my application that told the app to use the referenced class library instead of a script.

Furthermore I made an additional interface that allowed the scripts to be configured: A script could define a set of properties which were then displayed by my application in a property grid. I think I used this version here as this seemed a bit more flexible at the time. This way you can not only allow users to configure the script, but you can also provide help in form of descriptions, choices, default values... The property grid is quite extensible: In one case we had a script that would display a special form to define some complex settings.

Edit: The application of course did not have a reference to "debug" class library. It just need to load the assembly...

Alyse

@Stefan Egli: CS-Script looks very promising and it is very well documented. For instance "Simplified Hosting Model", csscript.net/help/simplified_hosting.html and in csscript.net, "CS-Script can be hosted in NAnt environment. You can execute of both script files and embedded C# code (CDATA) from NAnt build scripts.". Reading the documentation I realise that "object model" in the scripting interface sense of AppleScript and COM may not exist. – Lowpitched

I am not 100% sure if I understand you correctly when you refer to "object model". The solution I propose allows you define Interfaces (or classes) with methods and properties that the script can invoke to call back into your application. In my example I could use the IBatch Reference to check how many images I have and get their paths. Furthermore I could invoke methods like deleting an image which was handled by my application (batch.DeleteImage()). I think you are very free to do what you want. It is also up to you from where you store the scripts (could be a database)... – Alyse

see also this part of the documentation: csscript.net/help/Type_sharing_patern.html – Alyse

@Stefan Egli: (1/2) with AppleScript and COM the definition of classes (in the scripting interface) was separated from classes in the programming language used in the application. It was also a way to define (in a static sense, before any script was executed) a containment hiarchy of objects/classes. For AppleScript applications declared to the outside (in the resource fork, as data - unqiue 4 byte codes for classes, etc.) their "object model" - the set of classes, some of which could have lists of other classes (in effect defining the containment hiarchy). – Lowpitched

@Stefan Egli: (2/2) It may be that the separation does not exist in .NET and that is what confuses me. That is the main question. – Lowpitched

I think I know what you mean. With CS-Script you do not have this kind of separation: Interfaces define what your scripts can do and interfaces need to be defined in an assembly (using c# or some other .net language). CS-Script is a scripting engine in the sense that you can let it process c# source code directly. Behind the scenes the c# script is compiled into an assembly (it is not interpreted). Therefore is not really different then any class library you would write and use as a dll (this of course results in very good performance of the scripts). – Alyse

Answer 7

I'm not sure this will cover your needs, but via reflection you can compile C# code and execute it at runtime (example code here).

So you could write the script in eg. C# and then compile "it on the fly" and run it directly in the context of your application. Of course you have to keep security considerations in mind, but if the scripts are trusted then it might work for you, and you get the benefits of using a powerful managed language for your scripts.

If you need high performance or running thousands of scripts it might be too slow though.

Durr

Answer 8

I implemented CS-Script as the scripting platform last for a workflow system I wrote. Each wrokflow needed had different conditions that dictated what users would subscribe to various tasks and who would receive emails. With a scripting model it became easy to introduce new steps into the workflows and handle unique requirements that those tasks required.

The other nice by product of the scripting model was that workflows could be tested under a variety of conditions, and an iterative approach could be taken to finalize the workflow behaviors. During QA and user acceptance testing I included logging functions in the scripts to that I could hunt issues more easily.

With CS-Script you have complete access to you objects; that is, when your script imports your assemblies you can instantiate your object in your script code. Furthermore, you can retain your compiled scripts and simply supply parameters to them. If your assemblies use a parameters object or a Dictionary, you can pass that your script and execute methods on the objects contained in the parameter object.

Homeroom