Fire-and-forget approach on MEF plugin architecture

Question

This question might be design related or code related, but I'm stuck so I'm open to any kind of answer; a pointer in the right way!

I have used MEF (Managed Extensibility Framework) to develop a piece of WPF software that will act as a form of orchestrator for plugins. The application is simply redirecting data between the plugins as of the users choice, so what the plugin does is not known at all (especially since they can be developed by 3rd party devs). The application and the plugin are sharing an interface as a way of knowing what methods to call for, so the traffic goes both ways: a plugin calls a method in the main application sending it data and the main application is passing this data to another plugin.

This works so far, but I'm having a problem with synchronous behavior. All methods defined by the interface lack a return value (Void) and I'm struggling to get a "fire and forget" kind of approach where the calling application does NOT need to sit around waiting for the plugins receiving function to finish execute code (and calls that goes back to the main app!).

So whats the best approach to solving this? Letting every plugin (and the main app) put it's workload on a "stack" of some kind just to be able to return the control to the calling side and then have some mechanism that runs separately that works through the stack item by item (and do this stacking approach as async?)?

Other things worth noting is that the plugins are running in separate threads (according to the debugger thread window) and when they are initialized they get a reference from the calling main application so they can fire functions in the main app. The plugins also very often need to tell the main app what status they are in (idle, working, error etc) and also send data to be logged by the main app, so this very often creates a nested call hierarchy (if you follow me, hard to explain).

I'm using .Net 4.5 for this one.

Below is some simplified example of the code. I replaced some names, so if there is a spelling error somewhere, its just here and not in the real code. :)

The interface:

public interface IMyPluggableApp 
{
    void PluginStatus(string PluginInstanceGuid, PluginInstanceState PluginInstanceState);
    void DataReceiver(string PluginInstanceGuid, string ConnectorGuid, object Data);
    void Logg(string PluginInstanceGuid, LoggMessageType MessageType, string Message);
}

public interface IPluginExport
{
    PluginInfo PluginInfo { get; set; }
    void Initialize(string PluginInstanceGuid, Dictionary<string, string> PluginUserSettings, IMyPluggableApp MyPluggableApp);
    void Start(string PluginInstanceGuid, List<ConnectorInstanceInfo> ConnectedOutputs);
    void Stop(string PluginInstanceGuid);
    void PluginClick(string PluginInstanceGuid);
    void PlugginTrigger(string ConnectorGuid, object Data);
}

The plugin:

    public static IMyPluggableApp _MyPluggableApp

[PartCreationPolicy(CreationPolicy.NonShared)]
[Export(typeof(IPluginExport))]
public class PluginExport : IPluginExport
{
     public void Initialize(string PluginInstanceGuid, Dictionary<string, string> pluginUserSettings, IMyPluggableApp refMyPluggableApp)
    {
        _MyPluggableApp = refMyPluggableApp; // Populate global object with a ref to the calling application

        // some code for setting saved user preferences

        _MyPluggableApp.PluginStatus(PluginInfo.PluginInstanceGuid, PluginInstanceState.Initialized); // Tell main app we're initialized
    }
    public void Start(string PluginInstanceGuid, List<ConnectorInstanceInfo> ConnectedOutputs)
    {
        // Some code for preparing the plugin functionality

        _MyPluggableApp.PluginStatus(PluginInfo.PluginInstanceGuid, PluginInstanceState.Initialized); // Tell main app we started
    }
    public void PlugginTrigger(string ConnectorGuid, object Data)
    {
        _MyPluggableApp.PluginStatus(AvailablePlugins.PluginInfo.PluginInstanceGuid, PluginInstanceState.Running_Busy); // Tell main app we're busy

                // Run the code that actually provides the functionality of this plugin

        _MyPluggableApp.PluginStatus(AvailablePlugins.PluginInfo.PluginInstanceGuid, PluginInstanceState.Running_Idle); // Tell main app we're idle
    }

    // and so on ...
}

And the main application:

public partial class MainWindow : IMyPluggableApp 
    {
        [ImportMany(typeof(IPluginExport))]
        IPluginExport[] _availablePlugins;

        public void PluginStatus(string PluginInstanceGuid, PluginInstanceState PluginInstanceState)
        {
                    // Code for setting status in GUI
        }

        public void DataReceiver(string PluginInstanceGuid, string ConnectorGuid, object Data)
        {
                    ConnectorInfo connector_source = GetConnectorInfo(ConnectorGuid);
                    PluginInfo plugin_source = GetPluginInfo_ByPluginInstanceGuid(PluginInstanceGuid);

                        ConnectorInstanceInfo connector_destination = (from i in _project.PluginInstances
                                    from y in i.ConnectedConnectors
                                    where i.PluginInstanceGuid == PluginInstanceGuid
                                    && y.ConnectedFromOutput_ConnectorGuid == ConnectorGuid
                            select y).FirstOrDefault();

                        _availablePlugins.Where(xx => xx.PluginInfo.PluginInstanceGuid == connector_destination.ConnectedToInput_PluginInstanceGuid).First().PlugginTrigger(ConnectorGuid, Data);
        }

        public void Logg(string PluginInstanceGuid, LoggMessageType MessageType, string Message)
        {
                    // Logg stuff
          }
}

It's the DataReceiver function in the main app thats receives the data, looks what plugin should have it, and then sends it (via PlugginTrigger function).

Bisectrix

Would you be able to post a simplified version of your plugin interface, and an example of how the application currently calls a method on one of the plugins? – Doorsill

Answer 1

A couple of observations:

• Fire and forget is a requirement of the host so not something the plug-in implementations should have to worry about.
• I don't think (please correct me if I am wrong) the CLR supports calling methods in a "fire-and-forget"-ful way within the same AppDomain. If your plug-ins were loaded in to separate processes, and you were communicating with them using WCF then you could simply set the IsOneWay property on your OperationContractAttribute.

The second point suggests one solution, which seems slight overkill for your situation - but let us mention it anyway. Your plug-ins could host in-process WCF services, and all the communication between the WPF application and the plug-ins could be done through the WCF service proxies. However, this comes with a configuration nightmare and is really opening a can of worms to a whole bunch of other issues you would have to solve.

Let us start with a simple example of the initial problem, and attempt to solve it from there. Here is the code for Console application with a plug-in:

public class Program
{
    private static void Main(string[] args)
    {
        var host = new CompositionHost();
        new CompositionContainer(new AssemblyCatalog(typeof(Plugin).Assembly)).ComposeParts(host);
        var plugin = host.Plugin;
        plugin.Method();
        Console.ReadLine();

    }

    private class CompositionHost: IPartImportsSatisfiedNotification
    {
        [Import(typeof (IPlugin))] private IPlugin _plugin;

        public IPlugin Plugin { get; private set; }

        public void OnImportsSatisfied()
        {
            Plugin = _plugin;
        }
    }
}

public interface IPlugin
{
    void Method();
}

[Export(typeof(IPlugin))]
public class Plugin : IPlugin
{
    public void Method()
    {
        //Method Blocks
        Thread.Sleep(5000);
    }
}

The problem is the call to plugin.Method() is blocking. To solve this, we change the interface that is exposed to the Console application to the following:

public interface IAsyncPlugin
{
    Task Method();
}

A call to an implementation of this interface will not block. The only thing we need to change is the CompositionHost class:

    private class CompositionHost: IPartImportsSatisfiedNotification
    {
        [Import(typeof (IPlugin))] private IPlugin _plugin;

        public IAsyncPlugin Plugin { get; private set; }

        public void OnImportsSatisfied()
        {
            Plugin = new AsyncPlugin(_plugin);
        }

        private sealed class AsyncPlugin : IAsyncPlugin
        {
            private readonly IPlugin _plugin;

            public AsyncPlugin(IPlugin plugin)
            {
                _plugin = plugin;
            }

            public Task Method()
            {
                return Task.Factory.StartNew(() => _plugin.Method());
            }
        }
    }
}

Obviously this is a very simple example, and the implementation may have to vary slightly when applying it to your WPF scenario - but the general concept should still work.

Reckoning

Nice! I would not consider myself a super programmer in any way, and maybe thats why I dont see how I would fit your CompositionHost class in my solution since I'm using another way of composing (using DirectoryCatalog). But thats my fault. But I did try the Task.Factory.StartNew -way of calling the methods and that made the trick, at least it seems so! But am I safe not to include the rest of your proposed solution? Or maybe I should first try with a stress test of some kind (with many pluggins cCalling each other) to see if it "seems" ok? – Bisectrix

I think you could "Export" the CompositionHost class within your current MEF framework, and import this instance in to your application to consume your plugins. You certainly don't have to include the rest of my solution - whatever you find useful. You can't go wrong with a bit of testing... – Reckoning