MEF composition issues, multithreading
Asked Answered
C

2

7

I have a following code:

public class Temp<T, TMetadata>
{
    [ImportMany]
    private IEnumerable<Lazy<T, TMetadata>> plugins;

    public Temp(string path)
    {
        AggregateCatalog aggregateCatalog = new AggregateCatalog();
        aggregateCatalog.Catalogs.Add(new DirectoryCatalog(path));
        CompositionContainer container = new CompositionContainer(aggregateCatalog);
        container.ComposeParts(this);
    }

    public T GetPlugin(Predicate<TMetadata> predicate)
    {
        Lazy<T, TMetadata> pluginInfo;

        try
        {
            pluginInfo = plugins.SingleOrDefault(p => predicate(p.Metadata));
        }
        catch
        {
            // throw some exception
        }

        if (pluginInfo == null)
        {
            // throw some exception
        }

        return Clone(pluginInfo.Value); // -> this produces errors
    }
}

I have a single object of Temp and I call GetPlugin() from multiple threads. Sometimes I face strange composition errors, which I didn't find a way to reproduce. For example:

"System.InvalidOperationException: Stack empty.
    at System.Collections.Generic.Stack`1.Pop()
    at System.ComponentModel.Composition.Hosting.ImportEngine.TrySatisfyImports(PartManager partManager, ComposablePart part, Boolean shouldTrackImports)
    at System.ComponentModel.Composition.Hosting.ImportEngine.SatisfyImports(ComposablePart part)
    at System.ComponentModel.Composition.Hosting.CompositionServices.GetExportedValueFromComposedPart(ImportEngine engine, ComposablePart part, ExportDefinition definition)
    at System.ComponentModel.Composition.Hosting.CatalogExportProvider.GetExportedValue(CatalogPart part, ExportDefinition export, Boolean isSharedPart)
    at System.ComponentModel.Composition.ExportServices.GetCastedExportedValue[T](Export export)
    at System.Lazy`1.CreateValue()
    at System.Lazy`1.LazyInitValue()
    at Temp`2.GetPlugin(Predicate`1 predicate)..."

What could be a reason and how to cure this code?

Cowherb answered 16/1, 2014 at 7:25 Comment(5)
Have you tried using the lock statement in the try block?Chronaxie
@Chronaxie Nope, I can not reproduce the issue so often, so I want to grasp the idea on what's going on and why... I can't just try and see what will happenHartill
I'm guessing race condition.Chronaxie
@Chronaxie I am thinking about that, because errors are different, random and happen inside MEF internals... objects which are not thread-safe often demonstrate this kind of behaviourHartill
Well, you're (potentially) iterating the sequence multiple times, which is generally not a good sign, you're (potentially) calling the predicate multiple times per item, which could be a problem. We'd need to know what is actaully represented by the IEnumerable of lazy's, what the predicate is doing, but most importantly, we need to see how the lazy's are created, because that's where your root problem stems from, is evaluating their value. Without knowing where they come from, we can't possibly know what is wrong with them.Midriff
L
22

The CompositionContainer class has a little-known constructor which accepts an isThreadSafe parameter (which defaults to false for performance reasons). If you'll create your container with this value set to true, I believe your problem will be solved:

CompositionContainer container = new CompositionContainer(aggregateCatalog, true);

On a side note, unrelated to the original question, instead of calling Clone() on the plugin, you can use an export factory instead - this way you don't have to implement your own clone method, as MEF will create a new instance for you.

Loney answered 19/1, 2014 at 9:4 Comment(4)
Upvote for the link to ExportFactory :-), but the isThreadSafe flag on CompositionContainer does pretty much nothing to help the problems with ComposeContainer (at least as far as I've experienced -- I'd be glad to be proved wrong!)Libava
@IainBallard Looks like it helped at least one person. Here's the proof as requested ;)Loney
@AdiLester Yup. Thanks for ExportFactory! It's the first time I am using MEF. But I am not sure yet, that the problem has gone... need some time, because I can not invent a unit test to reproduce the issue. I was trying to call GetPlugin() from 1000 running tasks, but everything was cool even in my poor version.Hartill
@IainBallard You do need to create CompositionContainer in thread-safe mode if you want to access it from multiple threads. MEF components have lots of mutable state inside. In thread-safe mode it turns on synchronization on these points. I have observed myself that without this flag on a web server MEF might serve 1000 requests running in 30 parallel threads without issues, but then something goes wrong and the following 1000 requests fail an all threads. Turning thread-safe mode on resolved the issue. And it's supposed to do that :)Buffer
L
1

If you want to get a list of available Exports for a matching Import type, you don't need to use the (problematic) container.ComposeParts(this);

You can do something more like:

var pluginsAvailable = container.GetExports<T>().Select(y => y.Value).ToArray();

And that will give you an array of available instances, without all the threading issues that plague MEF.

I've been working on something like this today... please excuse the code dump:

using System;
using System.Collections.Generic;
using System.ComponentModel.Composition;
using System.ComponentModel.Composition.Hosting;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Threading;

namespace PluginWatcher
{
    /// <summary>
    /// Watch for changes to a plugin directory for a specific MEF Import type.
    /// <para>Keeps a list of last seen exports and exposes a change event</para>
    /// </summary>
    /// <typeparam name="T">Plugin type. Plugins should contain classes implementing this type and decorated with [Export(typeof(...))]</typeparam>
    public interface IPluginWatcher<T> : IDisposable
    {
        /// <summary>
        /// Available Exports matching type <typeparamref name="T"/> have changed
        /// </summary>
        event EventHandler<PluginsChangedEventArgs<T>> PluginsChanged;

        /// <summary>
        /// Last known Exports matching type <typeparamref name="T"/>.
        /// </summary>
        IEnumerable<T> CurrentlyAvailable { get; }
    }

    /// <summary>
    /// Event arguments relating to a change in available MEF Export types.
    /// </summary>
    public class PluginsChangedEventArgs<T>: EventArgs
    {
        /// <summary>
        /// Last known Exports matching type <typeparamref name="T"/>.
        /// </summary>
        public IEnumerable<T> AvailablePlugins { get; set; }
    }

    /// <summary>
    /// Watch for changes to a plugin directory for a specific MEF Import type.
    /// <para>Keeps a list of last seen exports and exposes a change event</para>
    /// </summary>
    /// <typeparam name="T">Plugin type. Plugins should contain classes implementing this type and decorated with [Export(typeof(...))]</typeparam>
    public class PluginWatcher<T> : IPluginWatcher<T>
    {
        private readonly object _compositionLock = new object();

        private FileSystemWatcher _fsw;
        private DirectoryCatalog _pluginCatalog;
        private CompositionContainer _container;
        private AssemblyCatalog _localCatalog;
        private AggregateCatalog _catalog;

        public event EventHandler<PluginsChangedEventArgs<T>> PluginsChanged;

        protected virtual void OnPluginsChanged()
        {
            var handler = PluginsChanged;
            if (handler != null) handler(this, new PluginsChangedEventArgs<T> { AvailablePlugins = CurrentlyAvailable });
        }

        public PluginWatcher(string pluginDirectory)
        {
            if (!Directory.Exists(pluginDirectory)) throw new Exception("Can't watch \"" + pluginDirectory + "\", might not exist or not enough permissions");

            CurrentlyAvailable = new T[0];
            _fsw = new FileSystemWatcher(pluginDirectory, "*.dll");
            SetupFileWatcher();

            try
            {
                _pluginCatalog = new DirectoryCatalog(pluginDirectory);
                _localCatalog = new AssemblyCatalog(Assembly.GetExecutingAssembly());
                _catalog = new AggregateCatalog();
                _catalog.Catalogs.Add(_localCatalog);
                _catalog.Catalogs.Add(_pluginCatalog);
                _container = new CompositionContainer(_catalog, false);
                _container.ExportsChanged += ExportsChanged;
            }
            catch
            {
                Dispose(true);
                throw;
            }

            ReadLoadedPlugins();
        }

        private void SetupFileWatcher()
        {
            _fsw.NotifyFilter = NotifyFilters.Attributes | NotifyFilters.CreationTime | NotifyFilters.FileName |
                                NotifyFilters.LastAccess | NotifyFilters.LastWrite    | NotifyFilters.Size     | NotifyFilters.Security;

            _fsw.Changed += FileAddedOrRemoved;
            _fsw.Created += FileAddedOrRemoved;
            _fsw.Deleted += FileAddedOrRemoved;
            _fsw.Renamed += FileRenamed;

            _fsw.EnableRaisingEvents = true;
        }

        private void ExportsChanged(object sender, ExportsChangeEventArgs e)
        {
            lock (_compositionLock)
            {
                if (e.AddedExports.Any() || e.RemovedExports.Any()) ReadLoadedPlugins();
            }
        }

        private void ReadLoadedPlugins()
        {
            CurrentlyAvailable = _container.GetExports<T>().Select(y => y.Value).ToArray();
            OnPluginsChanged();
        }

        private void FileRenamed(object sender, RenamedEventArgs e)
        {
            RefreshPlugins();
        }

        void FileAddedOrRemoved(object sender, FileSystemEventArgs e)
        {
            RefreshPlugins();
        }

        private void RefreshPlugins()
        {
            try
            {
                var cat = _pluginCatalog;
                if (cat == null) { return; }
                lock (_compositionLock)
                {
                    cat.Refresh();
                }
            }
            catch (ChangeRejectedException rejex)
            {
                Console.WriteLine("Could not update plugins: " + rejex.Message);
            }
        }

        public IEnumerable<T> CurrentlyAvailable { get; protected set; }

        ~PluginWatcher()
        {
            Dispose(true);
        }
        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
        protected void Dispose(bool disposing)
        {
            if (!disposing) return;

            var fsw = Interlocked.Exchange(ref _fsw, null);
            if (fsw != null) fsw.Dispose();

            var plg = Interlocked.Exchange(ref _pluginCatalog, null);
            if (plg != null) plg.Dispose();

            var con = Interlocked.Exchange(ref _container, null);
            if (con != null) con.Dispose();

            var loc = Interlocked.Exchange(ref _localCatalog, null);
            if (loc != null) loc.Dispose();

            var cat = Interlocked.Exchange(ref _catalog, null);
            if (cat != null) cat.Dispose();
        }
    }
}
Libava answered 20/1, 2014 at 15:22 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.