How can I implement my own type of extern?

Question

In our product, we have things called "services" which are the basic means of communication between different parts of the product (and especially between languages—an in-house language, C, Python and .NET).

At present, code is like this (Services.Execute utilising params object[] args):

myString = (string)Services.Execute("service_name", arg1, arg2, ...);

I'd rather like to be able to write code like this and get the benefits of type checking and less verbose code:

myString = ServiceName(arg1, arg2, ...);

This can be achieved with a simple function,

public static string ServiceName(int arg1, Entity arg2, ...)
{
    return (string)Services.Execute("service_name", arg1, arg2, ...);
}

But this is rather verbose, and not quite so easy to manage when doing it for scores of services, as I intend to be doing.

Seeing how extern and the DllImportAttribute work, I hope it should be possible to hook this up by some means like this:

[ServiceImport("service_name")]
public static extern string ServiceName(int arg1, Entity arg2, ...);

But I don't know how to achieve this at all and can't seem to find any documentation for it (extern seems to be a fairly vaguely defined matter). The closest I've found is a somewhat related question, How to provide custom implementation for extern methods in .NET? which didn't really answer my question and is somewhat different, anyway. The C# Language Specification (especially, in version 4.0, section 10.6.7, External methods) doesn't help.

So, I want to provide a custom implementation of external methods; can this be achieved? And if so, how?

Envision

Most common way to do this is with interfaces and remoting proxies. – Hornwort

See #7246007 – Pm

@PaulZahra: I have seen that question - I referred to it in my question – Envision

Answer 1

The C# extern keyword does very little, it just tells the compiler that the method declaration won't have a body. The compiler does a minimum check, it insists that you provide an attribute as well, anything goes. So this sample code will compile just fine:

   class Program {
        static void Main(string[] args) {
            foo();
        }

        class FooBar : Attribute { }

        [FooBar]
        static extern void foo();
    }

But of course it will not run, the jitter throws its hands up at the declaration. Which is what is required to actually run this code, it is the jitter's job to generate proper executable code for this. What is required is that the jitter recognizes the attribute.

You can see this done in the source code for the jitter in the SSCLI20 distribution, clr/src/md/compiler/custattr.cpp source code file, RegMeta::_HandleKnownCustomAttribute() function. That's code that's accurate for .NET 2.0, I'm not aware of additions to it that affect method calling. You'll see it handling the following attributes that relate to code generation for method calls, the kind that will use the extern keyword:

• [DllImport], you no doubt know it

• [MethodImpl(MethodImplOptions.InternalCall)], an attribute that's used on methods that are implemented in the CLR instead of the framework. They are written in C++, the CLR has an internal table that links to the C++ function. A canonical example is the Math.Pow() method, I described the implementation details in this answer. The table is not otherwise extensible, it is hard-baked in the CLR source code

• [ComImport], an attribute that marks an interface as implemented elsewhere, invariably in a COM server. You rarely program this attribute directly, you'd use the interop library that's generated by Tlbimp.exe instead. This attribute also requires the [Guid] attribute to give the required guid of the interface. This is otherwise similar to the [DllImport] attribute, it generates a pinvoke kind of call to unmanaged code but using COM calling conventions. This can of course only work properly if you actually have the required COM server on your machine, it is otherwise infinitely extensible.

A bunch more attributes are recognized in this function but they don't otherwise relate to calling code that's defined elsewhere.

So unless you write your own jitter, using extern isn't a viable way to get what you want. You could consider the Mono project if you want to pursue this anyway.

Common extensibility solutions that are pure managed are the largely forgotten System.AddIn namespace, the very popular MEF framework and AOP solutions like Postsharp.

Yuletide

Thanks for the explanation and the confirmation that it won't do what I want. – Envision

I have confirmed that you can create an extern method without any attribute, and the class will compile successfully; but at runtime it will throw on class load. ("Could not load type <type> from assembly <assembly> because the method <method> has no implementation (no RVA).") (I guess you could create some post-build process which will open the DLL, modify the assembly, and then re-save it.) – Synopsize

Answer 2

I needed to do something quite similar (relaying method calls) recently. I ended up generating a type that forwarded method calls dynamically during runtime.

For your use case, the implementation would look something like this. First create an interface that describes your service. You will use this interface wherever you want to call the service in your code.

public interface IMyService
{
    [ServiceImport("service_name")]
    string ServiceName(int arg1, string arg2);
}

Then run the code to generate a class that implements this interface dynamically.

// Get handle to the method that is going to be called.
MethodInfo executeMethod = typeof(Services).GetMethod("Execute");

// Create assembly, module and a type (class) in it.
AssemblyName assemblyName = new AssemblyName("MyAssembly");
AssemblyBuilder assemblyBuilder = AppDomain.CurrentDomain.DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run, (IEnumerable<CustomAttributeBuilder>)null);
ModuleBuilder moduleBuilder = assemblyBuilder.DefineDynamicModule("MyModule");
TypeBuilder typeBuilder = moduleBuilder.DefineType("MyClass", TypeAttributes.Class | TypeAttributes.Public, typeof(object), new Type[] { typeof(IMyService) });
typeBuilder.DefineDefaultConstructor(MethodAttributes.Public);

// Implement each interface method.
foreach (MethodInfo method in typeof(IMyService).GetMethods())
{
    ServiceImportAttribute attr = method
        .GetCustomAttributes(typeof(ServiceImportAttribute), false)
        .Cast<ServiceImportAttribute>()
        .SingleOrDefault();

    var parameters = method.GetParameters();

    if (attr == null)
    {
        throw new ArgumentException(string.Format("Method {0} on interface IMyService does not define ServiceImport attribute."));
    }
    else
    {
        // There is ServiceImport attribute defined on the method.
        // Implement the method.
        MethodBuilder methodBuilder = typeBuilder.DefineMethod(
            method.Name,
            MethodAttributes.Public | MethodAttributes.Virtual,
            CallingConventions.HasThis,
            method.ReturnType,
            parameters.Select(p => p.ParameterType).ToArray());

        // Generate the method body.
        ILGenerator methodGenerator = methodBuilder.GetILGenerator();

        LocalBuilder paramsLocal = methodGenerator.DeclareLocal(typeof(object[])); // Create the local variable for the params array.
        methodGenerator.Emit(OpCodes.Ldc_I4, parameters.Length); // Amount of elements in the params array.
        methodGenerator.Emit(OpCodes.Newarr, typeof(object)); // Create the new array.
        methodGenerator.Emit(OpCodes.Stloc, paramsLocal); // Store the array in the local variable.

        // Copy method parameters to the params array.
        for (int i = 0; i < parameters.Length; i++)
        {
            methodGenerator.Emit(OpCodes.Ldloc, paramsLocal); // Load the params local variable.
            methodGenerator.Emit(OpCodes.Ldc_I4, i); // Value will be saved in the index i.
            methodGenerator.Emit(OpCodes.Ldarg, (short)(i + 1)); // Load value of the (i + 1) parameter. Note that parameter with index 0 is skipped, because it is "this".
            if (parameters[i].ParameterType.IsValueType)
            {
                methodGenerator.Emit(OpCodes.Box, parameters[i].ParameterType); // If the parameter is of value type, it needs to be boxed, otherwise it cannot be put into object[] array.
            }

            methodGenerator.Emit(OpCodes.Stelem, typeof(object)); // Set element in the array.
        }

        // Call the method.
        methodGenerator.Emit(OpCodes.Ldstr, attr.Name); // Load name of the service to execute.
        methodGenerator.Emit(OpCodes.Ldloc, paramsLocal); // Load the params array.
        methodGenerator.Emit(OpCodes.Call, executeMethod); // Invoke the "Execute" method.
        methodGenerator.Emit(OpCodes.Ret); // Return the returned value.
    }
}

Type generatedType = typeBuilder.CreateType();

// Create an instance of the type and test it.
IMyService service = (IMyService)generatedType.GetConstructor(new Type[] { }).Invoke(new object[] { });
service.ServiceName(1, "aaa");

This solution is probably a bit messy, but if you want to save having to create the code yourself, it works quite well. Note that there is a performance hit associated with creating the dynamic type. However this is usually done during initialization and should not affect runtime too much.

Alternatively I advise you take a look at PostSharp that allows you to generate code during compile time. It is a paid commercial solution however.

Gyrostatic

Hmm. That technique would work, but if it's being done, it might as well be done at compile time in other code. (Our build system is quite capable of doing such things as generating C# code from a Python script and then compiling it, which would reduce the runtime perf hit.) Thanks! – Envision

Answer 3

Even though it's not quite what you asked for, I would recommend creating your own T4 template which will generate these helper methods. This is especially useful if you have some programmatic API to get a list of service names and it's applicable parameter types.

Hexyl

Services are actually registered at runtime and acceptable arguments can't be queried—each just takes a list of arguments and can do what it likes with them (typically calling a "verify" function, specifying the types it wants, but not always). But I expect I probably will end up doing code generation. Thanks! – Envision