ref and out parameters in C# and cannot be marked as variant

Question

What does the statement mean?

From here

ref and out parameters in C# and cannot be marked as variant.

1) Does it mean that the following can not be done.

public class SomeClass<R, A>: IVariant<R, A>
{
    public virtual R DoSomething( ref A args )
    {
        return null;
    }
}

2) Or does it mean I cannot have the following.

public delegate R Reader<out R, in A>(A arg, string s);

public static void AssignReadFromPeonMethodToDelegate(ref Reader<object, Peon> pReader)
{
    pReader = ReadFromPeon;
}

static object ReadFromPeon(Peon p, string propertyName)
    {
        return p.GetType().GetField(propertyName).GetValue(p);
    }

static Reader<object, Peon> pReader;

static void Main(string[] args)
    {
        AssignReadFromPeonMethodToDelegate(ref pReader);
        bCanReadWrite = (bool)pReader(peon, "CanReadWrite");

        Console.WriteLine("Press any key to quit...");
        Console.ReadKey();
    }

I tried (2) and it worked.

Hennessey

Answer 1

"out" means, roughly speaking, "only appears in output positions".

"in" means, roughly speaking, "only appears in input positions".

The real story is a bit more complicated than that, but the keywords were chosen because most of the time this is the case.

Consider a method of an interface or the method represented by a delegate:

delegate void Foo</*???*/ T>(ref T item);

Does T appear in an input position? Yes. The caller can pass a value of T in via item; the callee Foo can read that. Therefore T cannot be marked "out".

Does T appear in an output position? Yes. The callee can write a new value to item, which the caller can then read. Therefore T cannot be marked "in".

Therefore if T appears in a "ref" formal parameter, T cannot be marked as either in or out.

Let's look at some real examples of how things go wrong. Suppose this were legal:

delegate void X<out T>(ref T item);
...
X<Dog> x1 = (ref Dog d)=>{ d.Bark(); }
X<Animal> x2 = x1; // covariant;
Animal a = new Cat();
x2(ref a);

Well dog my cats, we just made a cat bark. "out" cannot be legal.

What about "in"?

delegate void X<in T>(ref T item);
...
X<Animal> x1 = (ref Animal a)=>{ a = new Cat(); }
X<Dog> x2 = x1; // contravariant;
Dog d = new Dog();
x2(ref d);

And we just put a cat in a variable that can only hold dogs. T cannot be marked "in" either.

What about an out parameter?

delegate void Foo</*???*/T>(out T item);

? Now T only appears in an output position. Should it be legal to make T marked as "out"?

Unfortunately no. "out" actually is not different than "ref" behind the scenes. The only difference between "out" and "ref" is that the compiler forbids reading from an out parameter before it is assigned by the callee, and that the compiler requires assignment before the callee returns normally. Someone who wrote an implementation of this interface in a .NET language other than C# would be able to read from the item before it was initialized, and therefore it could be used as an input. We therefore forbid marking T as "out" in this case. That's regrettable, but nothing we can do about it; we have to obey the type safety rules of the CLR.

Furthermore, the rule of "out" parameters is that they cannot be used for input before they are written to. There is no rule that they cannot be used for input after they are written to. Suppose we allowed

delegate void X<out T>(out T item);
class C
{
    Animal a;
    void M()
    {
        X<Dog> x1 = (out Dog d) => 
        { 
             d = null; 
             N(); 
             if (d != null) 
               d.Bark(); 
        };
        x<Animal> x2 = x1; // Suppose this were legal covariance.
        x2(out this.a);
    }
    void N() 
    { 
        if (this.a == null) 
            this.a = new Cat(); 
    }
}

Once more we have made a cat bark. We cannot allow T to be "out".

It is very foolish to use out parameters for input in this way, but legal.

---

UPDATE: C# 7 has added in as a formal parameter declaration, which means that we now have both in and out meaning two things; this is going to create some confusion. Let me clear that up:

• in, out and ref on a formal parameter declaration in a parameter list means "this parameter is an alias to a variable supplied by the caller".
• ref means "the callee may read or write the aliased variable, and it must be known to be assigned before the call.
• out means "the callee must write the aliased variable via the alias before it returns normally". It also means that the callee must not read the aliased variable via the alias before it writes it, because the variable might not be definitely assigned.
• in means "the callee may read the aliased variable but does not write to it via the alias". The purpose of in is to solve a rare performance problem, whereby a large struct must be passed "by value" but it is expensive to do so. As an implementation detail, in parameters are typically passed via a pointer-sized value, which is faster than copying by value, but slower on the dereference.
• From the CLR's perspective, in, out and ref are all the same thing; the rules about who reads and writes what variables at what times, the CLR does not know or care.
• Since it is the CLR that enforces rules about variance, rules that apply to ref also apply to in and out parameters.

In contrast, in and out on type parameter declarations mean "this type parameter must not be used in a covariant manner" and "this type parameter must not be used in a contravariant manner", respectively.

As noted above, we chose in and out for those modifiers because if we see IFoo<in T, out U> then T is used in "input" positions and U is used in "output" positions. Though that is not strictly true, it is true enough in the 99.9% use case that it is a helpful mnemonic.

It is unfortunate that interface IFoo<in T, out U> { void Foo(in T t, out U u); } is illegal because it looks like it ought to work. It cannot work because from the CLR verifier's perspective, those are both ref parameters and therefore read-write.

This is just one of those weird, unintended situations where two features that logically ought to work together do not work well together for implementation detail reasons.

Gratia

Hi Eric, I am going to have to get some sleep before I can assure you of a devoted following of your discussion. For now, thanks a million. Your passion for your work is evidently insurmountable, unsurpassable (if there is a word as such). While I do appreciate the reasons behind the improbabilities of having a ref/out type as a variant parameter or return type, I'll re-read your answer when I'm well rested to do some justice to the effort you've put in. Thank you many times over again. – Hennessey

Cool explanation. I just found this thread after a search. Initially I thought that a generic type T (belonging to a delegate type) that was used only in out parameter "position", should be allowed to be marked covariant. I thought the explanation that won't compile, was the issue that ref and out are "the same" behind the scenes. But your last example convinces me. That also explains why for normal method calls (forget delegates and generic variance, think C# version 1.2), the variable put into an out parameter must mach the type exactly (being a less derived type is not OK). – Zane

@JeppeStigNielsen: Conceptually, it should be possible for a compiler to allow variable Foo of type Animal to be passed to an out parameter of type Dog by passing a temporary variable of type Dog to the routine and then copying that to Foo once the routine exists. Conceptually, that's what out should mean anyhow. Unfortunately, the creators of .net didn't want to require languages to implement out parameters, and thus .net cannot make any assumptions that would break if a function's caller treated out as ref. – Cowhide

@Cowhide You're right. But only if it was always illegal for a method to "read" its own out parameter. But as Lippert points out, it is not. When you "read" from your out parameter you break covariance. Actually, if the out parameter is (has been passed) a field of some class, another thread than yours could modify your out parameter reference to point to a new object of another runtime type, while your method is running. – Zane

The kind of refness supercat is describing is called "copy-in-copy-out" and there are languages that use it. This idea of passing a variable by reference, or, equivalently, making an alias to a variable, is in my opinion a case of a highly efficient implementation detail being surfaced as a confusing language feature. The fact that so many people do not understand how it works indicates that it is maybe not a great feature. – Gratia

@EricLippert: There are things which can only be done with true ref parameters. The big problem is not that ref parameters exist, but rather that read-write ref parameters are the only alternative to value-in parameters, and there aren't any copy-out, in-out, and read-only-ref parameters. A related problem is that there is no way for methods to specify how this should be passed; the fact that it is passed by value for class types and by ref for structs means that heap-stored objects must have reference semantics even in cases where value semantics (using immutable heap objects)... – Cowhide

...would be cleaner. Further, properties could safely be passed to methods which have explicit in-out parameters (as distinct from those with ref parameters), by reading the property, making the call, and then writing it back. – Cowhide

So it's all about out parameter acts as limited ref , but not like another returned value. So theoretically in some cases worth introduce additional keyword for latter behaviour. Although it could be workaround-ed by returning Tuple. – Moonraker

@КоеКто that's correct; from the CLR perspective, out and ref are the same thing. They only have different semantics for purposes of definite assignment because the compiler makes it so. And yes, use value tuples to return multiple values; I do not know why it took so long to get value tuples! – Gratia

typo in snippet 5 on line 14: x<Animal> typename should be a capital X to match the type definition. (this is too small a change for me to make, only the author can do so) – Ereshkigal

Answer 2

Eric Lippert has a great explanation of why this constraint exists.

If you are looking to work around this limitation. You can do it like this:

public static class ResultExtension{
    public static bool TryGetValue<T>(this IResult<T> self, out T res) {
        if (self.HasValue) {
            res = self.Value;
            return true;

        }
        res = default;
        return false;
    }
}

public interface IResult<out T>
{
    bool HasValue { get; }
    T Value { get; }
}

This works because there are two structures. One structure gets the covariance and the other gets the out parameter. The out parameter is not marked as variant so the compiler is happy.

Sagacious

Answer 3

It means you can't have the following declaration:

public delegate R MyDelegate<out R, in A>(ref A arg);

Edit: @Eric Lippert corrected me that this one is still legal:

public delegate void MyDelegate<R, in A>(A arg, out R s);

It actually makes sense, since the R generic parameter is not marked as a variant, so it doesn't violate the rule. However, this one is still illegal:

public delegate void MyDelegate<out R, in A>(A arg, out R s);

Amygdaloid

This is incorrect. The second line is perfectly legal. You are correct that the first line is illegal. – Gratia

@Eric - ah, the second line is legal because the R generic parameter is not marked as variant. is that right? – Amygdaloid

Many thanks, Eric. I should've jumped at that myself if only I had the patience and proof-reading skills you do. But let me assure you, I completely follow the reasons behind the error that you've shed light on. – Hennessey

Answer 4

But, the following code can be compiled:

interface IFoo<out T>
{
    T Get();

    //bool TryGet(out T value); // doesn't work: Invalid variance: The type parameter 'T' must be invariantly valid on 'IFoo<T>.TryGet(out T)'. 'T' is covariant.

    bool TryGet(Action<T> value); // works!
}

Twophase