Why can't I use the null propagation operator in lambda expressions?

Question

I often use null propagating operator in my code because it gives me more readable code, specially in long queries I don't have to null-check every single class that is used.

The following code throws a compile error that we can't use null propagating operator in lambda.

var cnt = humans.AsQueryable().Count(a => a.House?[0].Price == 5000);

The error :

Error CS8072 An expression tree lambda may not contain a null propagating operator.

C# Could easily translate above code to the code to following code if really can't do anything else!

var cnt = humans.AsQueryable().Count(a => a.House != null && a.House[0].Price == 5000);

I'm curious why C# does nothing and simply throws a compiler error?

Janka

Foo?.Bar is not equivalent to Foo != null ? Foo.Bar : null because Foo is evaluated once with the null-propagating operator, and twice with the conditional, so the translation wouldn't be correct in all cases. – Dunlin

Note that if its code for EF, there is the possibility that you don't really need the null propagating operator, because when a query is converted to SQL call, SQL doesn't throw nulls :-) – Bandog

N.B.: It would also be useful to write var q = from c in Categories join p in Products on c equals p.Category into ps from p in ps.DefaultIfEmpty() select new { Category = c, ProductName = (p?.ProductName)??"(No products)"}; instead of having to write ProductName = (p == null) ? "(No products)" : p.ProductName because EF currently does not support the ?. operator. – Resolve

Answer 1

It's complicated since expression tree lambdas (unlike delegate lambdas) are interpreted by already existing LINQ providers which don't yet support null propagating.

Converting to a conditional expression is not always accurate as there are multiple evaluations while with ?. there's only a single evaluation for example:

customer.Where(a => c.Increment()?.Name) // Written by the user 
customer.Where(a => c.Increment() == null ? null : c.Increment().Name) // Incorrectly interpreted by an old LINQ provider

You can go deeper in the relevant discussion on CodePlex where 3 solutions are offered: NullPropagationExpression, ConditionalExpression & a hybrid

Silicic

I would certainly not be surprised if certain query providers couldn't support it, but that's not a reason to not have the C# language support it. – Sino

@Sino actually I think it's a great reason. But anyways, a PR was offered and not yet declined. – Silicic

The fact that certain query providers don't yet support it isn't a reason to prohibit all query providers from ever being able to use it. – Sino

@Sino "ever" is a harsh word. But for now, that seems very reasonable. Especially when a single evaluation requires more than just the support of the provider. It requires support from the data store itself. – Silicic

And obviously no query provider is going to take the time to support handling such a request until users of that provider would be able to actually create expression trees that represent it. For this to be supported, the first thing that needs to happen is for lambdas to be able to represent it. After that exists, query providers can begin to support it, as they feel it is appropriate. There are also lots of providers out there doing all sorts of different things. It's not like EF is the only query provider in the world. – Sino

@Sino No provider would support it before EF (or Linq2SQL) will. And that single provider would probably cost more than the null propagation feature as a whole. And again, the underlying store needs to support that before the provider can. Anyway, you can lobby that issue with the Roslyn guys if you like. – Silicic

There are lots of providers out there that support all sorts of things that EF doesn't. It's wrong to say that no provider would support it before EF would. Providers mapping to language that has analogous null propagating semantics, or that have a simple transformation that would have the desired semantics (hint, both of those things are true for SQL, so having EF support this would be very easy) are going to have an easier time than those that don't. And of course certain organisations create their own query providers precisely because EF doesn't support the features they want. – Sino

Saying that it's not worth the effort to add because they assume no query providers will ever want to implement it is radically different than saying that they shouldn't implement it because no query providers currently support it (because after all it's functionally impossible for them to support it until MS adds this). – Sino

@Sino again, "ever" is a harsh word (that I didn't say). No providers currently support it and so implementing it has only costs with no immediate value. No one is stopping them from doing it in C# 7.0. – Silicic

Yes, actually they are, and it's this change not being supported. Why would any query provider spend effort supporting expressions that none of their users are actually going to be able to create? It's like writing source code using a language feature that hasn't yet been announced as a valid feature for the language. Obviously people can't write code using a feature until the language supports using that feature. It's the same here. Saying that providers could implement support for this expression now is simply ludicrous. – Sino

That's what I was asking you. – Sino

@Sino No one is saying that people should implement this feature in providers before the C# compiler supports it as well. – Silicic

That's what it sounds like your answer is saying. If that's not what you're saying, then what is your answer saying? – Sino

@Sino That's not what it sounds like at all, and I doubt you were confused about it. My answer states that it would break current providers and hacking it like the OP suggested isn't accurate (which is correct). It basically means that adding null propagating in LINQ is out of scope and too big to be worth it. It doesn't stop them from adding it in the future like they add every feature, by announcing it to partners and the public and providing a reference implementation in their flagship provider. – Silicic

How would it break current providers? There isn't currently any code out there that uses the null propagation operator in lambdas, so adding support for it won't break anything. You're quite right that it shouldn't do the translation that the OP suggests, because it's not valid. It should support the operator and have an Expression object that represents it. – Sino

Your answer opens with It's complicated since expression tree lambdas (unlike delegate lambdas) are interpreted by already existing LINQ providers which don't necessarily support null propagating. which sounds very much like you're saying that support for the null propagation operator in lambdas shouldn't be added because none of the existing query providers support using the null propagation operator in lambda. If that's not what you meant, it is what it read like, so you should clarify it. – Sino

@Sino current providers, not current code using these providers. New code trying to use the current providers would always fail. And again, that sentence doesn't sound like that at all. It doesn't mention what should, shouldn't or can't be done. It only explains the current situation. Anyways, I edited it. I hope you can understand it now... – Silicic

There are a million ways to break current providers. Just sticking in any custom method into a provider breaks it. Obviously if support is added to allow null propagation in lambdas and that code is used by providers that don't support it, it'll break. Not all C# code is valid in expressions for any given query provider; anyone using a query provider needs to know that. That's not a reason to not implement null propagation in lambdas. That's not something that will ever go away. The current providers won't be able to support this feature until its implemented. – Sino

@Sino There are feature that only work for some providers but this feature will fail for all current providers. That's why it's reasonable for them not to include it in a small C# compiler feature. When they do decide to add such a feature to linq they announce in advance to let partners who make providers know and prepare (and that includes internal MS providers). They're not expected to implement a feature that won't work unless someone, somewhere decides they want to support it. – Silicic

This isn't a language patch. It's a new major version of the language. They also will have had to go out of their way, from the moment they decided to add the operator, to not put it in lambdas. There are very few things that can't be done in explicitly in expressions lambdas that can be done in delegate lambdas. It takes quite a bit to go out of the way to not support it. Probably more so than actually supporting it, in fact. As a rule, a query provider needs to expect that any Expression objects that accept can represent any valid C# expressions. That's what they're there for. – Sino

The whole point of Expression is to be able to represent all C# expressions semantically as code. It's not designed to be just some small subset of the language. – Sino

@Sino this version is more a language patch than a new major version. All the features planned for v6.0 are small and syntactic-sugari and not all that were planned made the cut. This release cycle focuses on the compiler rewrite and VS tools. There's no async-await, LINQ, dynamic or generics, just null propagation and nameof etc. – Silicic

This is a chicken or egg problem, how can providers implement support for null conditional operators via expressions, if the expressions don't support it in the first place? – Fusco

Seems like this still isn't resolved 3 years later - shouldn't Microsoft have been able to find the time by now? They seem to have a bad habit of using time and resources as an excuse for half-implementing new features in C# these days. – Brynne

@Silicic do you have an updated link for the discussion that was on codeplex? Im intrigued.. many thanks. – Olindaolinde

That codeplex discussion is a dead link. Has that been migrated anywhere? – Eshelman

@EdNorman Closest I could find: github.com/dotnet/csharplang/issues/2545 and web.archive.org/web/20150328213020/https://roslyn.codeplex.com/… – Eshelman

There are more developers relying on inspecting LINQ expressions than just query providers. It would break them all, in runtime probably. Is such a breaking change than I can totally see why Microsoft erred on the side of caution. – Nunuance

What makes this even worse is that, in practical terms, you need one version of an expression that works for things like Linq to Sql, and another that works on in-memory objects because otherwise you get null reference exceptions. 🤦‍♂️ – Guardsman