Why return NotImplemented instead of raising NotImplementedError

Question

Python has a singleton called NotImplemented.

Why would someone want to ever return NotImplemented instead of raising the NotImplementedError exception? Won't it just make it harder to find bugs, such as code that executes invalid methods?

Wachtel

Answer 1

It's because __lt__() and related comparison methods are quite commonly used indirectly in list sorts and such. Sometimes the algorithm will choose to try another way or pick a default winner. Raising an exception would break out of the sort unless caught, whereas NotImplemented doesn't get raised and can be used in further tests.

http://jcalderone.livejournal.com/32837.html

To summarise that link:

"NotImplemented signals to the runtime that it should ask someone else to satisfy the operation. In the expression a == b, if a.__eq__(b) returns NotImplemented, then Python tries b.__eq__(a). If b knows enough to return True or False, then the expression can succeed. If it doesn't, then the runtime will fall back to the built-in behavior (which is based on identity for == and !=)."

Dichotomous

I would be careful using it, as this link points out near the end of the document. – Macon

When Python interpreter checks whether a.__eq__(b) returned NotImplemented, couldn't it just as easily catch NotImplementedError instead (and call b.__eq__(a) or whatever then)? – Beedon

@Veky. Raising an exception probably has a higher overhead. Any overhead in a sort operation will get magnified by the size of the list so even if the difference was very small it would still make sense to find a faster implementation. You also don't want to be breaking out of your loops and reentering them which a try/catch implementation would require. – Dichotomous

For the first point, even faster solution would be to automatically synthesize lt as reversed gt, instead of always calling something that will return NotImplemented - and Python doesn't do that. I don't think speed is the reason here. And for the second, I don't understand what you're saying: return will require just as much breaking out of loops as raise would. In fact, you can imagine return as raising a special Return exception, which is always caught in the calling scope. – Beedon

>> "magnified by the size of the list" At least, unless you have an O(n) sort the world ought to know about. – Brainstorm

Agreed, speed is not the (only) reason. The biggest issue is that a and b could be different classes. Maybe class A a generic standard class written ten years ago by someone who'd never heard of class B, so A.__lt__ has no idea how to handle the comparison. But maybe class B is an updated, specialized class that was specifically written to be backwards-compatible with class A, so B.__gt__ knows exactly how to handle the comparison. – Location

Answer 2

Because they have different use cases.

Quoting the docs (Python 3.6):

NotImplemented

should be returned by the binary special methods (e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate that the operation is not implemented with respect to the other type

exception NotImplementedError

[...] In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method, or while the class is being developed to indicate that the real implementation still needs to be added.

See the links for details.

Mohr

Answer 3

One reason is performance. In a situation like rich comparisons, where you could be doing lots of operations in a short time, setting up and handling lots of exceptions could take a lot longer than simply returning a NotImplemented value.

Terryn

Answer 4

Returning NotImplemented by a function is something like declaring that the function is unable to process the inputs but instead of raising exception, the control is transferred to another function known as Reflection Function with a hope that the Reflection Function might be able to process the inputs.

Firstly, the Reflection Functions associated with any functions are predefined. Secondly when the original function returns NotImplemented, interpreter runs the Reflection Function but on the flipped order of input arguments.

You can find detailed examples here

Carbonyl

Note that this answer is wrong: (1) this only applies to special functions like eq, add etc..., your regular functions will return NotImplemented singleton as usual and (2) there is no such thing as "reflection function", this is something random bloggers made up. Official docs ( docs.python.org/3/library/constants.html#NotImplemented ) say "interpreter will try the reflected operation on the other type" - so if x.__add__(y) returns NotImplemented, python will call y._add__(x) – Vanillin

Answer 5

If a special method supporting a binary operation is not implemented it should return NotImplemented. On the other hand, NotImplementedError should be raised from abstract methods inside user defined base classes to indicate that derived classes should override those methods.

Basically, NotImplemented helps the interpreter support a binary operation, whereas the NotImplementedError throws an exception. You can find a general usage here.

Clipper

Slightly misses the point. You can also return NotImplemented to indicate that the operation isn't supported for these particular arguments, even if it is implemented for other arguments. For instance, imagine that the int class didn't know how to compare itself to the float class. Evaluating 5<10 would invoke 5.__lt__(10) which would return True, but evaluating 5 < 10.1 would invoke 5.__lt__(10.1) which would return NotImplemented. (In the second case, the interpreter would then call 10.1__gt__(5), to see if the float class knew how to handle the comparison instead) – Location