Why is NotImplemented truthy in Python 3?

Question

This question is spurred from the answers and discussions of this question. The following snippet shows the crux of the question:

>>> bool(NotImplemented)
True

The questions I have are the following:

1. Why was it decided that the bool value of NotImplemented should be True? It feels unpythonic.
2. Is there a good reason I am unaware of? The documentation seems to just say, "because it is".
3. Are there any examples where this is used in a reasonable manner?

Reasoning behind why I believe it's unintuitive (please disregard the lack of best practice):

>>> class A:
...     def something(self):
...         return NotImplemented
...
>>> a = A()
>>> a.something()
NotImplemented
>>> if a.something():
...     print("this is unintuitive")
...
this is unintuitive

It seems an odd behavior that something with such a negative connotation (lack of implementation) would be considered truthy.

Relevant text from:

NotImplemented

Special value which should be returned by the binary special methods (e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate that the operation is not implemented with respect to the other type; may be returned by the in-place binary special methods (e.g. __imul__(), __iand__(), etc.) for the same purpose. Its truth value is true.

^{— From the Python Docs}

Edit 1

To clarify my position, I feel that NotImplemented being able to evaluate to a boolean is an anti-pattern by itself. I feel like an Exception makes more sense, but the prevailing idea is that the constant singleton was chosen for performance reasons when evaluating comparisons between different objects. I suppose I'm looking for convincing reasons as to why this is "the way" that was chosen.

Blinkers

True is the default (bool(object()) is True), and there isn't a compelling reason to change it. – Rescript

All objects are truthy unless the class provides for false-y objects. The question is, is there a good reason for NotImplemented to not be truthy? – Dormant

"3. Are there any examples where this is used in a reasonable manner?" Are there any examples where it makes sense to apply a boolean test to it at all? What motivated the question? – Craddock

@MarkRansom it was this question that motivated my own: #53984616 Granted, the REASON behind the question is kind of wrong, IE using __eq__ on None is bad practice, but it just kind of made me wonder about why NotImplemented is truthy in the first place. – Blinkers

The reasoning in your example code snippet is entirely unconvincing. Why would entering if not a.something() be any more intuitive than entering if a.something()? – Sixtyfour

@Sixtyfour It wouldn't be any more intuitive. I agree, honestly I think it should raise an exception. Are you trying to say that it's intuitive in the way that it works? – Blinkers

You think bool(NotImplented) should raise an exception? numpy arrays do similar to avoid accidental ambiguity, but I'm not aware of any precedence for this in Python directly (and still not convinced on the use-case). – Sixtyfour

@Sixtyfour Hmm, not exactly. If I were to start anew, I would say that NotImplemented should not exist and anywhere we return NotImplemented we should raise NotImplementedError or something similar. Unfortunately, that would be complex and difficult to achieve (while keeping compatibility). So that's where I went with making it falsy instead. Having it raise an exception might do the trick though...I'd need to think on it more. – Blinkers

If NotImplemented didn't exist, how would __radd__ and all the other reflected operations in the datamodel work? They use this singleton to indicate "I don't support this operation, but the other operand might". – Sixtyfour

@wim: And the singleton is only intended for that use case, so the OP returning it from something that isn't a binary operator overload is a misuse; they should have done raise NotImplementedError("We forgot to implement something, sorry") or the like. – Adhamh

@Sixtyfour I suppose an exception would work, but that would definitely add some performance overhead to simple operations. I understand the use case of the singleton and why it works, but the point I am trying to get at is the oddly unapparent, yet seemingly accepted, fact that bool(NotImplemented) == True. Honestly, even if it was fasly, couldn't it still work as it is? Just rework some of the internal reflected operation logic inside the datamodel? – Blinkers

To make it falsey would be adding a special case, and to add a special case needs to be justified with a convincing use-case. The idea that it's currently unintuitive, unapparent, or displaying odd behavior is not a use-case, it's just an opinion. – Sixtyfour

Answer 1

By default, an object is considered truthy (bool(obj) == True) unless its class provides a way to override its truthiness. In the case of NotImplemented, no one has ever provided a compelling use-case for bool(NotImplemented) to return False, and so <class 'NotImplementedType'> has never provided an override.

Dormant

Relevant documentation – Casmey

Although I want to fervently disagree about not having a compelling use-case, I believe you have answered the question. How do you feel about the edited example I added to my original question and something that might justify changing NotImplemented to falsy? – Blinkers

@Julian: Nothing would justify changing it to falsy. Your use of it is in error (it's only for binary operator overloads, to trigger delegation to the other object being operated on, not intended for indicating a method hasn't been implemented yet). Correct code would use raise NotImplementedError("msg") to avoid silently treating the result as either truthy or falsy. IMO, the only reasonable change would be to make NotImplementedType.__bool__ raise TypeError, so people couldn't accidentally misuse it in a boolean context the way Python used to. – Adhamh

@Adhamh I completely get that this isn't the right way to use the NotImplemented singleton, but that's kind of the point I am getting at. The zen of python states that "There should be one-- and preferably only one --obvious way to do it", which I think the NotImplemented singleton violates quite egregiously. The purpose of the question isn't "why can't I do this this way?" so much as it is to point out the ambiguity of the reasoning behind this singleton and how it is nebulous in its purpose (unless you're deeply knowledgable about the underlying datamodel logic). – Blinkers

@Adhamh I believe wholeheartedly that the raising of an error is the proper way to fix the code example I used. The reason I suggested making bool(NotImplemented) == True is that it would then make logical sense regardless of the context and the underlying datamodel could simply update it's logic to use this updated singleton without causing undue confusion for people unaware of the idiosyncrasies of NotImplemented and NotImplementedError – Blinkers

@Julian: This isn't about idiosyncrasies. The docs for NotImplemented are super clear here, and even include a specific note on how it differs from NotImplementedError. "Only one obvious way to do it" doesn't mean "I use a name without having read the docs for it even once, and it should do what I mean". It wouldn't be more correct on any level to have it be falsy (it's equally incorrect to treat it as either truthy or falsy, but at least defaulting to truthy matches every other Python type). – Adhamh

@Adhamh Fair enough. Evaluating its boolean value does seem strange. I would like to see it just raise an exception I suppose. I feel as though I’m satisfied with why it exists the way it does, but I still feel a nagging pain that it’s not named right, or something. It just doesn’t sit right still, but I suppose time may change that. – Blinkers

Answer 2

As the accepted answer already explains, all classes in python are considered truthy (bool(obj) returns True) unless they specifically change that via Truth Value Testing. It makes sense in some cases to override that, like an empty list, 0, or False (see a good list here).

However there is no compelling case for NotImplemented to be falsy. It's a special value used by the interpreter, it should only be returned by special methods, and shouldn't reach regular python code.

Special value which should be returned by the binary special methods (e.g. __eq__(), __lt__(), __add__(), __rsub__(), etc.) to indicate that the operation is not implemented with respect to the other type.

Incorrectly returning NotImplemented will result in a misleading error message or the NotImplemented value being returned to Python code.

It's used by the interpreter to choose between methods, or to otherwise influence behaviour, as is the case with with the comparison operator == or bool() itself (it checks __bool__ first and then, if it returns NotImplemented, __len__).

Note that a NotImplementedError exception exists, presumably for when it actually is an error that an operation isn't implemented. In your specific example, something of class A should probably raise this exception instead.

Desiccator

Yeah, I completely agree with you now. I realize that NotImplemented is very specifically for the interpreter and should make its way into user code. The exception is certainly the way to go. – Blinkers