What is the rationale for all comparisons returning false for IEEE754 NaN values?

Question

Why do comparisons of NaN values behave differently from all other values? That is, all comparisons with the operators ==, <=, >=, <, > where one or both values is NaN returns false, contrary to the behaviour of all other values.

I suppose this simplifies numerical computations in some way, but I couldn't find an explicitly stated reason, not even in the Lecture Notes on the Status of IEEE 754 by Kahan which discusses other design decisions in detail.

This deviant behavior is causing trouble when doing simple data processing. For example, when sorting a list of records w.r.t. some real-valued field in a C program I need to write extra code to handle NaN as the maximal element, otherwise the sort algorithm could become confused.

Edit: The answers so far all argue that it is meaningless to compare NaNs.

I agree, but that doesn't mean that the correct answer is false, rather it would be a Not-a-Boolean (NaB), which fortunately doesn't exist.

So the choice of returning true or false for comparisons is in my view arbitrary, and for general data processing it would be advantageous if it obeyed the usual laws (reflexivity of ==, trichotomy of <, ==, >), lest data structures which rely on these laws become confused.

So I'm asking for some concrete advantage of breaking these laws, not just philosophical reasoning.

Edit 2: I think I understand now why making NaN maximal would be a bad idea, it would mess up the computation of upper limits.

NaN != NaN might be desirable to avoid detecting convergence in a loop such as

while (x != oldX) {
    oldX = x;
    x = better_approximation(x);
}

which however should better be written by comparing the absolute difference with a small limit. So IMHO this is a relatively weak argument for breaking reflexivity at NaN.

Benis

Once a NaN has entered the computation, it will typically never leave, so your convergence test would become an infinite loop. It's usually preferable to report the failure to converge to the calling routine, possibly by returning NaN. Thus, the loop structure would typically become something like while (fabs(x - oldX) > threshold), exiting the loop if convergence happens or a NaN enters the computation. Detection of the NaN and appropriate remedy would then happen outside the loop. – Avaricious

If NaN were the minimal element of the order that while loop would still work. – Benis

Food for thought: grouper.ieee.org/groups/1788/email/pdfmPSi1DgZZf.pdf page 10 – Benis

Answer 1

I was a member of the IEEE-754 committee, I'll try to help clarify things a bit.

First off, floating-point numbers are not real numbers, and floating-point arithmetic does not satisfy the axioms of real arithmetic. Trichotomy is not the only property of real arithmetic that does not hold for floats, nor even the most important. For example:

• Addition is not associative.
• The distributive law does not hold.
• There are floating-point numbers without inverses.

I could go on. It is not possible to specify a fixed-size arithmetic type that satisfies all of the properties of real arithmetic that we know and love. The 754 committee has to decide to bend or break some of them. This is guided by some pretty simple principles:

1. When we can, we match the behavior of real arithmetic.
2. When we can't, we try to make the violations as predictable and as easy to diagnose as possible.

Regarding your comment "that doesn't mean that the correct answer is false", this is wrong. The predicate (y < x) asks whether y is less than x. If y is NaN, then it is not less than any floating-point value x, so the answer is necessarily false.

I mentioned that trichotomy does not hold for floating-point values. However, there is a similar property that does hold. Clause 5.11, paragraph 2 of the 754-2008 standard:

Four mutually exclusive relations are possible: less than, equal, greater than, and unordered. The last case arises when at least one operand is NaN. Every NaN shall compare unordered with everything, including itself.

As far as writing extra code to handle NaNs goes, it is usually possible (though not always easy) to structure your code in such a way that NaNs fall through properly, but this is not always the case. When it isn't, some extra code may be necessary, but that's a small price to pay for the convenience that algebraic closure brought to floating-point arithmetic.

---

Addendum: Many commenters have argued that it would be more useful to preserve reflexivity of equality and trichotomy on the grounds that adopting NaN != NaN doesn’t seem to preserve any familiar axiom. I confess to having some sympathy for this viewpoint, so I thought I would revisit this answer and provide a bit more context.

My understanding from talking to Kahan is that NaN != NaN originated out of two pragmatic considerations:

• That x == y should be equivalent to x - y == 0 whenever possible (beyond being a theorem of real arithmetic, this makes hardware implementation of comparison more space-efficient, which was of utmost importance at the time the standard was developed — note, however, that this is violated for x = y = infinity, so it’s not a great reason on its own; it could have reasonably been bent to (x - y == 0) or (x and y are both NaN)).

• More importantly, there was no isnan( ) predicate at the time that NaN was formalized in the 8087 arithmetic; it was necessary to provide programmers with a convenient and efficient means of detecting NaN values that didn’t depend on programming languages providing something like isnan( ) which could take many years. I’ll quote Kahan’s own writing on the subject:

Were there no way to get rid of NaNs, they would be as useless as Indefinites on CRAYs; as soon as one were encountered, computation would be best stopped rather than continued for an indefinite time to an Indefinite conclusion. That is why some operations upon NaNs must deliver non-NaN results. Which operations? … The exceptions are C predicates “ x == x ” and “ x != x ”, which are respectively 1 and 0 for every infinite or finite number x but reverse if x is Not a Number ( NaN ); these provide the only simple unexceptional distinction between NaNs and numbers in languages that lack a word for NaN and a predicate IsNaN(x).

Note that this is also the logic that rules out returning something like a “Not-A-Boolean”. Maybe this pragmatism was misplaced, and the standard should have required isnan( ), but that would have made NaN nearly impossible to use efficiently and conveniently for several years while the world waited for programming language adoption. I’m not convinced that would have been a reasonable tradeoff.

To be blunt: the result of NaN == NaN isn’t going to change now. Better to learn to live with it than to complain on the internet. If you want to argue that an order relation suitable for containers should also exist, I would recommend advocating that your favorite programming language implement the totalOrder predicate standardized in IEEE-754 (2008). The fact that it hasn’t already speaks to the validity of Kahan’s concern that motivated the current state of affairs.

Avaricious

I read your points 1 and 2. Then I observed that in real arithmetic (extended to allow NaN in the first place) NaN is equal to itself - simply because in math, any entity is equal to itself, without exception. Now I am confused: why did IEEE not "match the behavior of real arithmetic", which would make NaN == NaN? What am I missing? – Oread

Thank you @StephenCanon for clarifying this. I am curious, however, why are such identities bad? In which realistic cases such an identity would create a problem for a programmer? (The lack of reflexivity creates problems such as these: #9905199) – Oread

Agreed; the nonreflexivity of NaNs has created no end of pain for languages like Python, with its equality-based containment semantics. You really don't want equality to fail to be an equivalence relation when you're trying to build containers on top of it. And having two separate notions of equality isn't much of a friendly option either, for a language that's supposed to be easy to learn. The result (in the case of Python) is an unpleasantly fragile compromise between respect for IEEE 754 and not-too-broken containment semantics. Fortunately, it's rare to put NaNs into containers. – Ejector

Some nice observations here: bertrandmeyer.com/2010/02/06/… – Ejector

Out of curiosity, what do you think of the two-zeroes design? I appreciate the logic of having (1/INF) and (-1/INF) return different values from each other, but if they're going to do so I would think both values should also be distinct from from (1-1). Was the decision to use two zeroish values rather than one or three motivated by hardware considerations? – Svoboda

@supercat: The encodings for two zeros fall out pretty naturally. More than two would make the encodings awkward, and likely force very simple hardware implementations to trap on zero arguments or results, which you don't want (zeros are common!). There was considerable debate about whether there should be one or two zeros and infinities -- for the mathematicians out there, whether to choose the one- or two-point compactification of the reals. Ultimately, signed zero and infinity was chosen, partially because it's easy to implement unsigned if your hardware supports signed. – Avaricious

... as for "what I think of it": it is what it is. It has drawbacks, but so do the alternatives. I think it's a good tradeoff, all considered. – Avaricious

Thanks for writing. Was the decision to use two zeroes and two infinities based on a desire to have 1/INF be zero, but 1/(+INF + -INF) be NaN? Also, I've sometimes wondered about denormals: I would think that one could simplify hardware, while retaining 99% of the usefulness, if rather than supporting denormals, hardware forced all results to be rounded to 2^-127 (or whatever the smallest non-zero normalized number happened to be for a particular format). So if 2^-127 was the smallest normalized number, the next would be 2^-126, then 1.1b*2^-126, then 2^-125, etc. So... – Svoboda

...there wouldn't be a specialized "denormal" format whose bits had to be interpreted specially--there would just be special rounding required when outputting results with small exponents. – Svoboda

@supercat: It's a reasonable idea, but the special rounding is a good portion of the work, and in the specific case of single-precision, you'd be giving up a lot of precision -- effectively, you could no longer carry full precision over 1/8th of the exponent range. (Note though that your suggestion is pretty much what decimal floating point does, so it's definitely not a terrible idea). – Avaricious

@MarkDickinson: For containers, simply making all NaNs compare equal to each other (and arbitrarily greater or lesser than everything else) as keys provides correct comparison semantics. Alternatively, an implementation could make it an error to use a NaN as a key. I wouldn't consider either of these options to be especially onerous. – Avaricious

@StephenCanon: Unfortunately it's not that easy: the first option introduces two different rules for equality and breaks simple invariants and user expectations (e.g., two lists x and y should satisfy x == y if and only if their lengths are equal and x[i] == y[i] for all relevant i). This is in fact close to what Python does (containers do an identity test followed by an equality test), but it's not nice. The second is a heavyweight solution that would break general-purpose Python functions that use sets and dictionaries for one reason or another (caching is one obvious example). – Ejector

@MarkDickinson: I am by no means an expert on the implementation details of Python, but it seems to me that there's no great difficulty in using different notions of equality in different contexts (though there is a lot of subtlety). Some semantic contexts require strict dichotomy, and some do not. It's worth noting that the semantics of NaN comparisons date back to Konrad Zuse's work, so there's an enormous weight of history behind them operating the way that they do. – Avaricious

@StephenCanon: No great difficulty, agreed; it just gets messy and confusing for users (and there's plenty of evidence of that confusion on the various mailing lists). The notion of containment is bound up with this as well, which doesn't help. There's no really good solution here. I agree with your earlier comment that this is a tradeoff; I just happen to think that the drawbacks of the specified behaviour outweigh the benefits, rather than the other way around. (And now I'm going to get told off by SO for extended conversations in chat. :-) Maybe there's a better place for this?) – Ejector

@StephenCanon: In what way would (0/0) == (+INF) + (-INF) be any more nonsensical than having 1f/3f == 10000001f/30000002f? If floating-point values are considered to be equivalence classes, then a=b doesn't mean "The computations which yielded a and b, if done with infinite precision, would yield identical results", but rather "What is known about a matches up with what is known about b". I'm curious if you know of any examples of code where having "Nan != NaN" makes things simpler than they would be otherwise? – Svoboda

0/0 can be viewed as the set of solutions of 0 * x = 0, so all real numbers. If you view Inf as the set of all real numbers greater than the largest finitely represented one then Inf - Inf is also the set of all real numbers, so the equality wouldn't be that strange. On the other hand the sets represented by acos 2 and log -1 are both empty. – Benis

@starblue: Your argument would allow for the existence of multiple different types of NaN which would compare unequal to each other, and I would have no objection to an implementation which stated that NaN values produced by different operations, or operations on different operands, may be arbitrarily subdivided into equivalence classes. It would be nice to further require that the number of equivalence classes for operations involving NaNs be bounded, but even without such a requirement specifying that a NaN must be equivalent to itself would be better than the situation now. – Svoboda

@Svoboda You have to cut it somewhere, because you only have a finite representation. IMHO it is a mistake that NaN behaves differently from all other values, rather it should be equal to itself and minimal in the order (then approximation loops would terminate on NaN, see comment on the question above). I suppose there were not enough people interested in general data processing on the IEEE 754 committee. – Benis

@starblue: IMHO, the right solution is to have different means available to compare numbers, with different ways of handling positive and negative zero as well as NaN. Equivalence-based comparisons are perhaps most important, though, since without them nearly all mathematical axioms go out the window. Indeed, I would posit that most pre-IEEE floating-point implementations obeyed more axioms of mathematics than IEEE ones (e.g. b==c --> a+b==a+c; a < b and b < c --> a < c; a+0 == a; a*1.0 == a; etc.) How many axioms of math does IEEE floating-point actually obey? – Svoboda

If NaN was meant to "infect" operations in order to bubble the error up, then I honestly can't understand why that would stop with equality operators. "Is (some unknowable value) equal to (some unknowable value)?" Unknowable. Not the same as "no", so it shouldn't be represented with the "no" value. It would then follow that !(NaN==NaN) is also NaN. And the right behaviour when branching if(NaN){ } else { } would be to... execute neither? (just kidding) Throw an Error or something! Why would we rather branch based on a value known to be potentially mistaken, than stop right there? – Johnathan

@supercat: there are no “axioms of math” (excepting possibly the axioms of deductive logic, depending on who you ask). There are only axioms of specific mathematical systems. Floating-point numbers aren’t a group, ring, or field, so they don’t necessarily satisfy the axioms of those things. They are their own thing, with their own axioms, that happens to be a convenient approximation to the real numbers for many use cases. To pine for the days before IEEE-754 is to be hopelessly naive; many arithmetic systems at the time weren’t even closed; 0./0. might well panic your computer. – Avaricious

@StephenCanon: It's true that floating-point math cannot behave as a group, much less a ring or field, but that does not imply that floating-point numbers should not define an equivalence relation. I understand that the IEEE recommended a set of transitive relational operators in 2007, but find myself puzzled as to why a problem which should have become apparent the first time someone tried to sort a list of numbers containing a NaN still requires a tricky multi-step workaround. Sorting code is often one of the more performance-sensitive uses for floating-point comparisons... – Svoboda

...so it would seem like that would be an important place to avoid inefficiencies. Further, if one is trying to produce a list of all the different results a function might return, having every NaN appear as a separate result is not exactly helpful. – Svoboda

@StephenCanon: PS--Are there any mathematical systems of note, other than IEEE floating-point arithmetic, where there would exist some combination of values a, b, and c, such that a==b but (a+c)!=(b+c)? Perhaps you would describe the axiom that a==b implies (a+c)==(b+c) as being an axiom of individual mathematical systems' definitions of the + operator, but every system I can think of that defines +, with the sole exception of IEEE floating-point arithmetic, would include that axiom. – Svoboda

@TasosBitsios: I could certainly see uses for a trapping equality test. A non-trapping equivalence test is also an essential thing to have, however, since it allows logic such as "if this iteration yielded the same result as this other iteration, don't bother iterating any more" or "if calculating a function with the same inputs as a previous time, return a cached result". – Svoboda

@MarkDickinson Fortunately, it's rare to put NaNs into containers., except when it's arrays of floating points such as in NumPy or Matlab, where the latter has a special function to compare whether arrays are equal with equal nans, and a similar need exists for NumPy. – Pamper

@gerrit: Agreed, especially about the need for a non test-oriented version of np.testing.assert_equal. – Ejector

Since you were on the IEEE-754 committee do you have any comments on this question? – Infusorian

How is addition not associative in JavaScript? (1 + 5) + 2 === (2 + 1) + 5 // true; also, how come Trichotomy does not hold for floating point numbers? – Unspoken

" Every NaN shall compare unordered with everything, including itself." -- this is another manifestation that NaN has to be treated as numerical exception instead of being forced into the primitive type number, see #2802101 – Unreligious

@chopperdrawlion4 0.1 + (0.2 + 0.3) === (0.1 + 0.2) + 0.3 // false in JS – Reprint

Theoretically, if you had NaN == NaN and no isNaN, you could still test for NaN with !(x < 0 || x == 0 || x > 0), but it would have been slower and clumsier than x != x. – Bier

@user2357112: If that idiom had caught on, optimizing compilers would know how to optimize it to an efficient isnan(), at least with -fno-trapping-math. godbolt.org/g/y92eBP (Without that, gcc for x86 does all three comparisons separately. It has to raise the exception 3 times if x is a signalling NaN). x86 FP comparisons don't just check a single predicate, they give you above, equal, below, or unordered. Comparing against zero instead of itself is a tiny bit more expensive. It could be slower on an ISA with no direct way to check for NaN, but hopefully that'd be unusual. – Goldenseal

I think one shouldn't rationalize past decisions too much beyond "there wasn't enough experience to avoid mistakes back then". A much more egregious example is floating point denormals, which hardly anyone would argue weren't a very expensive mistake. Or the way the two middle pairs in an Ethernet cable are arranged; that one got to have cost the world many billions in unnecessary labor costs over the years, not to mention slight signal degradation. Now we know better, but back then it wasn't known, and without the benefit of hindsight it seems like a very reasonable decision. – Shuster

@Dmytry: why do you think everyone thinks denormals were a mistake? (Or expensive, for that matter?) Handling them at speed cost some area, but if you're building an fma anyway (which everyone does, now) it's really only an issue if you're trying to maximize compute / mm, and that crowd simply doesn't use them, so it's a non-issue. For the rest of us, they're a modest win for software robustness and analyzability of algorithms. – Avaricious

Nothing in practical use supports fast denormals, and for the rest of us, there's a massive loss for software robustness as the performance can massively decrease for seemingly no reason, e.g. when a solver settles onto values that are small, and for some reason it doesn't have denormals to zero turned on. Recall that normal floats store the leading 1 implicitly, and denormals store it explicitly. It's two essentially unrelated number representations within one data type, so the way it is handled is executing far slower microcode for denormals. – Shuster

Ohh and having multiple different modes is bad as code can easily end up running under an unintended mode. With DAZ/FTZ you have cases where result of subtraction between two different very small floats is zero but floats are different in their lower bits, with standard denormals that doesn't happen, which I guess could theoretically be helpful but in practice if an algorithm will end up relying on this property then it'll break when running under FTZ/DAZ, e.g. when you have to run it a lot and need performance. A standard that is impractical to adhere to leads to deviations from standard. – Shuster

@Dmytry: "Nothing in practical use supports fast denormals"--hundreds of millions of iPhones support denormals at full speed, to name just one. Are they not in practical use? Cellphones can handle denormals at speed, and there's no real obstacle to other CPUs doing so. – Avaricious

@Stephen Canon: Are you sure? google.com/search?q=iphone+denormal+float+performance Ultimately the denormals give a slightly better floating point but a far worse standard (i.e. a popular implementation practice remains non standardized). Standards bodies should standardize existing practice when such practice exists. – Shuster

@Shuster Yes, I’m sure. That answer is from 2012, and specific to armv7 ios devices. All 64b ios devices default to running with denormals enabled, and have no slowdown associated with denormals as of a few years ago. (The first generation of 64b ios devices had a small performance penalty). – Avaricious

Your assertion that the OP's "that doesn't mean that the correct answer is false" is wrong is rather blunt, and I don't think you've given a convincing argument for it. Your reasoning is "The predicate (y < x) asks whether y is less than x. If y is NaN, then it is not less than any floating-point value x, so the answer is necessarily false." This is no more compelling to me than the statement you're refuting. – Leanaleanard

"To be blunt: the result of NaN == NaN isn’t going to change now. Better to learn to live with it than to complain on the internet." That comes across as condescending, and it contributes nothing to your answer. How about removing it? – Leanaleanard

@DonHatch I neither intend, nor read it as condescending, but you are certainly free to make an edit. – Avaricious

If you had NaN == NaN and no IsNaN, you could also test for NaN with simply x == NaN. – Riba

@mm201: the concern was that existing programming languages had no “NaN” literal, so there was no way (at the time) to write ‘x == NaN’, but you could write ‘x != x’. This was short-sighted, but not totally unreasonable—it arose from 754’s origins as one specific arithmetic (the 8087) that only later became a language standard. If it were a language standard from day one, it likely would have been different. – Avaricious

@StephenCanon wouldn't that be easily circumvented by checking x == (0/0)? Even if you don't have a word for NaN, generating one should be easy. – Oneidaoneil

@JSQuareD: The context of IEEE 754's origins as the 8087 arithmetic matters here; at the time that it was introduced, there were other floating-point arithmetics in use, some of which would trap on 0/0. x != x could be used as a check for NaN that was safe on these other arithmetics as well (it might always return true, but it wouldn't trap). – Avaricious

Does this mean that the decision to have NaN != NaN hold true is specific to floating-point arithmetic, and that it might be totally reasonable for say, a custom integer arithmetic implementation that also includes NaN, to allow NaN == NaN to hold true, as long as a constant for NaN is provided? – Yoong

Answer 2

NaN can be thought of as an undefined state/number. similar to the concept of 0/0 being undefined or sqrt(-3) (in the real number system where the floating point lives).

NaN is used as a sort of placeholder for this undefined state. Mathematically speaking, undefined is not equal to undefined. Neither can you say an undefined value is greater or less than another undefined value. Therefore all comparisons return false.

This behaviour is also advantageous in the cases where you compare sqrt(-3) to sqrt(-2). They would both return NaN but they are not equivalent even though they return the same value. Therefore having equality always returning false when dealing with NaN is the desired behaviour.

Ritter

What should be the result of sqrt(1.00000000000000022)==sqrt(1.0)? How about (1E308+1E308-1E308-1E308-1E308)==(1E308+1E308)? Also, only five of the six comparisons return false. The != operator returns true. Having NaN==NaN and NaN!=NaN both return false would allow code that compares x and y to choose what should happen when both operands are NaN by choosing either == or !=. – Svoboda

Just here to point out that 0/0 is (unexpectedly) equal to NaN – Dribble

Answer 3

To throw in yet another analogy. If I hand you two boxes, and tell you that neither of them contains an apple, would you tell me that the boxes contain the same thing?

NaN contains no information about what something is, just what it isn't. Therefore these elements can never definitely be said to be equal.

Rickety

All empty sets are equal, by definition. – Miscegenation

The boxes you are given are NOT known to be empty. – Typhoon

Would you tell me the boxes don't contain the same thing? I can understand the rationale for (NaN==Nan)==false. What I don't understand is the rationale for (Nan!=Nan)==true. – Svoboda

I assume NaN != NaN is true because x != y is defined as !(x == y). Granted, I don't know if the IEEE spec defines it that way. – Jessjessa

But in this analogy, if you gave me a box, said that it didn't contain apples, then asked me if it was equal to itself, you expect me to say no? Because that is what I would have to say according to IEEE. – Morales

@Morales Asking if the box is itself seems like a different question to me. e.g. (from clojure) (def a Double/NaN) (== a a) ; false (identical? a a) ; true – Reprint

@DavidMeister I mean sure. But a = NaN a == a returns False in basically every language, and has to according to IEEE. – Morales

@Morales I think IEEE makes enough sense with this metaphor, because == is an assertion that means "the value of these two things is definitely the same". Not, "these two things refer to the same object" (they are the same box) or "these two things may have the same value". As NaN indicates that the value is unknowable or missing, == must be false. – Reprint

I mean two NaN values are completely indistinguishable from each other. So it seems pretty reasonable to say they are equal, seeing as it is literally impossible to prove otherwise. And ruining the reflexitivity of == for something as questionable as that is IMO very stupid. For one it makes a lot of potential optimizations (such as pointer equality for shortcutting equality checking of large objects) unsound, and it just makes polymorphic reasoning very difficult, as polymorphic equality's reflexitivity law can no longer be utilized without a lot of extra care. – Morales

@Morales i suppose it depends how you look at it. As has been pointed out by other people here, you can either say: "if it quacks like a NaN, etc..." or you can say "NaN represents the result of something impossible, which isn't a result at all so equality itself is meaningless so the assertion of equality is necessarily false". You said it is "literally impossible to prove otherwise" - if you only look at the NaN in isolation as a value what you said is true, but if you are allowed to peek at the calculation (e.g. 0/0 vs. ∞/∞) that produced the NaN then it is false. – Reprint

@Morales OTOH, why is Infinity == Infinity true? – Reprint

@DavidMeister I mean I guess, but programming language don't generally rewind time when checking for equality. When using a function or operator you generally look at the values passed in as values in isolation. I would support a special equality operator designed specially for the mathematical meaning behind floats, and have it be monorphic, maybe going as far as rounding to some precision when equality checking, idk. But polymorphic == IMO should be reflexive, symmetric and transitive. – Morales

@DavidMeister By monomorphic I guess just limit it to floats, so polymorphic between Float and Double etc. is fine. I just like my polymorphic operators and functions to be possible to reason about, and not too reliant on knowing the exact type I am dealing with at all times. – Morales

@Morales it probably would be nice if things worked that way, but equality weirdness isn't limited to NaN, e.g. #5447653 and https://mcmap.net/q/24498/-is-javascript-39-s-double-equals-always-symmetric – Reprint

@Morales incidentally, NaN does support a payload that in theory can be used to encode the reason for failure (e.g. 0/0 vs ∞/∞) - #33968304 – Reprint

@DavidMeister I mean this is a language agnostic question. So showing me a thread showing why JS has weird equality (I know, I hate JS, and that is one of many many reasons why), doesn't really change anything. The specific language I typically use, Haskell, has a much more sane == EXCEPT for this NaN bs. Pretty much all == instances (as in the implementations for each possible type) are reflexive, symmetric, and transitive, and generally structural (a == b => f a == f b) ignoring certain deep inspection functions (such as showTree on a binary tree representing a set). – Morales

@DavidMeister I would not necessarily be against actually using the NaN payload for things from equality to error messages etc. But I think reflexivity is a very important aspect of any == implementation. x == x should either not-typecheck / error (e.g: functional equality is undecidable so sum == sum erroring is understandable, which is what Haskell does) or it should definitely return True. – Morales

@Morales well, just as much as the question isn't about whether equality make sense in specific languages in general, it's not really about whether NaN != NaN is considered beneficial or "most correct" in 2017 either... it's just about what the original rationale was back in 1985, which the currently accepted answer seems to cover quite well. The original rationale did consider language specific issues very important. They wanted something that would work "well enough" even if languages completely failed to implement the spec properly (by providing a robust isNaN implementation). – Reprint

@Morales and i present javascript once more as a reason why concerns that a popular, mainstream language might fail to get isNaN right for "several years while the world waited" are not completely unjustified https://mcmap.net/q/24500/-is-number-isnan-more-broken-than-isnan. Looks like isNaN was broken for something like 15 years in JS and a !== a was the only reliable solution for that entire time... – Reprint

@DavidMeister I understand the original rationale and don't really have a whole lot of complaints with it. I am just explaining why I think it should be fixed now in newer version of languages, reflexive == is really nice to have, while it's true that floats break a lot of laws, most of them are only broken "slightly", and optimizations assuming they exist will almost never cause real problems, but this is totally different and plenty of optimizations are unsafe under NaN != NaN. such as shortcutting equality by comparing pointers first. – Morales

@DavidMeister And as I said JS does A LOT of things wrong, isNaN works just fine in Haskell for example, it's polymorphic on all RealFloat (e.g Double, Float) values, and will give you the result you want. – Morales

Answer 4

From the wikipedia article on NaN, the following practices may cause NaNs:

• All mathematical operations> with a NaN as at least one operand
• The divisions 0/0, ∞/∞, ∞/-∞, -∞/∞, and -∞/-∞
• The multiplications 0×∞ and 0×-∞
• The additions ∞ + (-∞), (-∞) + ∞ and equivalent subtractions.
• Applying a function to arguments outside its domain, including taking the square root of a negative number, taking the logarithm of a negative number, taking the tangent of an odd multiple of 90 degrees (or π/2 radians), or taking the inverse sine or cosine of a number which is less than -1 or greater than +1.

Since there is no way to know which of these operations created the NaN, there is no way to compare them that makes sense.

Hamburg

Moreover, even if you knew which operation, it wouldn't help. I can construct any number of formulas which go to 0/0 at some point, which have (if we assume continuity) well-defined and different values at that point. – Cayla

Answer 5

I don't know the design rationale, but here's an excerpt from the IEEE 754-1985 standard:

"It shall be possible to compare floating-point numbers in all supported formats, even if the operands' formats differ. Comparisons are exact and never overflow nor underflow. Four mutually exclusive relations are possible: less than, equal, greater than, and unordered. The last case arises when at least one operand is NaN. Every NaN shall compare unordered with everything, including itself."

Veliavelick

Answer 6

It only looks peculiar because most programming environments that allow NaNs do not also allow 3-valued logic. If you throw 3-valued logic into the mix, it becomes consistent:

• (2.7 == 2.7) = true
• (2.7 == 2.6) = false
• (2.7 == NaN) = unknown
• (NaN == NaN) = unknown

Even .NET does not provide a bool? operator==(double v1, double v2) operator, so you are still stuck with the silly (NaN == NaN) = false result.

Lapoint

Answer 7

I'm guessing that NaN (Not A Number) means exactly that: This is not a number and thus comparing it does not really make sense.

It's a bit like arithmetic in SQL with null operands: They all result in null.

The comparisons for floating point numbers compare numeric values. Thus, they can't be used for non numeric values. NaN therefore cannot be compared in a numeric sense.

Positively

"This is not a number and thus comparing it does not really make sense." Strings are not numbers but comparing them makes sense. – Minacious

yes, comparing a string to a string makes sense. But comparing a string to, say, apples, does not make much sense. Since apples and pears are not numbers, does it make sense to compare them? Which is greater? – Positively

@DarenThomas: In SQL, neither "IF NULL=NULL THEN FOO;" nor "IF Null<>Null THEN CALL FOO;" [or whatever the syntax is] will execute FOO. For NaN to be equivalent if (NaN != NaN) foo(); shouldn't execute foo, but it does. – Svoboda

@DarenThomas comparing strings to apples does make sense when you want to sort them, for example! – Octastyle

Answer 8

MeToo came here to understand the reasoning, why NaN == NaN equals false.

After reading (nearly) all I still was puzzled, why a == NaN cannot replace a function like isNaN(), because it seems to be so obvious.

But things are not that simple.

Nobody has mentioned vector geometry yet. But many computations take place in the 2nd or 3rd dimension, so in vector space.

After thinking about this a bit, I immediately realized, why it is a good thing to have NaN not to compare to itself. Following hopefully is easy enough to understand for others, too.

Vectors

Bear with me, it takes a while until NaN shows up.

First let me explain a bit for people who are not deep inside math

In vector geometry we usually use something like complex numbers.

A complex number is made of two floats a + bi (where i denotes the imaginary value with i * i == -1) which allow us to address all points on the 2 dimensional plane. With floating point we cannot express each value, so we have to approximate a bit. So if we round the values to some value we can express, we can still try to create numerically stable algorithms, which give us some good approximation of what we want to archive.

Enter infinity

No NaN here yet. Please be patient. I'll get to the point later down below.

If we want to specify some point far far away, we might leave the range of numbers we can express, which results in infinity. In IEEE floats we luckily have +inf (I write it as inf) or -inf for this (written as -inf).

This is good:

a + inf i makes sense, right? It is the vector to some point on the x-axes at location a and on the y-axes at location "positive infinity". But wait a bit, we are talking vectors here!

Vectors have an origin and a point they point to. Normalized vectors are those, which start at location (0,0).

Now think of a vector with origin of (0,0) which points to (a,inf).

Still makes sense? Not quite. As we look a bit closer, we will see, that the normalized vector (0,inf) is the same vector! As the vector is so long, the derivation of a in the infinty can no more be seen. Or said otherwise:

For infinitively long vectors in the cartesian coordinate system, the finite axis can be expressed as 0, because we are allowed to approximate (if we are not allowed to approximate, we cannot use floating point!).

So the replacement-vector (0,inf) is still suitable. In fact, any (x,inf) is a suitable replacement for a finite x. So why not use 0 from our origin of our normalized vector.

Hence what do we get here? Well, with allowing inf in our vectors, we actually get 8 possible infinite vectors, each 45 degrees rotated (degrees in parentheses):

(inf,0) (0), (inf,inf) (45), (0,inf) (90), (-inf,inf) (135), (-inf,0) (180), (-inf,-inf) (225), (0,-inf) (270) and (inf,-inf) (315)

All this does not cause any trouble. In fact, it is good to be able to express more than just finite vectors. This way we have a natural extension of our model.

Polar coordinates

Still no NaN here, but we are getting closer

Above we used complex numbers as cartesian coordinates. But complex numbers also have a 2nd option how we can write them. That is polar coordinates.

Polar coordinates are made up of a length and an angle like [angle,length]. So if we transform our complex number into polar coordinates, we will see, that we can express a bit more than just 8 angles in [angle,inf].

Hence, if you want to create a mathematical model which allows infinitely long vectors in some multidimensional space, you definitively want to use polar coordinates in your calculation as much as you can.

All you have to do for this is to convert the cartesian coordinates into the polar ones and vice versa.

How to do this is left as exercise for the reader.

Enter NaN

Now, what do we have?

• We have a mathematical model which calculates with polar coordinates.
• And we have some output device, which uses cartesian coordinates, probably.

What we now want to do is to be able to convert between those two. What do we need for this?

We need floating point, of course!

And as we perhaps need to calculate with some few terabillion coordinates, (perhaps we render some weather forecast or have some collision data from the large hadron collider) we do not want to include slow and error prone error processing (WTF? Error prone error processing? You bet!) in all those complex mathematical (hopefully numerically stable) steps.

How do we propagate errors then?

Well, as said by IEEE: We use NaN for error propagation

So what we have up to here?

• Some calculation in the polar coordinate space
• Some conversion into cartesian space
• NaN as rescue if something fails

And this then leads to ..

.. why NaN == NaN must be false

To explain this, let's reduce this complex stuff above all to a simple result of 2 vectors in cartesian coordinates:

• (a,b) and (c,d)

And we want to compare those two. This is how this comparison looks like:

• a == c && b == d

Everything correct so far?

Yes. But only until we observe following two polar vectors which might be the source of our two cartesian vectors:

• [NaN,inf] and [0,NaN]

Certainly those two vectors are not equal in the polar coordinate space. But after conversion into cartesian space, both come out as:

• (NaN,NaN) and (NaN,NaN)

Well, should they suddenly compare equal?

Surely not!

Thanks to IEEE defining that NaN == NaN must return false, our very primitive vector comparison still gives us the expected result!

And I think, that exactly is the motivation behind, why IEEE defined it as it is.

Now we have to live with this mess. But is it a mess, indeed? I'm undecided. But, at least, I now can understand the (probable) reasoning.

Hopefully I did not miss something.

Some last words

The primitive way of comparing things usually is not fully appropriate when it comes to floating point numbers.

In floating point, you usually do not use ==, you rather use something like abs(a-b) < eps with eps being some very small value. This is because already something like 1/3 + 1/3 * 2.0 == 1.0 might not be true, depending on which hardware you run.

1/3 + 1/3 * 2.0 == 1/3 + 1/3 + 1/3 should be true on all reasonable hardware. So even == can be used. Only carefully. But is not ruled out.

However this does not render above reasoning void. Because above is not a mathematically proof for that the IEEE is right. It is just an example, which should allow to understand the source of the reasoning behind, and why it is probably better to have it defined the way it is.

Even that it is a PITA for all programming people like me.

Camion

Answer 9

The over-simplified answer is that a NaN has no numeric value, so there is nothing in it to compare to anything else.

You might consider testing for and replacing your NaNs with +INF if you want them to act like +INF.

Thaumatology

Answer 10

While I agree that comparisons of NaN with any real number should be unordered, I think there is just cause for comparing NaN with itself. How, for example does one discover the difference between signaling NaNs and quiet NaNs? If we think of the signals as a set of Boolean values (i.e. a bit-vector) one might well ask whether the bit-vectors are the same or different and order the sets accordingly. For example, on decoding a maximum biased exponent, if the significand were left shifted so as to align the most significant bit of the significand on the most significant bit of the binary format, a negative value would be a quiet NaN and any positive value would be a signaling NaN. Zero of course is reserved for infinity and the comparison would be unordered. MSB alignment would allow for the direct comparison of signals even from different binary formats. Two NaNs with the same set of signals would therefore be equivalent and give meaning to equality.

Gibbet

Answer 11

Because mathematics is the field where numbers "just exist". In computing you must initialize those numbers and keep their state according to your needs. At those old days memory initialization worked in the ways you could never rely on. You never could allow yourself to think about this "oh, that would be initialized with 0xCD all the time, my algo will not broke".

So you need proper non-mixing solvent which is sticky enough to not not letting your algorithm getting sucked into and broken. Good algorithms involving numbers are mostly going to work with relations, and those if() relations will be omitted.

This is just grease which you can put into new variable at creation, instead of programming random hell from computer memory. And your algorithm whatever it is, will not break.

Next, when you still suddenly finding out that your algorithm is producing NaNs, it is possible to clean it out, looking into every branch one at a time. Again, "always false" rule is helping a lot in this.

Studding

Answer 12

Very short answer:

Because the following: nan / nan = 1 must NOT hold. Otherwise inf/inf would be 1.

(Therefore nan can not be equal to nan. As for > or <, if nan would respect any order relation in a set satisfying the Archimedean property, we would have again nan / nan = 1 at the limit).

Spillage

No, that doesn't make sense. We have inf = inf and inf / inf = nan, so nan = nan won't prevent nan / nan = nan either. – Benis

@Benis You mean nan / nan = 1? Anyway... Your reasoning does make sense if inf and nan were just as any other numbers. It is not the case. The reason why inf/inf must be nan (or indeterminate form in math) and not 1 is more subtle than simple algebraic manipulation (see De L'Hospital theorem). – Spillage