Minimum and maximum of signed zero

Question

I am concerned about the following cases

min(-0.0,0.0)
max(-0.0,0.0)
minmag(-x,x) 
maxmag(-x,x)

According to Wikipedia IEEE 754-2008 says in regards to min and max

The min and max operations are defined but leave some leeway for the case where the inputs are equal in value but differ in representation. In particular:

min(+0,−0) or min(−0,+0) must produce something with a value of zero but may always return the first argument.

I did some tests compare fmin, fmax, min and max as defined below

#define max(a,b) \
   ({ __typeof__ (a) _a = (a); \
       __typeof__ (b) _b = (b); \
     _a > _b ? _a : _b; })
#define min(a,b) \
   ({ __typeof__ (a) _a = (a); \
       __typeof__ (b) _b = (b); \
     _a < _b ? _a : _b; })

and _mm_min_ps and _mm_max_ps which call the SSE minps and maxps instruction.

Here are the results (the code I used to test this is posted below)

fmin(-0.0,0.0)       = -0.0
fmax(-0.0,0.0)       =  0.0
min(-0.0,0.0)        =  0.0
max(-0.0,0.0)        =  0.0
_mm_min_ps(-0.0,0.0) =  0.0
_mm_max_ps(-0.0,0.0) = -0.0

As you can see each case returns different results. So my main question is what does the C and C++ standard libraries say? Does fmin(-0.0,0.0) have to equal -0.0 and fmax(-0.0,0.0) have to equal 0.0 or are different implementations allowed to define it differently? If it's implementation defined does this mean that to insure the code is compatible with different implementation of the C standard library (.e.g from different compilers) that checks must be done to determine how they implement min and max?

What about minmag(-x,x) and maxmag(-x,x)? These are both defined in IEEE 754-2008. Are these implementation defined at least in IEEE 754-2008? I infer from Wikepdia's comment on min and max that these are implementation defined. But the C standard library does not define these functions as far as I know. In OpenCL these functions are defined as

maxmag Returns x if | x| > |y|, or y if |y| > |x|, otherwise fmax(x, y).

minmag Returns x if |x| < |y|, or y if |y| < |x|, otherwise fmin(x, y).

The x86 instruction set has no minmag and maxmag instructions so I had to implement them. But in my case I need performance and creating a branch for the case when the magnitudes are equal is not efficient.

The Itaninum instruction set has minmag and maxmag instructions (famin and famax) and in this case as far as I can tell (from reading) in this case it returns the second argument. That's not what minps and maxps appear to be doing though. It's strange that _mm_min_ps(-0.0,0.0) = 0.0 and _mm_max_ps(-0.0,0.0) = -0.0. I would have expected them to either return the first argument in both cases or the second. Why are the minps and maxps instructions defined this way?

#include <stdio.h>
#include <x86intrin.h>
#include <math.h>

#define max(a,b) \
   ({ __typeof__ (a) _a = (a); \
       __typeof__ (b) _b = (b); \
     _a > _b ? _a : _b; })

#define min(a,b) \
   ({ __typeof__ (a) _a = (a); \
       __typeof__ (b) _b = (b); \
     _a < _b ? _a : _b; })
   
int main(void) {
    float a[4] = {-0.0, -1.0, -2.0, -3.0};   
    float b[4] = {0.0, 1.0, 2.0, 3.0};
    __m128 a4 = _mm_load_ps(a);
    __m128 b4 = _mm_load_ps(b);
    __m128 c4 = _mm_min_ps(a4,b4);
    __m128 d4 = _mm_max_ps(a4,b4);
    { float c[4]; _mm_store_ps(c,c4); printf("%f %f %f %f\n", c[0], c[1], c[2], c[3]); }
    { float c[4]; _mm_store_ps(c,d4); printf("%f %f %f %f\n", c[0], c[1], c[2], c[3]); }
    
    printf("%f %f %f %f\n", fmin(a[0],b[0]), fmin(a[1],b[1]), fmin(a[2],b[2]), fmin(a[3],b[3]));
    printf("%f %f %f %f\n", fmax(a[0],b[0]), fmax(a[1],b[1]), fmax(a[2],b[2]), fmax(a[3],b[3]));

    printf("%f %f %f %f\n", min(a[0],b[0]), min(a[1],b[1]), min(a[2],b[2]), min(a[3],b[3]));
    printf("%f %f %f %f\n", max(a[0],b[0]), max(a[1],b[1]), max(a[2],b[2]), max(a[3],b[3]));    
}
//_mm_min_ps: 0.000000, -1.000000, -2.000000, -3.000000
//_mm_max_ps: -0.000000, 1.000000, 2.000000, 3.000000
//fmin: -0.000000, -1.000000, -2.000000, -3.000000
//fmax: 0.000000, 1.000000, 2.000000, 3.000000
//min: 0.000000, -1.000000, -2.000000, -3.000000
//max: 0.000000, 1.000000, 2.000000, 3.000000

Edit:

In regards to C++ I tested std::min(-0.0,0.0) and std::max(-0.0,0.0) and the both return -0.0. Which shows that that std::min is not the same as fmin and std::max is not the same as fmax.

Byelostok

Don't you think your question deserves a better title? – Yggdrasil

I'm not sure this question belongs here actually. It is more a discussion that a particular question. And the answer why CPU instructions have been implemented either way is deefinitively better placed at the developer - Intel. There are actually three different questions included. Would that not better be split? – Vigen

While its an interesting question, I wonder in what situation it would actually matter whether a function returns +0.0 or -0.0. – Prisca

@Olaf, I think it's a relevant question. After five years on SO nobody seemed point out that min(-0.0,0.0) can return 0.0 or -0.0. – Byelostok

@MikeMB, it matters for me in double-double. See njuffa's comment here "As I recall it is very important that famax() and famin() are used in such a way that when |a|==|b| the further computation continues using both a and b which may be of opposite sign. So for TwoSum() a.k.a. add12() you would want something like this: s=a+b; x=famax(a,b); y=famin(b,a); e=(x-s)+y; return (e, s); Note the argument swap between the calls to famax() and famin()" – Byelostok

I don't really understand what you mean here: If a and b are both +/-0.0, then e and s will always be 0 too, no? – Prisca

@MikeMB, good point. So it does not matter (in this case) for min(-0.0,0.0) and max(-0.0,0.0) but it matter for minmag(-x,x) and maxmag(-x,x). – Byelostok

Also, no one has asked how to make a shepherd's pie. There might be a good reason why no one has asked that. – Vigen

So they would return zero. Why does it matter, whether they return plus or minus zero? As I said, I find it interesting from an academic point of view and I also think its a valid question for SO, but I can't imagine a standard conforming, bugfree C program, for which this is relevant. Or formulated differently: I would be curious to see such a program, because I've only limited experience with large scale production code, so my imagination might be a little limited. – Prisca

@Prisca I mean that e.g. minmag(-3,3) and maxmag(-3,3). I agree I don't have an example where the sign of min(-0.0,0.0) matters. I originally wanted to make this question about minmag and maxmag but these are no in the C/C++ standards. They are defined in IEEE and OpenCL defines them. – Byelostok

@MikeMB, what about dividing my zero. If you did something like x/min and y/max and you wanted use use -infinity and +infinity then this would matter. I don't have any code doing this. – Byelostok

@Zboson: Not sure the +/- 0 case matters nearly as much as the +/- 3 case. Your results are obviously garbage if you compute 3 + (-3) = 6. (I'm not sure what relevance the sign bit of nothing has in double-double computation.) – Raucous

@MikeMB: Kahan has an interesting paper/rant called "Branch Cuts for Complex Elementary Functions, or Much Ado About Nothing's Sign Bit." One point it makes is that, since the imaginary part of a real written as a complex number can have either sign bit, there need not be any complex number for which sqrt isn't defined; sqrt(-1.0 + 0.0i) can be +i and sqrt(-1.0 - 0.0i) can be -i. This is actually how C99's csqrt works. Another point the paper makes is that this "usually makes programs work better." – Raucous

@Raucous yeah +/-3 matters for me and I'm not sure +/-0 does. But more people are familiar with min and max and the logic I used is the same. I mean minmag(-x,x) and maxmag(-x,x) can be defined to return -x,-x or x,x or -x,x or x,-x just like the various min/max definitions I used returned 0,0, or -0,-0 or 0,-0. – Byelostok

So your question is, why there are library functions and assembler instructions, that perform almost the same functionality, but differ in the handling of corner cases? I'd guess you would have to ask the designers for that. Things I can imagine are different requriements, different implementation constraints, historical reasons or the designers just didn't know or didn't care what the other group did. Btw. division by zero is undefined behavior, so if you use that you are in the world of non-portable code anyway. – Prisca

@MikeMB, my question is about whether or not there is implementation defined behavior in IEEE and if the C/C++ standards add further constratins so that these corner cases are not implementation defined (OpenCL's min(max)mag(a,b) definition's handle the corner case of a=-b). This is basically a couple yes/no answers but I was hoping somebody could elaborate on that since I just learned about it. Min/max is not a good example since -0,0 does not seem to matter but maybe there are other functions that do (and the min(max)mag functions are not in the C/C++ standards). – Byelostok

@Prisca Concerning "what situation it would actually matter whether a function returns +0.0 or -0.0", see https://mcmap.net/q/23120/-what-operations-and-functions-on-0-0-and-0-0-give-different-arithmetic-results/2410359 – Euphrosyne

@chux: I see some divisions by zero (UB in C) and functions that are explicitly designed to extract the sign. I know, you can determine if a value is +0.0 or -0.0, but I just can't envision a sane program for which that makes a logical difference. – Prisca

@Prisca C, which does not require -0 is not likely to have any specifications about function/operator results with -0 input. atan2(+/-zero, -1.0) usefully returns +machine_π /2 or -machine_π /2. That is not, of course, define by C, but IEEE 754. -0 is useful in noting profit/loss (print in black/red) as total, like $-10 may be rounded to its nearest $1,000, but still need to convey a loss. Any need for -0 will certainly be a niche issue. – Euphrosyne

@chux: That's indeed something I didn't consider, although I really hope those guys don't use floating point numbers to handle currency. atam2 returns + or - PI. I would expect those values to be handled equally anyway, but that might not always be the case (by accident or by design). – Prisca

@MikeMB, atan2(+-0, -1.0) is interesting (and is defined in C/C++). The range of principle values is (-pi,pi] but the range of atan2 is [-pi,pi] so it includes one value, -pi, from another branch due to -0. In some sense my question is related to multi-value functions such as y = +-sqrt(x) e.g. Min/maxmag(x,-x) = +-x. So it's a question of which branch to take. I should not have made my question about C/C++. I am really interested in how to handle multi-value functions in IEEE 754. – Byelostok

Answer 1

Why not read the standard yourself? The Wikipedia article for IEEE contains links to the standard.

Note: The C standard document is not available freely. But the final draft is (that's what I linked, search to find the pdf version). However, I've not seen the final document being cited here and AFAIK there had mostly been some typos corrected; nothing changed. IEEE is, however, available for free.

Note that a compiler need not stick to the standards (some embedded compilers/versions for instance do not implement IEEE-conforming floating point values, but are still C-conforming - just read the standard for details). So see the compiler documentation to see the compatibility. MS-VC for instance is not even compatible to C99 (and will never ben), while gcc and clang/llvm are (mostly) compatible to C11 in the current versions (gcc since 4.9.2 at least, in parts since 4.7).

In general, when using MS-VC, check if it actually does support that all standard features used. It is actually not fully compliant to the current standard, nor C99.

Vigen

I just search through the draft it and a footnote says "Ideally, fmax would be sensitive to the sign of zero, for example fmax(-0.0, +0.0) would return +0; however, implementation in software might be impractical." – Byelostok

Are fmin and fmax defined in C++? – Byelostok

This is for C. C++ is a different language. Please find the answer yourself or open a new question. (please serach yourself first) – Vigen

I think many people are familiar with both languages and they overlap far more than they disagree. Since they agree more then they disagree isn't it a bit pedantic to say they are different when in most cases they the same. Doing a bit of search shows fmin and fmax are define exactly the same in both languages but I'm not expert. – Byelostok

I will not discuss this issue here. Just note that they differ in much more places some "experts" might think (according to the C++ questions with C tag) - and, yes, I do know both. Here, the differerentiation is well accepted and enforced- unless you can prove the opposite. Note: just that you can program C-like in C++ does not make them "almost" identical. That would have to be bijective for most features. So, leave it at that! – Vigen

@I don't want to debate this either. I should have used the C++ tag as well I guess. – Byelostok

Not at all. Just think how overloading might complicate your problem. – Vigen

Well, I just added the C++ tag and a test with std:min(-0.0,0.0) and std::max(-0.0,0.0) and they both return -0.0. – Byelostok

MSVC is catching up with C99 see What is the official status of C99 support in VS2013? for the details – Keijo

@ShafikYaghmour: I read somewhere else different: MS stated they will not bring VC to C99-compatibility. However, even if: C11 is already about 4 years old now, with the feature set fixed much longer. Congratulations, MS; i expect C11 support by 2030 then - if I would care. – Vigen

"it is a good idea not to use MS-VC to compile modern C code". Er... Why? We routinely do exactly that on everyday basis. Modern MS VC support for C99 is rather extensive, at least in core language features department. Where does that "and will never be" come from? – Busk

@AnT: If you call missing length modifiers (hh) and other flaws "extensive", well, then. However, I would not call a withdrawn, more than 16 year old standard not "modern", but refer mostly to C11. Not sure what you mean by "core language featues", but the standard for a hosted environment does not differentiate about "core features" (except for some minor, optional features). However, I changed that to a warning. – Vigen

@Olaf: By core language features I mean what's described by "Language" section of the standard (as opposed to its "Library" section). – Busk

@AnT: as the library is a vital part of the standard and it does not differentiate between those, I would assume I am right. Unless you are refering to a freestanding environment, of course. For the language features, I only found this 2012 article. To me it reads as if VC is more of a C++ compiler with some "compatilbility layer" to C. but not native C support. However, I alrady have edited my answer; If I do state something wrong, please correct me (with proper proof; "it works for me/us" is actually none). – Vigen

@Olaf: Firstly, the language standards for both C and C++ have always been clearly separated into two large distinctive "halves": [Core] Language and [Standard] Library. Nobody's trying to diminish the importance of standard library here, but the qualitative distinction between core language features and library features has always been there. It is a very major distinction in the design of both languages. Secondly, MSVC has never been "C++ compiler with C compatibility layer". MSVC has always supported C natively with a very dedicated pure C compiler. – Busk

@AnT: I used " around the "compatibility layer" intentionally. According to the text I linked, they concentrate on C++ and have no actualy interest about implementing full compatibility. For the seperation between "core" and library": While you are right, they are seperate sections in the standard document (for good reasons), none is optinal for standard compliance (I'll leave freestanding environments aside here). Stange enough a google search for "visual-c c11 compliance" does not even pop up a MS-site about C11, only for C++11 compliance. – Vigen

Answer 2

The fundamental issue in this case is the actual underlying mathematics, ignoring representational issues. There are several implications in your question that I believe are erroneous. -0.0 < 0.0 is false. -0.0 is a negative number is false. 0.0 is a positive number is false. In fact, there's no such thing as -0.0, though there is an IEEE 754 representation of zero with a sign bit set.

In addition, the behavior of min/max functions is only a small slice of legal floating-point operations that can yield zeros with different sign bits. Since floating point units are free to return (-)0.0 for expressions like -7 - -7, you'd also have to figure out what to do with that. I'd also like to point out that |0.0| could in fact return 0.0 with the sign bit set, since -0.0 is an absolute value of 0.0. Put simply, as far as mathematics is concerned 0.0 is -0.0. They are the same thing.

The only way that you can test for 0.0 with a set sign bit is to abandon mathematical expressions and instead examine the binary representation of such values. But what's the point of that? There's only one legitimate case I can think of: the generation of binary data from two different machines that are required to be bit-for-bit identical. In this case, you'll need to also worry about signaling and quiet NaN values, since there are very many more aliases of these values (10^22-1 SNaN's and 10^22 QNaN's for single-precision floats, and about 10^51 values of each for double-precision).

In these situations where binary representation is critical (it's absolutely NOT for mathematical computation), then you'll have to write code to condition all floats on write (zeros, quiet NaN's, and signaling NaN's).

For any computational purpose, it's useless to worry about whether the sign bit is set or clear when the value is zero.

Radioscopy

You're forgetting operations that are mathematically undefined on zero, but have separate limits when approaching zero from the positive end and when approaching it from the negative end. This applies to some of the standard math library functions, and to a lesser extent to division: in floating-point arithmetic, division by zero produces infinity, with a sign depending on that of the zero. – Daphie

Agree, "-0.0 is a negative number is false" is supported by sqrt(-0.0) does not cause an exception, per IEEE 754. I think it even returns -0.0. (zero with sign bit set.). Concerning "computational purpose, it's useless to worry about whether the sign bit is set or clear", consider What operations and functions on +0.0 and -0.0 give different arithmetic results? – Euphrosyne

@hvd: As far as I know, division by zero is undefined behavior in c and c++. – Prisca

@Prisca Yes and no. It's explicitly undefined for both integer and floating point division in the mandatory parts of the specification, but it gets defined for floating point division in an optional part of the specification, and that optional part of the specification has a feature detection macro. Because of that, it's possible for a strictly conforming program that uses that feature detection macro to divide by zero. – Daphie

@hvd, what feature detecting macro are you referring to? If it's __STDC_IEC_559__ it does not seem very useful because GCC does not define this as far as I know. I think I'm ignorant about UB but if GCC defines floating point division by zero is it really UB? I mean It's UB by the C standard by not the compiler. – Byelostok

The underlying mathematics here is floating point arithmetic with signed zero. It's just as much math as the real analysis with infinite range you grew up with. It has it's own set of rules. For example floating point arithmetic is not associative. Signed zero has meaning in these rules (e.g. atan(+-0,-1) = +-pi. Incidentally, computer integer arithmetic is also not infinite range and has it's own set of rules based on modular arithmetic. – Byelostok

@Zboson Yes, that's the one. GCC doesn't define it, but glibc does. (It doesn't -- or didn't -- do so entirely correctly, but that's a different issue.) GCC does define its own not-entirely-identical extensions to the C standard that still support division by zero, which applies to other platforms. Whether it's appropriate to say that code using a supported extension of a compiler has UB is a reasonable question. As you hint at, it depends on what you mean by UB. No behaviour is defined by the standard. Behaviour is defined by the implementation. – Daphie

@hvd, what include do I need from glibc to get __STDC_IEC_559__? Do you have a link about this? Maybe a link that discusses your statement "It doesn't -- or didn't -- do so entirely correctly, but that's a different issue."? – Byelostok

@Zboson It used to be in <bits/features.h>, which is included by <features.h>, which in turn is included by pretty much every standard library header there is. A mailing list thread about the bad definition is here. I know the situation has changed since then, but I'm not entirely aware of how it has changed. – Daphie

@hvd, thank you. I'm a bit confused as to what is compliant when glibc defines __STDC_IEC_559__. I mean glibc defines functions but not the arithmetic. So does that mean it's possible the functions such as atan2 are IEEE 754 compliant but arithmetic which does not call a function such as 1/0 may not not conform to IEEE 754? Ideally I would expect that if __STDC_IEC_559__ is defined that all floating point is IEEE 754 compliant but it's not clear to me if this is the case. I guess I should ask a question about this. – Byelostok

@Zboson The standards don't handle that, the standards just see the implementation as a whole, and the __STDC_IEC_559__ macro is supposed to indicate the status of the implementation as a whole. If the compiler conforms and defines __STDC_IEC_559__, but the standard library doesn't, then the implementation as a whole does not conform to the standard. If the library conforms and defines __STDC_IEC_559__, but the compiler doesn't, then the implementation as a whole does not conform to the standard. The compiler and standard library have to work together. – Daphie

@hvd, I understand that. What I mean is when I include <features.h> which defines __STDC_IEC_559__ can I assume that the compiler and the standard library work together? What use is __STDC_IEC_559__ if I can't assume this? – Byelostok

@Zboson You should be able to assume that. In the past, you couldn't, and that was an implementation bug. But like I said, while I know the situation has changed, I don't know exactly how, and I don't know if other bugs still remain. – Daphie

@hvd. I tested __STDC_IEC_559__ with GCC 4.9.2. It's defined even when I don't use any includes. e.g. #ifdef __STDC_IEC_559__ return 1; #else return 0; #endif and then echo $?. I did gcc -O3 test.c. This is strange because it says here that it's not defined. It's also defined for g++. – Byelostok

@Zboson I think that GCC might include one specific system header file without you actually naming it in your source code. glibc may then define the macro in that one specific file. There are various options that you can use to investigate whether that is indeed the case, I'm not sure which are the simplest. You might see it listed in the gcc -E output. One of the -d options (-dD?) can also make sure the #define directives are preserved in the preprocessor output. – Daphie

@hvd, gcc -E shows that the file /usr/include/stdc-predef.h is included. When I look in that file it defines __STDC_IEC_559__. – Byelostok

@Zboson And that's a glibc header file, which explains why you won't find it mentioned in the GCC documentation. – Daphie

@hvd, I made a question about this because I thought it might be interesting to other people status-of-stdc-iec-559-with-modern-c-compilers. If you want to delete most of your comments here I can delete mine as well. – Byelostok