floating-point Questions
2
In rust, is there a version of f32 / f64 that implements Eq?
The only reason I can see for f32 / f64 not implementing Eq is that NaN != NaN.
Potential ways such a type could behave:
The type could...
Chon asked 18/8, 2022 at 20:50
8
Solved
I'm trying to get trim off trailing decimal zeroes off a floating point number, but no luck so far.
echo "3/2" | bc -l | sed s/0\{1,\}\$//
1.50000000000000000000
I was hoping to get 1.5 but some...
Akilahakili asked 4/5, 2015 at 15:21
3
How do I use the SET command in windows for floating point arithmatic. /A stands for arithmetic and %VAR% prints the data of VAR not the name.
For example when I do:
SET /A VAR="2.5+3.1"
ECHO %VA...
Stormy asked 20/9, 2014 at 16:59
1
Solved
I recently tried to read assemblies of the binary of my code and found that a lot of floating-point operations are done using XMM registers and SSE instructions. For example, the following code:
fl...
Deposit asked 11/9 at 11:52
2
Solved
Given a 64-bit floating point type (ieee-754), and a closed range [R0,R1] assuming both R0 and R1 are within the representable range and are not NaN or +/-inf etc.
How does one calculate the number...
Stellite asked 7/9 at 22:27
1
PyTorch 1.12 changed the default fp32 math to be "highest precision", and introduced the torch.set_float32_matmul_precision API, allowing users to specify which precision out of medium, h...
Contention asked 30/7, 2023 at 9:39
1
Consider the following Python code:
from decimal import Decimal
d = Decimal("1.23")
print(f"{d = }, {d.shift(1) = }")
When I execute it in Python 3.12.4, I get this output:
d ...
Jala asked 17/8 at 2:2
2
Solved
I just read this topic (especially the last comments).
Then I was wondering, why we actually need this was of giving the remainder. But it seems, that not many people "on google" were interested ...
Mayday asked 31/10, 2014 at 10:4
2
Solved
I need a function to atomically add float32 values in Go.
This is what came up with based on some C code I found:
package atomic
import (
"sync/atomic"
"unsafe"
"math"
)
func AddFloat32(addr ...
Leadership asked 15/12, 2014 at 20:14
4
Solved
How to extract the decimal number part from a float (float64) in a dataframe? (a very common scenario but I can 't find a solution in StackOverFlow)
Note: be careful with the 196.09, I need 09, not...
Tamathatamaulipas asked 1/8, 2018 at 16:39
4
Solved
Okay so I'm an a fairly annoying situation where I don't have access to typed arrays such as Float32Array, but still need to be able to convert a Javascript number into bytes. Now, an integer I can...
Nocturn asked 10/4, 2013 at 20:3
10
Today, I noticed that when I cast a double that is greater than the maximum possible integer to an integer, I get -2147483648. Similarly, when I cast a double that is less than the minimum possible...
Trihedron asked 8/2, 2009 at 17:27
9
Are there any lossless compression methods that can be applied to floating point time-series data, and will significantly outperform, say, writing the data as binary into a file and running it thro...
Numbersnumbfish asked 25/12, 2011 at 17:18
8
Anyone have a recommendation on a good compression algorithm that works well with double precision floating point values? We have found that the binary representation of floating point values resul...
Halo asked 10/2, 2010 at 17:5
7
Solved
I know it's really stupid question, but I don't know how to do this in bash:
20 / 30 * 100
It should be 66.67 but expr is saying 0, because it doesn't support float.
What command in Linux can re...
Carmencita asked 10/8, 2009 at 9:48
8
Solved
Initially I thought Math.Sign would be the proper way to go but after running a test it seems that it treats -0.0 and +0.0 the same.
Examinee asked 19/1, 2011 at 19:36
10
Solved
Assuming I am really pressed for memory and want a smaller range (similar to short vs int). Shader languages already support half for a floating-point type with half the precision (not just convert...
Hunger asked 23/4, 2011 at 20:54
4
I'm working with floating point numbers. If I do:
import numpy as np
np.round(100.045, 2)
I get:
Out[15]: 100.04
Obviously, this should be 100.05. I know about the existence of IEEE 754 and t...
Hierogram asked 16/5, 2018 at 15:25
8
Solved
While working on a project, I came across a JS-script created by a former employee that basically creates a report in the form of
Name : Value
Name2 : Value2
etc.
The peoblem is that the values...
Thaw asked 29/10, 2008 at 9:14
2
Series I'm working with:
import pandas as pd
from decimal import Decimal, BasicContext
df = pd.Series([14978.22,
16025.429160000002,
209.97803999999996,
618.20369,
605.607,
1431.0916,
30.53575,
2...
Incombustible asked 10/8, 2018 at 20:7
4
Solved
I know that size of various data types can change depending on which system I am on.
I use XP 32bits, and using the sizeof() operator in C++, it seems like long double is 12 bytes, and double is 8....
Spaetzle asked 11/8, 2010 at 0:55
1
Context: https://stackoverflow.com/a/39707410/1778275 :
If the floating point type used a base 10 presentation, LDBL_DIG and LDBL_DECIMAL_DIG would have the same value.
Question: why base-2 leads...
Flexile asked 5/10, 2021 at 11:28
7
Solved
Say I have a float number. If it is an integer (e.g. 1.0, 9.0, 36.0), I want to remove the ".0 (the decimal point and zero)" and write to stdout. For example, the result will be 1, 9, 36. If the fl...
Misconceive asked 9/7, 2016 at 14:12
3
Solved
When creating a nan which contains a char-sequence, how do I get the char-sequence back? [duplicate]
When doing
const double d = std::nan ("Hello");
you get a NAN containing the string "Hello". How can one back out this string from the variable d? Is there simply no standard conforming wa...
Enlargement asked 16/8, 2018 at 7:41
7
Solved
Like:
float(1.2345678901235E+19) => string(20) "12345678901234567890"
Can it be done?
(it's for json_decode...)
Knotgrass asked 29/7, 2011 at 17:16
1 Next >
© 2022 - 2024 — McMap. All rights reserved.