floating-accuracy Questions
4
Solved
i am kind of confused why python add some additional decimal number in this case, please help to explain
>>> mylist = ["list item 1", 2, 3.14]
>>> print mylist ['list item ...
Chemoprophylaxis asked 6/6, 2010 at 22:22
2
Solved
I'm struggling to understand how TO IMPLEMENT the range reduction algorithm published by Payne and Hanek (range reduction for trigonometric functions)
I've seen there's this library:
http://www.ne...
Puccoon asked 26/5, 2015 at 16:4
2
Solved
In the following code, why is there a comparison against float.Epsilon and not 0?
// Coroutine to move elements
protected IEnumerator SmoothMovement (Vector3 end)
{
// Distance computation
float...
Japha asked 13/5, 2015 at 13:37
4
Solved
>>> float(str(0.65000000000000002))
0.65000000000000002
>>> float(str(0.47000000000000003))
0.46999999999999997 ???
What is going on here?
How do I convert 0.4700000000000000...
Dramaturgy asked 22/11, 2009 at 10:28
1
Numpy's allclose Returns True if two arrays are element-wise equal within a tolerance.
Is there any equivalent in Matlab?
Arleenarlen asked 10/3, 2015 at 23:16
3
Solved
Why `0.4/2` equals to `0.2` meanwhile `0.6/3` equals to `0.19999999999999998` in python? [duplicate]
I know these are float point division. But why did these two formula behave differently?
And I did some more investigation, the result confusing me even more:
>>>0.9/3
0.3
>>...
Ygerne asked 9/3, 2015 at 4:28
12
Solved
Is it possible to get division by 0 (or infinity) in the following example?
public double calculation(double a, double b)
{
if (a == b)
{
return 0;
}
else
{
return 2 / (a - b);
}
}
In n...
Tirewoman asked 12/2, 2015 at 9:55
1
Solved
I found Stevens Computing Services – K & R Exercise 2-1 a very thorough answer to K&R 2-1. This slice of the full code computes the maximum value of a float type in the C programming langua...
Mcginley asked 9/2, 2015 at 19:1
3
Solved
What is the difference between two following?
float f1 = some_number;
float f2 = some_near_zero_number;
float result;
result = f1 / f2;
and:
float f1 = some_number;
float f2 = some_near_zero_n...
Upholster asked 5/2, 2015 at 12:17
5
Solved
I've seen a lot of discussion on SO related to rounding float values, but no solid Q&A considering the efficiency aspect. So here it is:
What is the most efficient (but correct) way to round ...
Knowlton asked 23/8, 2012 at 11:56
2
Solved
Consider these two cases:
fmt.Println(912 * 0.01)
fmt.Println(float64(912) * 0.01)
(Go Playground link)
The second one prints 9.120000000000001, which is actually fine, I understand why that is...
Radiotransparent asked 7/1, 2015 at 12:4
3
Solved
I know that float arithmetic is tricky, but I am not sure if it is possible to get a mistake if you have a division followed by the inverse multiplication. Written in code, is it possible, that thi...
Stenotype asked 28/12, 2014 at 20:29
2
Solved
Why does this assertion fail?
import std.conv;
void main()
{
auto y = 0.6, delta=0.1;
auto r = to!int(y/delta);
assert(r == 6);
}
r's value should be 6 and yet it's 5, Why?
Deepsix asked 26/12, 2014 at 7:24
1
Solved
What sort of stability issues arise or are resolved by using std::pow()?
Will it be more stable (or faster, or at all different) in general to implement a simple function to perform log(n) iterat...
Festus asked 24/12, 2014 at 11:42
2
Solved
In Java, is multiplying a double by 0.0000001 the same as dividing it by 10000000? My intuition is that there could be a difference because 0.0000001 cannot be represented exactly in a double.
Extenuatory asked 15/12, 2014 at 15:22
4
Solved
Is the floating point specialisation of std::hash (say, for doubles or floats) reliable regarding almost-equality? That is, if two values (such as (1./std::sqrt(5.)/std::sqrt(5.)) and .2) should co...
Toluene asked 18/2, 2013 at 19:25
3
Solved
What is the correct way to perform == and != operators in template classes?
Assume this code:
template<typename T>
class C {
T x, y;
public:
C(T a, T b) : x(a), y(b) {}
bool cmp() {
r...
Transept asked 1/12, 2014 at 12:50
1
Solved
I have a large code that at one point samples values from an array according to the probabilities taken from a probability density function (PDF).
To do this I use the numpy.random.choice which wo...
Pardner asked 23/9, 2014 at 0:35
3
Solved
A float (a.k.a. single) value is a 4-byte value, and supposed to represent any real-valued number. Because of the way it is formatted and the finite number of bytes it is made off, there is a...
Morpho asked 15/9, 2014 at 13:56
3
Solved
Let's say that we have declared the following variables
float a = 1.2291;
float b = 3.99;
float variables have precision 6, which (if I understand correctly) means that the difference between th...
Canyon asked 6/9, 2014 at 21:4
0
I need to enhance precision for function in one of mine Atmospheric scattering GLSL fragment shader which computes the intersection between single ray and axis aligned ellipsoid.
This is the core f...
Jennifferjennilee asked 24/8, 2014 at 9:58
9
Solved
I wrote a program to calculate nth root of a number upto 2 decimal places. eg 4th root of 81 is 3., 3rd root of 125 is 5.Its working nicely except for the 2nd root of 4. It's giving the output 1.99...
Galaxy asked 16/8, 2010 at 15:15
1
Solved
Regarding minimising the error in floating-point operations, if I have an operation such as the following in C:
float a = 123.456;
float b = 456.789;
float r = 0.12345;
a = a - (r * b);
Will the...
Marianomaribel asked 7/8, 2014 at 12:27
4
Basically, if I want something like this,
double b = sin(2.2);
but accidentally write something like this,
double b = sin(2.2f);
there is no error or even warning message, even though this cl...
Southey asked 29/6, 2014 at 18:22
1
Solved
I am a numerical physicist, and I've seen some simulation codes in my community which use a 3D simulation box with a center in [0.5, 0.5, 0.5] and a normalized length of 1 (so the box coordinates g...
Alard asked 25/6, 2014 at 18:29
© 2022 - 2024 — McMap. All rights reserved.