double-precision Questions
4
Solved
Code:
class Main {
public static void main (String[] args) {
System.out.print("float: ");
System.out.println(1.35f-0.00026f);
System.out.print("double: ");
System.out.println(1.35-0.00026);
...
Sterner asked 18/10, 2013 at 7:34
29
Given a double, I want to round it to a given number of points of precision after the decimal point, similar to PHP's round() function.
The closest thing I can find in the Dart docs is double.toSt...
Disenthrall asked 9/2, 2015 at 21:0
3
Solved
I have a text representation of a double and want to know if it's safe to round-trip it to double and back. How do I know this if I also want to accept any kind of number-style of the input? Or how...
Tauro asked 22/8, 2012 at 9:3
8
Solved
float f = 0.7;
if( f == 0.7 )
printf("equal");
else
printf("not equal");
Why is the output not equal ?
Why does this happen?
Oospore asked 3/12, 2009 at 11:41
3
Solved
I know it's an odd question, but does JavaScript have the capacity to work with double's as opposed to single floats? (64 bit floats vs. 32 bits.)
Rhinoplasty asked 31/8, 2010 at 3:57
3
Solved
What is the guaranteed accuracy of multiplication operator for double values in java?
For example, 2.2 * 100 is 220.00000000000003, but 220 is a double number. 220.00000000000003 is the next doub...
Frear asked 23/6, 2011 at 19:6
8
Solved
Every time I start a new project and when I need to compare some float or double variables I write the code like this one:
if (fabs(prev.min[i] - cur->min[i]) < 0.000001 &&
fabs(pre...
Romeliaromelle asked 28/12, 2010 at 17:38
5
Solved
This is partly academic, as for my purposes I only need it rounded to two decimal places; but I am keen to know what is going on to produce two slightly different results.
This is the test t...
Urumchi asked 24/4, 2015 at 8:1
4
Solved
When BigDecimal is used with an input of double and BigDecimal with an input of String different results seem to appear.
BigDecimal a = new BigDecimal(0.333333333);
BigDecimal b = new BigDec...
Ophthalmology asked 14/4, 2015 at 16:19
4
Solved
I have a unit test, testing boundaries:
[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void CreateExtent_InvalidTop_ShouldThrowArgumentOutOfRangeException()
{
var in...
Rachellrachelle asked 16/12, 2014 at 14:5
8
Solved
I've been working on some simple code for creating histograms and found that following code:
double value = 1.2;
double bucketSize = 0.4;
double bucketId = value / bucketSize;
std::cout << ...
Tichonn asked 21/10, 2014 at 13:55
3
Solved
I would like some advice from people who have more experience working with primitive double equality in Java. Using d1 == d2 for two doubles d1 and d2 is not sufficient due to possible rounding err...
Amesace asked 6/8, 2014 at 12:26
1
Solved
In my code I have this multiplications in a C++ code with all variable types as double[]
f1[0] = (f1_rot[0] * xu[0]) + (f1_rot[1] * yu[0]);
f1[1] = (f1_rot[0] * xu[1]) + (f1_rot[1] * yu[1]);
f1[...
Cerf asked 26/6, 2014 at 14:3
3
Solved
I'm trying to convert a 64-bit float to a 64-bit integer (and back) in php. I need to preserve the bytes, so I'm using the pack and unpack functions. The functionality I'm looking for is basically ...
Matte asked 1/6, 2014 at 17:33
4
I have a function that returns a double value. In some cases the result is zero, and this results should be handled in the caller routing accordingly. I was wondering what is the proper way of retu...
Stine asked 16/5, 2014 at 14:7
2
Solved
I created a random double precision value in Matlab by
x = rand(1,1);
then display all possible digits of x by
vpa(x,100)
and obtain:
0.2238119394911369 7971853298440692014992237091064453125...
Informant asked 6/11, 2013 at 9:28
5
Whats wrong with this simple 'double' calculation in java?
I know some decimal numbers can not be represented in float / double binary formats properly, but with the variable d3, java is abl...
Before asked 23/5, 2013 at 6:52
1
What order of magnitude difference should I be expecting for a subtraction between two theoretically equal double precision numbers?
I have two double precision arrays. They are expected to be the...
Standice asked 19/3, 2013 at 6:54
2
Solved
I would like to understand why this code:
double r,d,rc;
scanf("%lf %lf", &r, &d);
rc = (r * r) - (d/2) * (d/2);
printf("%.2f\n", M_PI * rc);
returns more precise result than this one (w...
Sickle asked 14/3, 2013 at 22:54
2
Solved
I ask because I am computing matrix multiplications where all the matrix values are integers.
I'd like to use LAPACK so that I get fast code that is correct. Will two large integers (whose produc...
Housewarming asked 27/12, 2012 at 22:4
3
Solved
I have been reading timestamp values from sensor readings, but since they are provided in nanoseconds, I thought I would cast them to double and make the conversion. The resulting number is a 17 di...
Halpin asked 12/12, 2012 at 15:11
3
Solved
Is it possible to establish, even roughly, what the maximum precision loss would be when dealing with two double values in java (adding/subtracting)? Probably the worst case scenario is when two nu...
Prittleprattle asked 20/11, 2012 at 10:2
3
Solved
I've wrote program, and compiled it for x64 and x86 platform in Visual Studio 2010 on Intel Core i5-2500. x64 version take about 19 seconds for execution and x86 take about 17 seconds. What can be ...
Ramakrishna asked 14/2, 2012 at 20:28
2
Solved
I am confused about using expm1 function in java
The Oracle java doc for Math.expm1 says:
Returns exp(x) -1. Note that for values of x near 0, the exact sum of
expm1(x) + 1 is much closer to th...
Conduction asked 17/8, 2012 at 19:33
1
Solved
I use the formula exp(X) as the rate for a markov chain. So the ratio of selecting one link over another is exp(X1)/exp(X2). My problem is that sometimes X is very large, so exp(X) will exceed the ...
Daughterinlaw asked 17/8, 2012 at 19:10
1 Next >
© 2022 - 2024 — McMap. All rights reserved.