I'm learning about unit quaternions and how to use them to represent and compose rotations. Wikipedia says they are more numerically stable than matrix representations, but doesn't give a reference...

I have a Tensorflow multiclass classifier that is generating nan or inf while computing probabilities using tf.nn.softmax. See the following snippet (logits is of shape batch_size x 6, since I have...

I know how to make softmax stable by adding to element -max _i x_i. This avoids overflow and underflow. Now, taking log of this can cause underflow. log softmax(x) can evaluate to zero, leading to...

Any non-zero recurrent_dropout yields NaN losses and weights; latter are either 0 or NaN. Happens for stacked, shallow, stateful, return_sequences = any, with & w/o Bidirectional(), activation=...

In GLSL (specifically 3.00 that I'm using), there are two versions of atan(): atan(y_over_x) can only return angles between -PI/2, PI/2, while atan(y/x) can take all 4 quadrants into account so th...

When I use this random generator: numpy.random.multinomial, I keep getting: ValueError: sum(pvals[:-1]) > 1.0 I am always passing the output of this softmax function: def softmax(w, t = 1.0)...

I am using CVXPY (version 1.0) to solve a quadratic program (QP) and I often get this exception: SolverError: Solver 'xxx' failed. Try another solver. which makes my program really fragile. I...

I had a function: float lerp(float alpha, float x0, float x1) { return (1.0f - alpha) * x0 + alpha * x1; } For those who haven't seen it, this is preferable to x0 + (x1-x0) * alpha because th...

Why does chisq.test function in R sorts data before summation in descending order? The code in question is: STATISTIC <- sum(sort((x - E)^2/E, decreasing = TRUE)) If I was worried about nume...

If I have a random number Z that is defined as the sum of two other random numbers, X and Y, then the probability distribution of Z is the convolution of the probability distributions for X and Y. ...

This is a branching from another quesion/answer I want a function equivalent to this: def softmax(x, tau): """ Returns softmax probabilities with temperature tau Input: x -- 1-dimensional array...

Is the addition x + x interchangeable by the multiplication 2 * x in IEEE 754 (IEC 559) floating-point standard, or more generally speaking is there any guarantee that case_add and case_mul always ...

Here is my problem: I am going to process data coming from a system for which I will have a good idea of the impulse response. Having used Python for some basic scripting before, I am getting to kn...

What is the most numerically stable way of calculating: log[(wx * exp(x) + wy * exp_y)/(wx + wy)] where the weights wx, wy > 0? Without the weights, this function is logaddexp and could be i...

I understand how mathematically-equivalent arithmentic operations can result in different results due to numerical errors (e.g. summing floats in different orders). However, it surprises me that a...

In a numerical solver I am working on in C, I need to invert a 2x2 matrix and it then gets multiplied on the right side by another matrix: C = B . inv(A) I have been using the following definiti...

I'm guessing there is some standard trick that I wasn't able to find: Anyway I want to compute a large power of a number very close to 1(think 1-p where p<1e-17) in a numerically stable fashion....

I need to compute a normalized exponential of a vector in Matlab. Simply writing res = exp(V)/sum(exp(V)) overflows in an element of V is greater than log(realmax) = 709.7827. (I am not sure a...

I've been trying to compute the jordan normal form of a 36-by-36 matrix composed of only three distinct entries, 1, 1/2, and 0. The matrix is a probability transition matrix so, given these entries...

I have translated the experimental C# "float" version of Clipper library to javascript. In the newest sandbox version there is a function IsAlmostEqual which seems to be hard to translate. Double e...

I am currently trying to implement a machine learning algorithm that involves the logistic loss function in MATLAB. Unfortunately, I am having some trouble due to numerical overflow. In general, f...

Can anyone recommend any C++ libraries/routines/packages that contain strategies for maintaining the stability of various floating point operations? Example: suppose you would like to sum across a...

I have some math (in C++) which seems to be generating some very small, near zero, numbers (I suspect the trig function calls are my real problem), but I'd like to detect these cases so that I can ...

i'm trying to do factor analysis for a co-occurrence matrix(C) , which is computed from the term-document matrix(TD) as follows: C=TD*TD' In theory C should be positive semi-definite , but it isn'...

I'm trying to implement a version of the Fuzzy C-Means algorithm in Java and I'm trying to do some optimization by computing just once everything that can be computed just once. This is an iterati...