Numerical Instability Kalman Filter in MatLab

Question

I am trying to run a standard Kalman Filter algorithm to calculate likelihoods, but I keep getting a problema of a non positive definite variance matrix when calculating normal densities. I've researched a little and seen that there may be in fact some numerical instabitlity; tried some numerical ways to avoid a non-positive definite matrix, using both choleski decomposition and its variant LDL' decomposition. I am using MatLab. Does anyone suggest anything? Thanks.

Smaltite

Post a minimal example (code) – Rate

When updating the noise covariance are you using (I-KH)P or (I-KH)P(I-KH)*P*(I-KH)' + K*R*K'? (where P is the noise covariance before correction, K is the kalman gain, H is the observation model and R is the observation noise covariance) See "Joseph Form" at en.wikipedia.org/wiki/… – Isobath

Is your system observable? Check the singular values of your error covariance matrix P. If the system is unobservable, these will grow without bounds from the start. If it is indeed a numerical instability issue, the max singular value of P will first decrease, and then start increasing. Of course, it is impossible to say without describing the problem more. Are you applying this on a linear system? Nonlinear system? – Aetna

I will update my question with more details soon, but one thing caught my attention in Mr. Fegur comments: my system's observation equation is just an identity with the transition equation; that is, I observe all variables with no measurement error. Do you think this may be causing problems? The system is linear. – Smaltite

Answer 1

I have encountered what might be the same problem before when I needed to run a Kalman filter for long periods but over time my covariance matrix would degenerate. It might just be a problem of losing symmetry due to numerical error. One simple way to enforce your covariance matrix (let's call it P) to remain symmetric is to do:

P = (P + P')/2  # where P' is transpose(P)

right after estimating P.

Narine

Using the Joseph’s form will ensure a symmetric covariance matrix P as well as fix other numerical errors. – Carolecarolee

@NirRegev I had never seen Joseph's form; I looked it up thanks to you. I see that the covariance matrix will always be positive semi-definite. Thank you. – Narine

Answer 2

post your code.

As a rule of thumb, if the model is not accurate and the regularization (i.e. the model noise matrix Q) is not sufficiently "large" an underfitting will occur and the covariance matrix of the estimator will be ill-conditioned. Try fine tuning your Q matrix.

Carolecarolee

Slightly tuning Q made a huge difference for me (being restricted to float32 due to GPU programming). Thank you for the tip! – Elemental

Answer 3

The Kalman Filter implemented using the Joseph Form is known to be numerically unstable, as any old timer who once worked with single precision implementation of the filter can tell. This problem was discovered zillions of years ago and prompt a lot of research in implementing the filter in a stable manner. Probably the best well-known implementation is the UD, where the Covariance matrix is factorized as UDU' and the two factors are updated and propagated using special formulas (see Thoronton and Bierman). U is an upper diagonal matrix with "1" in its diagonal, and D is a diagonal matrix.

Mozambique