System is computationally singular: reciprocal condition number in R

Question

x <- matrix(rnorm(80, mean = 0, sd = 0.1), 8, 8)
c <- cov(x)
solve(c)

I get the error message:

Error in solve.default(c) : system is computationally singular: reciprocal condition number = 6.57889e-18

I have been trying to figure out what is the reason behind the problem, and other threads at Stack Overflow have suggested the issue might be due to singular matrices, highly correlated variables, linear combination etc. However, I assumed that rnorm would avoid the mentioned problems.

For another matrix that I am working with det() gives 8.313969e-95, but it is still invertible with solve().

Father

Also see two related posts on partner sites: math.stackexchange.com/q/889425 and stats.stackexchange.com/questions/76488/…. – Infection

Answer 1

Two fundamental linear algebra properties:

1. A singular (square) matrix is a (square) matrix that is not invertible.
2. A matrix is not invertible if its determinant equals zero.

If you check

set.seed(2018);
x <- matrix(rnorm(80, mean = 0, sd = 0.1), 8, 8)
c <- cov(x)
det(c)
#[1] -3.109158e-38

So indeed, det(c) is zero (within machine precision); hence c is not invertible, which is exactly what solve(c) is trying to do.

PS 1: Take a look at ?solve to see that solve(a) will return the inverse of a.
PS 2: There exists a nice post on Mathematics on the interpretation of the determinant of the covariance matrix. Take a look to understand why you're seeing what you're seeing.

Bryonbryony

In addition, you can check qr(c)$rank – Shoshana

Thanks for the explanation. For another matrix that I am working with det() gives 8.313969e-95, but it is still invertible with solve(). – Father

@Mataunited17 Sounds like that's related to floating point arithmetic/precision; a determinant of 8.313969e-95 is zero (within machine precision), and therefore the inverse does not exist. Whatever matrix is returned by solve is probably not numerically stable. Perhaps edit your post to include this example; might be instructive for future readers. – Bryonbryony

I don't understand why financial time (returns) series have determinants of zero. Would that make sense? – Father

I would assume that a covariance-matrix of financial time series is positive definite, which would assume non-zero determinant. – Father

@Mataunited17 To understand why the determinant of the covariance is zero in the above example think about the following: What is the covariance of a square matrix? [Hint: For a matrix C to be invertible it has to have det C ≠ 0; an alternative statement is: For C to be invertible it has to have full rank. What is the rank of the covariance matrix of a square matrix? Compare for example the ranks of cov(mtcars[1:3, c("drat", "wt", "qsec")]) and cov(mtcars[1:4, c("drat", "wt", "qsec")])]. Does that clear things up? – Bryonbryony