factor analysis using R

Question

Im trying to do a factor analysis using R with varimax rotation, but not successful. I run the same exact data on SAS and can get result.

in R, if I use

fa(r=cor(m1), nfactors=8, fm="ml", rotate="varimax")

I will get

In smc, the correlation matrix was not invertible, smc's returned as 1s
In smc, the correlation matrix was not invertible, smc's returned as 1s
Error in optim(start, FAfn, FAgr, method = "L-BFGS-B", lower = 0.005,  : 
  L-BFGS-B needs finite values of 'fn'
In addition: Warning messages:
1: In cor.smooth(R) : Matrix was not positive definite, smoothing was done
2: In cor.smooth(R) : Matrix was not positive definite, smoothing was done
3: In log(e) : NaNs produced

if I use

factanal(cor(m1), factors=8)

i will get

Error in solve.default(cv) : 
  system is computationally singular: reciprocal condition number = 4.36969e-19

Can anyone help me how to do factor analysis successfully using R. Thanks.

Tq in advance

Detection

Both functions indicate that the correlation matrix is singular. Did you look into the SAS documentation to see what the function does in case of singular matrices? Maybe it has some way to get around it, and is that the reason it gives an output. – Unpopular

an update, if i set no of factors < 8, I can get the correct results – Detection

from SAS doc "The squared multiple correlations (SMC) of each variable with all the other variables are used as the prior communality estimates. If your correlation matrix is singular, you should specify PRIORS=MAX instead of PRIORS=SMC." – Detection

This will also occur if you have strong autocorrelation between the covariates, e.g., if you shift X1 with respect to X2, preserving order, and then take their difference. – Degrade

@Arthur, could you explain in more detail please - I believe this is what's happening with my data – Diaper

@Diaper If any X1 is serially correlated with any X2 you might see this message. That is, this can arise if variable X1 has values {x0, x1, x2, x3, ...} and variable X2 has values {x1, x2, x3, x4, ...}, where each value is some measurement in time and X2 has X1's values shifted "forwards in time." More generally, any serial correlation/ autocorrelation may result in a "singular system" here. – Degrade

Answer 1

The warnings and errors indicates that your matrix is singular, thus no solution exists to the optimization problem.

This means you need to use a different method of factor analysis. Using fa() in package psych you have two alternatives to perform factor analysis given a singular matrix:

• pa (Principal axis factor analysis)
• minres (Minimum residual factor analysis)

However, given your data, only minres seems to yield useful results, albeit with many health warnings:

library(psych)
library(GPArotation)
fa(r=cor(m1), nfactors=8, rotate="varimax", SMC=FALSE, fm="minres")

This gives:

In smc, the correlation matrix was not invertible, smc's returned as 1s
In factor.stats, the correlation matrix is singular, an approximation is used
In factor.scores, the correlation matrix is singular, an approximation is used
I was unable to calculate the factor score weights, factor loadings used instead
Factor Analysis using method =  minres
Call: fa(r = cor(m1), nfactors = 8, rotate = "varimax", SMC = FALSE, 
    fm = "minres")
Standardized loadings (pattern matrix) based upon correlation matrix
                MR1   MR3   MR2   MR6   MR5   MR4   MR7   MR8   h2    u2
Adorable       0.64  0.69  0.04  0.26  0.05  0.04  0.01  0.14 0.98 0.020
Appealing      0.69  0.66  0.06  0.22  0.06  0.00  0.03  0.08 0.98 0.021
Beautiful      0.39  0.82 -0.16  0.11  0.24 -0.05 -0.07 -0.08 0.93 0.071
Boring        -0.49 -0.70  0.33 -0.27  0.01  0.03  0.11 -0.16 0.95 0.054
Calm           0.76  0.42  0.33  0.10  0.28 -0.04  0.02  0.05 0.96 0.038
Charming       0.62  0.75  0.04  0.15  0.07 -0.03  0.03  0.01 0.98 0.024
Chic           0.07  0.94 -0.13  0.17 -0.03  0.12 -0.02  0.02 0.95 0.048
Childish      -0.13  0.00  0.04  0.04 -0.04  0.98  0.01  0.00 0.98 0.016
Classic        0.82  0.16  0.28 -0.31  0.14  0.10  0.16  0.06 0.94 0.058
Comfortable    0.66  0.50  0.19  0.39  0.27 -0.02  0.13  0.08 0.97 0.033
Cool           0.81  0.43  0.03  0.32  0.00  0.01 -0.03  0.20 0.98 0.016
Creative       0.78  0.37 -0.41  0.14 -0.05  0.06 -0.05  0.20 0.98 0.024
Crowded       -0.34 -0.12 -0.77 -0.13 -0.18  0.04  0.44  0.00 0.96 0.041
Cute           0.50  0.78  0.03  0.18  0.07  0.25 -0.09  0.14 0.98 0.024
Elegant        0.67  0.70  0.07 -0.04  0.10 -0.14  0.03  0.07 0.98 0.021
Feminine       0.09  0.96  0.00  0.01  0.01 -0.02  0.04  0.03 0.93 0.069
Fun            0.58  0.45 -0.21  0.56  0.01  0.20 -0.06 -0.08 0.95 0.054
Futuristic     0.91  0.26 -0.10  0.14 -0.07 -0.03 -0.18 -0.08 0.98 0.021
Gorgeous       0.82  0.52 -0.04  0.14  0.05 -0.09 -0.08 -0.01 0.98 0.019
Impressive     0.82  0.48 -0.02  0.23  0.05  0.00 -0.10  0.07 0.98 0.021
Interesting    0.72  0.55  0.05  0.34  0.15  0.01 -0.13  0.03 0.98 0.020
Light          0.20  0.49  0.30  0.72  0.22  0.03 -0.03  0.02 0.93 0.065
Lively         0.62  0.66 -0.06  0.37  0.16  0.00 -0.04 -0.03 0.98 0.021
Lovely         0.68  0.68 -0.04  0.12  0.19 -0.03 -0.08  0.01 0.98 0.019
Luxury         0.89  0.36 -0.02  0.00  0.08 -0.15 -0.04 -0.07 0.96 0.036
Masculine      0.91 -0.06 -0.05  0.24  0.05 -0.08  0.00 -0.17 0.94 0.063
Mystic         0.95  0.05  0.13  0.01 -0.03  0.00 -0.10  0.00 0.93 0.069
Natural        0.47  0.32  0.42  0.19  0.57 -0.17  0.23  0.02 0.95 0.050
Neat          -0.07  0.06  0.27  0.08  0.93 -0.01 -0.06 -0.01 0.96 0.042
Oldfashioned  -0.64 -0.54  0.20 -0.31  0.16  0.13  0.27 -0.16 0.97 0.026
Plain         -0.23 -0.19  0.88 -0.06  0.18  0.06  0.14 -0.14 0.94 0.062
Pretty         0.66  0.68  0.06  0.17  0.16 -0.11  0.01  0.10 0.97 0.029
Professional   0.82  0.41  0.09  0.18  0.16 -0.18  0.04  0.13 0.96 0.039
Refreshing     0.54  0.58  0.19  0.45  0.30 -0.03  0.10  0.07 0.98 0.021
Relaxing       0.56  0.65  0.34  0.26  0.21 -0.04  0.13 -0.03 0.97 0.026
Sexy           0.35  0.81  0.27  0.05 -0.01 -0.24  0.01 -0.19 0.94 0.056
Simple         0.08  0.01  0.96  0.08  0.09  0.02  0.04  0.12 0.96 0.041
Sophisticated  0.86  0.44 -0.01  0.04 -0.04 -0.12  0.08  0.05 0.96 0.040
Stylish        0.77  0.58  0.06  0.15  0.00 -0.07  0.07  0.08 0.97 0.030
Surreal        0.85  0.39  0.14  0.18 -0.05  0.02  0.08 -0.02 0.93 0.067

                        MR1   MR3  MR2  MR6  MR5  MR4  MR7  MR8
SS loadings           16.50 11.81 3.57 2.45 1.89 1.34 0.55 0.37
Proportion Var         0.41  0.30 0.09 0.06 0.05 0.03 0.01 0.01
Cumulative Var         0.41  0.71 0.80 0.86 0.91 0.94 0.95 0.96
Proportion Explained   0.43  0.31 0.09 0.06 0.05 0.03 0.01 0.01
Cumulative Proportion  0.43  0.74 0.83 0.89 0.94 0.98 0.99 1.00

Test of the hypothesis that 8 factors are sufficient.

The degrees of freedom for the null model are  780  and the objective function was  NaN
The degrees of freedom for the model are 488  and the objective function was  NaN 

The root mean square of the residuals (RMSR) is  0.01 
The df corrected root mean square of the residuals is  0.02 

Fit based upon off diagonal values = 1
Measures of factor score adequacy             
                                               MR1 MR3 MR2  MR6  MR5  MR4  MR7  MR8
Correlation of scores with factors               1   1   1 1.00 1.00 1.00 1.00 0.99
Multiple R square of scores with factors         1   1   1 1.00 1.00 1.00 0.99 0.98
Minimum correlation of possible factor scores    1   1   1 0.99 0.99 0.99 0.98 0.97
Warning messages:
1: In cor.smooth(R) : Matrix was not positive definite, smoothing was done
2: In log(det(m.inv.r)) : NaNs produced
3: In log(det(r)) : NaNs produced
4: In cor.smooth(r) : Matrix was not positive definite, smoothing was done
5: In cor.smooth(r) : Matrix was not positive definite, smoothing was done

Yoke