Content Disclaimer
Copyright @2020.
All Rights Reserved.

StatsToDo: Factor Analysis

Links : Home Index (Subjects) Contact StatsToDo


Introduction Complete Program Sub Programs R Codes Parallel Analysis Sample Size

Introduction

This page makes no attempt to explain Factor Analysis comprehensively. It is assumed that users are already familiar with the concepts and purposes of Factor Analysis, and the explanations provided are to help users to make decisions on program parameters.

Only Exploratory Factor Analysis is covered. Confirmatory Factor Analysis is a separate subject and is not covered by StatsToDo

The page provides Factor Analysis using the following procedures

  • Principal Components are calculated from Eigen Values and Vectors, using the Jacobi method
  • Factor rotation allows the options of
    • Orthogonal rotation (Varimax) produces factors that are uncorrelated with each other
    • Oblique rotation (Oblimin, and in the R codes also Promax) produces a close fit, but allows the factors to be correlated
  • Conversion of the rotated factors into the W matrix, which are coefficients to calculate factor scores
  • Calculate factor scores if data is available
In addition, this page provides a discussion on sample size, and a program on Parallel Analysis. Paralle Analysis is a big subject and disccused in its own page in ParallelAnalysis.php

Options and Decisions

The following options are available in using the programs on this page

Sample size is discussed in the sample size panel

Principal components based on the Covariance or the Correlation matrix.

  • The Covariance matrix was used when Factor Analysis was first described, and it results in coefficients (W matrix) that can translate measurements directly into factor scores. The disadvantage is that the factor matrix is intuitively difficult to interpret, and it reflects the numerical values, so that length using inches or cms will result in very different factor values
  • The Correlation matrix is currently the most common option, and conceptually it turns all measurements into standard deviate measurements (z = (value-mean)/SD). The disadvantage is that, to calculate factor scores, the primary measurements need to be converted to the z value, thus requiring an additional calculation.
  • The program on this page uses the Correlation matrix by default when raw data is presented. However, there is an option to calculate starting with a matrix, and users can enter either the Covariance or the Correlation matrix
The number of factors to retain. Two options are available to the user
  • Specify the number of factors to retain. This must be a number from 1 to the number of variables
  • Use the K1 rule where all factors with Eigen value >=1 are retained. This is done by specify the number of factors to retain as <1 or > number of variables
  • Discussions on how to decide on which option is discussed in more details in ParallelAnalysis.php
Orthogonal (Varimax) or Oblique (Oblimin) Rotation
  • This is usually determined by the theoretical construct of the purpose of the exercise, whether the factors are correlated or not. For example, in the measurement of capability, the physical strength may be considered uncorrelated to mental acuity, but in measuring intelligence, the ability to comprehend and that of mathematics may be correlated.
  • Without a preconceived framework, the oblique rotation can be tried first to obtained the best fit, and if the results show no meaningful correlation between the factors, then the orthogonal used to produce the final results.

Other Panels on This Page

The Complete Program performs Factor Analysis in one step, but requires the user to specify tyoe of data entry, number of factors to retain, and type of rotation

The Subprograms performs the same functions, but in individual steps, allowing users to modify intermediary results

R codes contains the whole program in R

Parallel Analysis, a duplication of the Javascript program from ParallelAnalysis.php

Sample size, a discussion and some tables on how to determine the sample size necessary for Factor Analysis

References

Algorithms : It is difficult to find the algorithms for calculations associated with Factor Analysis, as most modern text book and technical manuals advise users to use one of the commercial packages. I have eventually found some useful algorithms in old text books, and they are as follows

    Press WH, Flannery VP, Teukolsky SA, Vetterling WT (1989). Numerical Recipes in Pascal. Cambridge University Press IBSN 0-521-37516-9 p.395-396 and p.402-404. Jacobi method for finding Eigen values and Eigen vectors

    Norusis MJ (1979) SPSS Statistical Algorithms Release 8. SPSS Inc Chicago

    • p. 86 for converting Eigen values and vectors to Principal Components
    • p. 91-93 for Varimax Rotation
    • p. 94-97 for Oblimin Rotation
    • p. 97-98 for Factor scores

Text books I learnt Factor Analysis some time ago, so all my text books are old, but they are adequate in explaining the basic concepts and provide the calculations used in these pages. Users should search for better and newer text books.

  • Thurston LL (1937) Multiple Factor Analysis. University of Chicago Press. I have not read this book, but this is quoted almost universally as it is the original Factor Analysis text book which set out the Principles and procedures
  • Gorsuch RL (1974) Factor Analysis. W. B. Saunders Company London 1974 ISBN 0-7216-4170-9 A standard text book teaching Factor Analysis at the Master's level. This is my copy and I believe there are later editions of this book available
Parallel Analysis Sample Size
Mundfrom DJ, Shaw DG, Tian LK (2005) Minimum sample size recommendations for conducting factor analysis. International Journal of Testing 5:2:p 159-168

Free Factor Analysis software and its user manual can be downloaded from http://psico.fcep.urv.es/utilitats/factor/Download.html. This is a package for Windows written by Drs. Lorezo-Seva and Ferrendo from Universitat Rovira i Virgili in Terragona in Spain. The presentation and options are very similar to that from SPSS, and the manual is excellent. The best part is that it is free and yes it is in English.

Teaching and discussion papers on the www There is an enormous list of discussion papers, technical notes and tutorials on the www that can be easily found by Google search. The following is a small sample of this.