Parallel

Content Disclaimer
Copyright @2020.
All Rights Reserved.

StatsToDo: Parallel Analysis in Factor Analysis

Links : Home Index (Subjects) Contact StatsToDo

Explanation Tables of Minimum Eigen Values Computer Program, R Code

Introduction Technical Considerations Example

Parallel Analysis is a procedure sometimes used to determine the number of Factors or Principal Components to retain in the initial stage of Exploratory Factor Analysis.

This discussion assumes that the user understands Factor Analysis and the procedure of Principal Component extraction, and no details for these are provided here.

A critical decision in Exploratory Factor Analysis is to determine how many Principal Components to retain, as each extraction produces decreasingly significant Factors. Retaining too few leads to a loss of information in the data, and retaining too many includes trivial and random information. Both of these produce misleading and unreproducible results.

Traditionally, researchers depend on one or more of the following criteria to determine how many components to retain.

The most common, and the default criteria in most statistical packages is the K1 rule. Principal Components are retained while the Eigen Value (the Variance associated with the component), is >=1. This is based on the argument that, in a correlation matrix, each variable contribute a variance of 1, so a component that accounts for less than that has no meaning and should be discarded. The criticism to the K1 rule is that the Eigen values are inflated by random associations in the data, so that the use of the K1 rule often retains more components or factors than appropriate, particularly when the sample size is small.
Another rule is using the Scree Test. The Eigen values are plotting against component number, and when the sharp decrease in Eigen values level off (the scree), the remaining components are abandoned. This is based on the arguments that the initial and significant components each extracts a large proportion of the variance from the correlation matrix, while the insignificant ones contain mostly data noise and so their Eigen values are similar. The criticism of using the Scree Test is that it depends on eye balling when there is no sharp transition where the scree begins. Thus researchers often disagree where the scree begins.
Some researchers tried different retention levels, and match the results with the theoretical model of the data. This does produce neat outcomes, but runs the risk that, if the underlying theory is flawed to start with, then the results tend not to be reproducible.

Parallel Analysis takes a different approach, and is based on the Monte Carlo simulation. A data set of random numbers, but having the same sample size and number of variables as the user's research data, are subjected to analysis, and the Eigen values obtained are recorded. This is repeated many times (often between 50 and 100 iterations, and the tables later on this page used 1000 iterations).

The mean and Standard Deviation of the replicated Eigen values for each component are then calculate, from which the 95^th percentile value is obtained (95^th percentile = mean + 1.64SD). These form the standard against which the Eigen value of each component from the research data is compared, and a components is retained if its Eigen value exceeds the 95^th percentile of the simulated values. The argument being that a component should be retained if its Eigen value is clearly greater (at 95^th percentile) than that obtained at random.

References

Hayton JC, Allen DG and Scarpello V (2004)Factor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis. Organizational Research Methods 2004; 7; 191

Watkins MW (2006)Determining Parallel Analysis Criteria. Journal of Modern Applied Statistical Methods Vol. 5, No. 2, 344-346

Free program to do Parallel Analysis from someone else downloadable from WWW

Ledesma RD (2007)Determining the Number of Factors to Retain in EFA: an easy-to-use computer program for carrying out Parallel Analysis. Practical Assessment, Research, and Evaluation Volume 12, Number 2, p. 1-11 Accesible on www.

Word file with SPSS commands for Parallel Analysis

Press WH, Flannery VP, Teukolsky SA, Vetterling WT (1989). Numerical Recipes in Pascal. Cambridge University Press IBSN 0-521-37516-9 p.395-396 and p.402-404. (computer program to extract Eigen Values and Vectors from a correlation matrix using the Jacobi algorithm).

The R program takes 3 parameters

nc is sample size and in this example is 25 cases or rows of data
nv is the number of variables and in this example is 6
ni is the number of iterations. In most cases this is between 50 and 1000, and in this example set at 100

The program then calculate the eigen values vector using normally distributed random values (nc rows by nv columns). This is carried out ni times, and the values are stored in vectors of Σx and Σx². From these the mean and Standard Deviations are calculated.

The final results are the 95 percentile of these values, which can then be used as the standard to determine the number of factors to retain

 
# parameters
nc = 25               # nc = number of cases, sample size
nv = 6                # nv = number of variabes 
ni = 100              # ni = number of iterations in Monte Carlo simulation

# parallel analysis begin
exVect <- rep(0, nv)      # null vector for Σx
exxVect <- rep(0, nv)     # null vector for Σx²
for(i in 1:ni)        # interate ni times
{
    dMx <- replicate(nv, rnorm(nc))   # create matrix of normally distributed random numbers
    evVect <- eigen(cor(dMx))$values  # vector of eigen values
    for(j in 1:nv)                    # add to vectors of sum x ans sum x sq
    {
      exVect[j] = exVect[j] + evVect[j]
      exxVect[j] = exxVect[j] + evVect[j]^2
    }
}                      # end of interation
resVect <- rep(0,nv)                # result vector containing 95 percentile eigen values
for(j in 1:nv)
{
    mean = exVect[j] / ni                             # mean = Σx/n
    sd = sqrt((exxVect[j] - exVect[j]^2/ni)/(ni-1))   # Standard Deviation
    lim = mean + qnorm(0.95) * sd                     # 95 percentile limit
    resVect[j] = lim 
}
print(resVect)                 # array of 95 percentile eigen values

The results are

[1] 1.9605460 1.5193420 1.2037633 1.0112186 0.7737773 0.5794345

Please note that the values will be slightly different for each calculation, as different random numbers are used. However the differences would be smaller if the number of iterations increases.