This page provides 4 programs used in the comparison of two Crnbach Alphas

• Sample Size: calculates sample size required from 6 parameters
  • Probability of Type I Error (α, p, Sig) to determine statistical significance. In most cases this would be p=0.05
  • Power (1 - β). In most cases this would be 0.8, corresponding to Probability of Type II Error (β) of 0.2
  • The number of items of measurements in the first Cronbach Alpha (k1)
  • The value of the first Cronbach Alpha (ca1)
  • The number of items of measurements in the second Cronbach Alpha (k2)
  • The value of the second Cronbach Alpha (ca2)
• Power ( 1 - β): Calculates power of the data already collected from 6 parameters
  • Probability of Type I Error (α, p, sig) to determine statistical significance. In most cases this would be p=0.05
  • The sample size (n) for each of the two Cronbach Alphas, assuming that they are the same. If they are not, the average of the two are used
  • The number of items of measurements in the first Cronbach Alpha (k1)
  • The value of the first Cronbach Alpha (ca1)
  • The number of items of measurements in the second Cronbach Alpha (k2)
  • The value of the second Cronbach Alpha (ca2)
• Confidence Interval: Calculate the confidence interval of the effect size (ratio of the two Dronbach Alphas). Please note the effect size delta is δ = (1-ca1) / (1 - ca2), with the null value of 1. This means that a significant difference can be concluded if the confidence interval does not traverse the null value of 1. Calculations are based on 6 parameters
  • Percent of confidence. In most cases the 95% condfidence interval is used
  • The sample size (n) for each of the two Cronbach Alphas, assuming that they are the same. If they are not, the average of the two are used
  • The number of items of measurements in the first Cronbach Alpha (k1)
  • The value of the first Cronbach Alpha (ca1)
  • The number of items of measurements in the second Cronbach Alpha (k2)
  • The value of the second Cronbach Alpha (ca2)
• Pilot Study: produces a table of confidence intervals from varying sample sizes. This requires 7 parameters
  • Percent of confidence. In most cases the 95% condfidence interval is used
  • The number of items of measurements in the first Cronbach Alpha (k1)
  • The value of the first Cronbach Alpha (ca1)
  • The number of items of measurements in the second Cronbach Alpha (k2)
  • The value of the second Cronbach Alpha (ca2)
  • The interval of sample size to be tested. In most cases 2 - 5 is used
  • The maximum sample size (for each of the two Cronbach Alphas) to be tested. In most cases it is 100, as the precision required seldom need more than 30-70 cases in a pilot study

Use of the 4 calculations

The pilot study and sample size estimation are used in the planning stage of a research projecr. The sample size estimate is used when the researcher has some backgorund information for the estimate.

In a completely new project, where the environment of the project is unknown, the pilot study is required to compare the outcomes of different sample size, and determine when further increase in sample size no longer further improves the confidnce interval in a meaningful way

The power estimate and confidence interval are carried out at the end of a project, using the data already collected. The power estimate provides an index whether the sample size was adequate for the requirements of the model. The confidence intrval provides an estimate of the precision of the effect size thus obtained.

One and two tail models

The programs provides results in both the one and two tail model. The two tail model is commonly presented, and this describes the range of the effect size for precision estimates and for comparison with other data.

In many cases, when Cronbach Alpha is used as a tool during the development of a multivariate measurement (such as when Factor Analysis or Multiple Regression is used), the researcher may have a minimum effect size in mind, and wishes to know if the two Alphas differ enough to stsatisfy this requirement. Under these considerations the one tail model is more appropriate as this requires a smaller sample size.

References

Bonett D G (2002) Sample Size Requirements for Comparing Two Alpha Coefficients Applied Psychological Measurement, Vol. 27 No. 1, January 2003, 72 - 74

Duhachek A and Iacobucci D (2004) Alpha's Standard Error (ASE): An Accurate and Precise Confidence Interval Estimate. Journal of Applied Psychology Vol. 89, No. 5, 792-808

Johanson GA and Brooks GP (2010) Initial Scale Development: Sample Size for Pilot Studies. Educational and Psychological Measurement Vol.70,Iss.3;p.394-400

Data

Sample Size : is a table of 6 columns   - Each row contains data from a separate study   - Col 1 = Probability of Type I Error α   - Col 2 = Power (1 - β)   - Col 3, 5 = number of items in Alpha1 (k1) & Alpha2 (k1) (items in a factor)   - Col 4, 6 = Expected Alpha1 (CA1) & Alpha2 expected (CA2)    Power Estimation : is a table of 6 columns   - Each row contains data from a separate study   - Col 1 = Probability of Type I Error α   - Col 2 = Sample size per Alpha (assuming equal size groups) ((n1+n2)/2)   - Col 3, 5 = number of items in Alpha1 (K1) & Alpha2 (K2)   - Col 4, 6 = Observed Alpha1 (CA1) & Alpha2 (CA2)    Confidence Interval : is a table of 6 columns   - Each row contains data from a separate study   - Col 1 = Percent confidence (usually 95 or 99)   - Col 2 = Sample size per Alpha (assuming equal size groups) ((n1+n2)/2)   - Col 3, 5 = number of items in Alpha1 (K1) & Alpha2 (K2)   - Col 4, 6 = Observed Alpha1 (CA1) & Alpha2 (CA2)    Pilot Stury : is for a single plan, a single column with 7 rows   - Row 1 : Percent confidence required (usually 95 or 99)   - Row 2, 4 : Number of items in the Alpha1 (k1) & Alpha2 (k2)   - Row 3, 5 : Value of the Alpha1 (CA1) & Alpha2 (CA2)   - Row 6 : Sample size interval for tabulation (usually 3 - 10)   - Row 7 : Maximum sample size (per Alpha) (Usually <=100)

# Program 1: Sample Size (per Crobach Alpha)
#                    Subroutine
ssAlpha <- function(alpha,beta,k1,k2,delta,tail)  # alpha and beta Type I & II error
{                 # k1 and k2 number of items delta calculated from a1 and ca2
  za = qnorm(alpha / tail)
  zb = qnorm(beta)   
  n = ceiling(2 * (k1  /(k1 - 1) + k2 / (k2 - 1)) * (za + zb)**2 / log(delta)**2 + 2)
  return (n)
}
#                 Main program
# Input Data
# Sig = Probability of Type I Error
# Power = 1 - Probability of Type II Error
# K1 = number of items in Cronbach Alpha 1
# Ca1 = Value of Cronbach Alpha 1
# K2 = number of items in Cronbach Alpha 2
# Ca2 = Value of Cronbach Alpha 2
dat = ("
Sig   Power K1 Ca1  K2 Ca2
0.05  0.80  5  0.7  7  0.3
0.01  0.95  5  0.7  7  0.3
0.05  0.80  7  0.8  9  0.2
0.01  0.95  7  0.8  9  0.2       ") 
df <- read.table(textConnection(dat),header=TRUE)  # conversion to data frame    
df   # optional display of input data 
# vectors for results
Delta <- vector()
SSiz_1 <- vector()
SSiz_2 <- vector()
# Calculations
for(i in 1 : nrow(df))
{
  sig = df$Sig[i]
  beta = 1 - df$Power[i]
  k1 = df$K1[i]
  ca1 = df$Ca1[i]
  k2 = df$K2[i]
  ca2 = df$Ca2[2]
  delta = (1 - ca1) / (1 - ca2)  
  Delta <- append(Delta, delta)
  # 1 tail
  SSiz_1 <- append(SSiz_1,ssAlpha(sig,beta,k1,k2,delta,1)) 
  # 2 tail
  SSiz_2 <- append(SSiz_2,ssAlpha(sig,beta,k1,k2,delta,2)) 
}
# include into data frame for display
df$Delta <- Delta
df$SSiz_1 <- SSiz_1
df$SSiz_2 <- SSiz_2
df # Input data + results

The results are : Delta = Effect Size, SSiz_1 and SSiz_2 are sample size for each Cronbach Alpha, 1 taii and 2 tail

> df # Input data + results
   Sig Power K1 Ca1 K2 Ca2     Delta SSiz_1 SSiz_2
1 0.05  0.80  5 0.7  7 0.3 0.4285714     44     55
2 0.01  0.95  5 0.7  7 0.3 0.4285714    109    122
3 0.05  0.80  7 0.8  9 0.2 0.2857143     21     25
4 0.01  0.95  7 0.8  9 0.2 0.2857143     49     55

#Program 2: Power Estimates comparing 2 Cronbach Alphas
#             Subroutines
# estimate sample size
ssAlpha <- function(alpha,beta,k1,k2,delta,tail)  # alpha and beta Type I & II error
{                 # k1 and k2 number of items delta calculated from a1 and ca2
  za = qnorm(alpha / tail)
  zb = qnorm(beta)   
  n = ceiling(2 * (k1  /(k1 - 1) + k2 / (k2 - 1)) * (za + zb)**2 / log(delta)**2 + 2)
  return (n)
}
# iterate beta until sample size compatible
betaAlpha <- function(sig,n,k1,k2,delta,tail)
{
  bL = 0
  bR = 1
  bM = (bL + bR) / 2
  nM = 1
  for(i in 1 : 100)  # interation should not need more than 100 cycles
  {  
    if(bM<0.00001 | bM>0.99999) return(c(resAr[0],bM))
    nM = ssAlpha(sig,bM,k1,k2,delta,tail) - n
    if(abs(nM)<0.0001) return(bM)
    if(nM>0) 
    { 
      bL = bM
    } 
    else 
    { 
      bR = bM
    }
    bM = (bL + bR) / 2;
  }
  return(bM)
}
#       Main Program
# Input Data
# Sig = Probability of Type I Error
# n = sample size each Cronbach Alpha (assuming same sample size for both)
# K1 = number of items in Cronbach Alpha 1
# Ca1 = Value of Cronbach Alpha 1
# K2 = number of items in Cronbach Alpha 2
# Ca2 = Value of Cronbach Alpha 2
dat = ("
Sig   n   K1 Ca1  K2 Ca2
0.05  55  5  0.7  7  0.3
0.01  55  5  0.7  7  0.3
0.05  10  7  0.8  9  0.5
0.01  10  7  0.8  9  0.5
       ")
df <- read.table(textConnection(dat),header=TRUE)  # conversion to data frame    
df   # optional display of input data 
# vectors for results
Delta <- vector()
Power_1 <- vector()
Power_2 <- vector()
# Calculations
for(i in 1 : nrow(df))
{
  sig = df$Sig[i]
  n = df$n[i]
  k1 = df$K1[i]
  ca1 = df$Ca1[i]
  k2 = df$K2[i]
  ca2 = df$Ca2[i]
  delta = (1 - ca1) / (1 - ca2) 
  Delta <- append(Delta,delta)
  Power_1 <- append(Power_1, 1 - betaAlpha(sig,n,k1,k2,delta,1))
  Power_2 <- append(Power_2, 1 - betaAlpha(sig,n,k1,k2,delta,2))
}  
df$Delta <- Delta
df$Power_1 <- Power_1
df$Power_2 <- Power_2
df # display input data and reaults

> df # display input data and reaults
   Sig  n K1 Ca1 K2 Ca2     Delta   Power_1   Power_2
1 0.05 55  5 0.7  7 0.3 0.4285714 0.8750000 0.7968750
2 0.01 55  5 0.7  7 0.3 0.4285714 0.6796875 0.5859375
3 0.05 10  7 0.8  9 0.5 0.4000000 0.3125000 0.2187500
4 0.01 10  7 0.8  9 0.5 0.4000000 0.1250000 0.0781250