The Kuder Richardson Coefficient of reliability (K-R 20) is used to test the reliability of binary measurements, to see if the items within the instruments obtained the same binary (no/yes, right/wrong) results over a population of testing subjects.

The formula for the coefficient can easily be obtained from Wikipedia on the Internet.

Please Note that the K-R 20 was first described in 1937. Hoyt in 1940 modified the formula so that it can be applied to measurements that are not binary. Hoyt's modification eventually was popularised and is now known as Cronbach's Alpha. Cronbach's Alpha, when applied to binary data, will therefore produce the same result as KR-20. Cronbach's Alpha is now much preferred, and will be discussed in its own page.

Example

The example data is artificially created to demonstrate the procedures, and not real. It purports to be a study of 4 no/yes questions in an exam. The questions are tested on 5 students. Please note This data set is deliberately small to make the demonstration easier to interpret. In a real study, many more questions (many tens) and students (serveral hundreds) would be required to ensure the results are reproducible.

	T1	T2	T3	T4
Student 1	wrong	correct	correct	wrong
Student 2	correct	correct	correct	correct
Student 3	wrong	correct	wrong	wrong
Student 4	wrong	wrong	correct	wrong
Student 5	correct	correct	correct	correct

We have 4 multiple choice questions (T1 to T4), administered to 5 students, as shown in the table on the left.

0	1	1	0
1	1	1	1
0	1	0	0
0	0	1	0
1	1	1	1

The number 0 is used to represent the wrong answer, and 1 the correct answer. The data set to be analysed is therefore as shown in the table to the right, and the results are K-R 20 = 0.75

The interpretation of the K-R 20 value is similar to that of Kappa. A K-R 20 of <0.2 is considered poor agreement, 0.21-0.4 fair, 0.41-0.6 moderate, 0.61-0.8 strong, and more than 0.8 near complete agreement.

The original descriptions of K-R 20 provided no test of statistical significance or confidence interval, although these can be obtained using the Cronbach's Alpha algorithm.

References

Kuder, G. F. ; M. W. Richardson (1937)The theory of the estimation of test reliability. Psychometrika, 2: 151-60

https://en.wikipedia.org/wiki/Kuder%E2%80%93Richardson_formulas Kuder Richardson Coefficient on Wikipedia

Data Entry The data is a matrix of numbers with multiple columns   - Each rows represents a subject   - Each column a judge or an instrument of evaluation   - Each cell is 1 for positive or 0 for negative that judge (col) and subject(row)

the following is the algorithm for Kuder Richardson Coefficient of Reliability

dat = ("
0	1	1	0
1	1	1	1
0	1	0	0
0	0	1	0
1	1	1	1
       ")

mx = read.table(textConnection(dat),header=FALSE)
rows = nrow(mx)
cols = ncol(mx)
SumPQ = 0   # sum of pq for each item or col
for(j in 1:cols)
{
  p = 0
  for(i in 1:rows)
  {
    if(mx[i,j]==1)
    {
      p = p + 1
    }
  }
  p = p / rows
  SumPQ = SumPQ + p * (1.0 - p)
}
ex = 0
exx = 0
for(i in 1:rows)
{
  p = 0
  for(j in 1:cols)
  {
    if(mx[i,j]==1)
    {
      p = p + 1
    }
  }
  ex = ex + p
  exx = exx + p * p
}
v = (exx - ex*ex/rows) / (rows)  # variance between subjects
kr_20 = (1.0 * cols / (cols - 1.0)) * ((v - SumPQ) / v)
kr_20  # Kuder Richardson Coefficient of Reliability

The result is

> kr_20  # Kuder Richardson Coefficient of Reliability
[1] 0.7536232