McNemar's Test
| + | - |
+ | a(n++) | b(n-+) |
- | c(n+-) | d(n--) |
The McNemar's test evaluates changes in related or paired binomial attributes,
whether in one direction is significantly greater than that in the
opposite direction.
The data structure is as shown in the table to the right, where columns represents before and rows represents after.
- a(n++) represents the numbers with positive attribute, unchanged
- d(n--) represents the numbers with negative attribute, unchanged
- b(n-+) represents the numbers with positive attribute that changed to negative
- c(n+-) represents the numbers with negative attribute that changed to positive
- b and c are used to calculate Chi Square
Chi Squares = (abs(b-c)-1)2 / (b+c) degree of freedom = 1.
Example
We are going to study whether a major speech by the leader of a political party
will affect the voting intentions of the public.
We recruited 250 subjects, and recorded their voting intensions before and after the
speech.
There were 130 subjects who intended to vote against the party before the speech,
45 (18% of the total) of them changed their voting intension towards favouring the party after the speech.
There were 120 subjects who intended to vote for the party before the speech,
24 (9,6% of the total) of them changed their voting intension to against the party
after the speech.
we found Chi Square = 5.8, df = 1, p=0.02. This indicates that
the speech was effective in changing voting intensions towards favouring the party.
Sample Size and Power
Sample size for the McNemar's test estimates the number of pairs (befre and after) required for anticipated p1 (proportion of cases that will switch one way (+ to -) and p2 (proportion of cases that will switch the other way (- to +). The other parameters are Type I error (α) and power (1-β).
Power estimation is the reverse of sample size. It estimate the power of the observed given α, sample size (the total number of pairs in the data), p1 and p2 observed
References
Siegel S and Castellan Jr. N J (1988) Nonparametric Statistics for the Behavioral Sciences 2nd. Ed.
McGraw Hill, Inc. Boston Massachusetts. ISBN 0-07-057357-3 p 75
Machin D, Campbell M, Fayers, P, Pinol A (1997) Sample Size Tables for
Clinical Studies. Second Ed. Blackwell Science IBSN 0-86542-870-0 p. 70 - 71
Program 1: McNemar Test
# McNemar's Test
n1 = 5 # number of cases switched from + to -
n2 = 12 # number of cases switched from - to +
#program starts
chiSq = (abs(n1 - n2) - 1)^2 / (n1 + n2) # McNemar's Chi Sq
chiSq
p = pchisq(chiSq, df=1)
p
The result is
> chiSq
[1] 2.117647
> p
[1] 0.8543899
Program 2: Sample size for McNemar's test
# parameters
alpha = 0.05 # Type I Error
power = 0.8 # 1 - beta
tail = 2 # 2 tail model
p1 = 0.25 # expected proportion that switches from + to -
p2 = 0.125 # expected proportion that switches from - to +
# program starts
zb = qnorm(power)
za = qnorm(1 - alpha / tail)
pie = p1 + p2
theta = p1 / p2
if(theta<1) theta = 1.0 / theta
tp = theta + 1;
tm = theta - 1;
top = za*tp + zb*sqrt(tp*tp-tm*tm*pie)
top = top * top
bot = tm * tm * pie
ssiz = ceiling(top / bot);
ssiz # number of pairs
The results is
> ssiz # number of pairs
[1] 186
Program 3: Power estimation
# parameters
alpha = 0.05 # Type I Error
ssiz = 186 # sample size, number of pairs in data
tail = 2 # 2 tail model
p1 = 0.25 # proportion observed that switches from + to -
p2 = 0.125 # proportion observed that switches from - to +
# program starts
za = qnorm(1 - alpha / tail)
pie = p1 + p2
theta = p1 / p2
tp = theta + 1;
tm = theta - 1;
if(theta<1) theta = 1.0 / theta
top = (tm * sqrt(ssiz * pie) - za * tp);
bot = sqrt(tp * tp - tm * tm * pie);
zb = top / bot;
pw = pnorm(zb)
pw # power (1 - beta)
The result is
> pw # power (1 - beta)
[1] 0.8000049