Content Disclaimer Copyright @2020. All Rights Reserved. |

**Links : **Home
Index (Subjects)
Contact StatsToDo

Explanations and References
This page provides 3 statistical tests for paired differences in scalar measurements. These are the Paired t Test, the Wilcoson, Paired Signed Rank Test, and the Permutation Test for paired differences.
Javascript Program
## Paired t Test
Please note: the data is computer generated and does not represent any research information.
We purports to conduct a study to test whether the order of birth resulted in differences in the birth weight of twins. We obtained the birth weights (grams) from 20 sets of twins, which are shown in the first two columns in the table to the right, and the differences (Twin1 - Twin 2), as shown in the third columns. The calculations are as follows -
Number of Paires = 20
Mean Difference = -101 Standard Deviation of Difference = 456 Standard Error of Mean = 105 t Test : t = -0.9659 df = 19 p(α) = 0.17 (1 tail) and 0.35 (2 tail) 95% Confidence Interval of Paired Difference (1 tail) = >-282 or <80 95% Confidence Interval of Paired Difference (2 tail) = -320 to 118 ## Wilcoxon Paired Signed Rank Test
Please note: the data is computer generated and does not represent any research information.
We purports to study whether a new analgesic is effective in relieving headaches. We ask the subjects to describe their headache as none (0), some (1), moderate (2), and severe (3), a 4 point scale (0 to 3), before and after administering the analgesics, and use the paired differences to evaluate the analgesics. We collected data from 22 subjects. The headache scales are as shown in the first 2 columns, and the paired difference (Before - After) in column 3 of the table to the left.
There were 8 subjects whose headache scores did not change (0), and these are not included in the table of counts. On the negative side, there were:
As the numbers included in the count is 14, less than 16, Segal's Table H is used to determine statistical significance, and this is p<0.05. We can therefore conclude that headaches decreased significantly after receiving the analgesic. ## Permutation Test for paired differencesThe general principle underlying the permutation of paired differences is that, in a randomly allocated study, the data obtained could have been in either of the paired measurements. In ther words, each paired difference can either be + or - of that difference. The test therefore consists of calculating every possible permutation of the paired differences, and examine the results. If the results from the original data is near the extremes (e.g. less than 5 percentile or more than 95 percentile in a one tail model), then a decision can be made that it is unlikely to be null and therefore statistically significant. The advantages of using the Permutation test are : - Exhaustive permutation allows the calculation of the precise probability that the data presented is null, so the tests calculate the Type I Error (α), with the power (1-β) of 100%.
- The test is not dependent on any assumption of data distribution, so they can be used in any regular interval data (where 10-9 is the same as 4-3). The test can therefore be used on parametric measurements, ratios, variances, and time.
- Because of the above two characteristics, the test can be used with a very small sample size
The disadvantages of using the test are related to the computation resources (time and memory) required. The number of permutation is 2 In theory, the Permutation Test can cope with any number of pairs. However, α<0.05 cannot be computed with less than 6 pairs unless the differences are uniformly in one direction, and computation will take an unacceptably long time with 22 pairs or more. The Permutation Test is therefore ideal for handling small sets of interval data with uncertain distributions. With larger sample size, Wilcoxon PSRT for non-parametric data or Paired t test for parametric data should be preferred. The mathematical argument of the Permutation Test is as follows - In a pair of measurements, the null hypothesis is that there is no difference between the pair. In other words, that the values observed can be in either of the pair.
- The Permutation Test therefore consists of examining the sum of paired differences, in all permutations where the
values in each pair are in either groups. The total permutation is therefore 2
^{n}, were n is the number of pairs. - The sum of differences in the original data is then compared with all possible outcomes, so that its position (thus probability) can be estimated.
The following example uses the same data as that presented in the Paired t Test. - There are 22 pairs of birth weights, so the number of possible permutations are 2
^{22}= 1048576 - The sum of paired differences in the input data = -2021
- There are 173010 permutated sums that are less than that from the input data (<-2021)
- There are 262 permutated sums that are the same as that from the input data (=-2021)
- There are 875304 permutated sums that are more than that from the input data (>-2021)
- Therefore, there are 262 + 173010 = 173272 permutated sums that are equal or less than from the input data (<=-2021)
- Therefore, the probability of having a permuted sum equal or less than that of the input data is 173272 / 1048576 = 0.1652
- Putting this another way
- The input data is at the 16.5
^{th}percentile of all possible permutationed values - Statistical significant (α) = 0.1652 (1 tail) and 2 * 0.1652 = 0.3305 (2 tail)
- We can conclude that paired differences is not significant differenct to null (0)
- The input data is at the 16.5
## Comparison of the 3 methodsUsing the same twin data by the 3 tests provides the following results- Paired t Test: α = 0.1731 (1 tail)
- Wilcoxon PSRT: α = 0.1753
- Permutation Test: α = 0.1652 (1 tail)
## ReferencesPaired t Test :
Armitage P. Statistical Methods in Medical Research (1971). Blackwell Scientific Publications. Oxford. P.189-207
## Program 1: Paired t test# Paired t test # data entry dat = (" V1 V2 3163 3124 3245 2807 3391 3014 2547 2727 3042 3254 3200 3826 3115 2596 3294 2952 3019 3279 3222 3325 2831 2984 3043 2765 2646 3109 3327 2757 3182 3781 2984 3061 2878 3658 2770 3665 3092 2739 2735 3324 ") df <- read.table(textConnection(dat),header=TRUE) # conversion to data frame df$Diff <- df$V1 - df$V2 #df # optional display of input data and paired differences # Clculation: paired t n = nrow(df) # sample size degFm = n - 1 # degrees of freedom mean = mean(df$Diff) # mean of paired difference sd = sd(df$Diff) # Standard Deviation of paired differences se = sd / sqrt(degFm) # Standard Error of mean t = mean / se # paired t p = 1 - pt(abs(t),degFm) # probability of Type I Error α # 95% confidence interval t = qt(1 - 0.05, df=degFm) # t for 95% CI 1 tail ll1 = mean - t * se # lower limit 1 tail ul1 = mean + t * se # upper limit 1 tail t = qt(1 - 0.025, df=degFm) # t for 95% CI 2 tail ll2 = mean - t * se # lower limit 2 tail ul2 = mean + t * se # upper limit 2 tail # result output c(n, mean, sd) # sample size, mean difference and SD difference c(degFm, se, t, p, p*2) # deg freedom, Standard Error t, p(α) 1 and 2 tail c(ll1, ul1) # 95% CI 1 tail c(ll2, ul2) # 95% CI 2 tailThe results are as follows > # result output > c(n, mean, sd) # sample size, mean difference and SD difference [1] 20.0000 -101.0500 455.9936 > c(degFm, se, t, p, p*2) # deg freedom, Standard Error t, alpha (1 and 2 tail) [1] 19.0000000 104.6121044 2.0930241 0.1731023 0.3462046 > c(ll1, ul1) # 95% CI 1 tail [1] -281.93822 79.83822 > c(ll2, ul2) # 95% CI 2 tail [1] -320.0057 117.9057 ## Program 2 Wilcoxon Paired Signed Rank Test# Data input dat = (" V1 V2 3 1 2 1 2 0 3 0 1 1 1 0 2 2 1 2 1 1 3 1 1 2 1 1 1 1 3 0 3 1 1 2 1 1 1 3 1 1 3 1 3 2 1 1 ") df <- read.table(textConnection(dat),header=TRUE) # conversion to data frame df$Diff <- df$V1 - df$V2 # paired difference df # optional display of input data and paired difference # Make Wilcoxon table with non-equal cases x <- vector() # rank of non-zero differences in abs values y <- vector() # neg or pos ranks for(i in 1:nrow(df)) { v = df$Diff[i] # v = diff if(v !=0) # non-zero v { x <- append(x, abs(v)) # abs diff # abs(diff) y <- append(y, v / abs(v)) # sign + or - 1 # -1 or +1 } } n = length(x) # number of non-zero differences x <- rank(x) # abs diff ranked #x mx <- xtabs(~ x + y) # matrix +/- ranks # calculate Wicoson W ranks <- rownames(mx) # ranks as value array #ranks # optional display during debug tp = 0; # T+ sum of positive ranks tm = 0; # T- sum of negative ranks for(i in 1 : nrow(mx)) { r = as.numeric(ranks[i]) # rank of ith row tm = tm + mx[i,1] * r # T+ sum of positive ranks tp = tp + mx[i,2] * r # T- sum of negative ranks } t = tp if(tm>t) t = tm # use the larger of the 2 sums for calculation # significance testing if(n>15) # large sample size use z and its probability { z = (t - n * (n + 1) / 4) / sqrt(n * (n + 1) * (2 * n + 1) / 24) p = 1-pnorm(abs(z)) res = sprintf("z=%.4f p=%.4f", z, p) } else # small sample size use coefficients { Sig <- c(c(0,0,0),c(0,0,0),c(0,0,0),c(0,0,0),c(0,0,0),c(15,0,0), # 1-6 c(19,0,0),c(24,28,0),c(31,45,0),c(37,42,0),c(45,50,55), # 7-11 c(53,59,65),c(61,69,76),c(70,79,87),c(80,90,99),c(90,100,111)) # Table H. Siegal p.332-334 mxSig <- matrix(data = Sig, nrow=3,ncol=16) # converted to matrix for reference nn = n + 1 if(n<5) { res = "p = n.s." } else if(mxSig[3,nn]>0 && t>=mxSig[3,nn]) { res = "p = <0.001"; } else if(mxSig[2,nn]>0 && t>=mxSig[2,nn]) { res = "p = <0.01"; } else if(mxSig[1,nn]>0 && t>=mxSig[1,nn]) { res = "p = <0.05"; } else { res = "p = n.s."; } } # result output mx # matrix of +/- ranks c(nrow(df), n, tm, tp) # sample size, pairs different, Sum Rank + and - res # statistical significanceThe results are as follows - Table of neg and pos ranks. x = absolute rank, y= - or +
- initial sample size (nrow), number of non-zero differences (n), sums of - (tm) and + (yp) ranks
- Statistical significance (p, α)
> # result output > mx # matrix of +/- ranks y x -1 1 3.5 3 3 9.5 1 5 13.5 0 2 > c(nrow(df), n, tm, tp) # sample size, pairs different, Sum Rank + and - [1] 22 14 20 85 > res # statistical significance [1] "p = <0.05" > ## Program 3: Permutation of paired differencesSection 1: declare global variabes
# Global variables rows <- 0 # number of rows (pairs) sumDiff <- 0 # sum of paired difference from input data arV <- vector() # array of abs(paired diff) from all pairs nLess <- 0 # number of pairs with values less than sumDiff nSame <- 0 # number of pairs with values same as sumDiff nMore <- 0 # number of pairs with values more than sumDiff Section 2: subroutine for recursive calculation of paired differences
All variables except col, sum, sumPlus and sumMinus are globally assigned and read Recurse <- function(col,sum) # col = pair number, sum = sum of paired diff { sumPlus = sum + arV[col] # sum with +ve [aired diff sumMinus = sum - arV[col] # sum with -ve [aired diff if(col==(rows)) # reaching last row { if(abs(sumPlus-sumDiff)<1e-10) # sum == sumDiff (allowing minor differences by binary math) { assign("nSame", nSame+1, envir = .GlobalEnv) # add to nSame } else { if(sumPlus<sumDiff) { assign("nLess", nLess+1, envir = .GlobalEnv) # add to nLess } else { assign("nMore", nMore+1, envir = .GlobalEnv) # add to nMore } } if(abs(sumMinus-sumDiff)<1e-10) { assign("nSame", nSame+1, envir = .GlobalEnv) # add to nSame } else { if(sumMinus<sumDiff) { assign("nLess", nLess+1, envir = .GlobalEnv) # add to nLess } else { assign("nMore", nMore+1, envir = .GlobalEnv) } } return() } arCount <- Recurse(col+1,sumPlus) # recursive call if not end row arCount <- Recurse(col+1,sumMinus) # recursive call if not end row return () } Main program
# input data dat = (" V1 V2 3163 3124 3245 2807 3391 3014 2547 2727 3042 3254 3200 3826 3115 2596 3294 2952 3019 3279 3222 3325 2831 2984 3043 2765 2646 3109 3327 2757 3182 3781 2984 3061 2878 3658 2770 3665 3092 2739 2735 3324 ") df <- read.table(textConnection(dat),header=TRUE) # conversion to data frame rows <- nrow(df) # number of pairs df$Diff <- df$V1 - df$V2 # paired differences #df # optional check t debug sumDiff <- sum(df$Diff) # sumDiff, sum of paired diff from input data arV <- abs(df$Diff) # vector of abs(paired differences for use in permutation col = 1 # start permutation with first row sum = 0 # start permutation with sum of diff = 0 arCount <- Recurse(col, sum) # permutate calculations subroutine # Assemble results nTot = nLess + nSame + nMore # total number of possible permutations pLess = nLess / nTot # proportion with smaller effect size pSame = nSame / nTot # proportion with the same effect size pMore = nMore / nTot # proportion with larger effect size # probability of obtaining reference effect size or more extreme p = min(pLess,pMore) + pSame # significance (probability, α) # result output c(rows, sumDiff) # number of pairs and sum of paired diff from input data c(nTot, nLess, nSame, nMore) # result numbers c(pLess, pSame, pMore) # result proportions p # probability of sum(x) or more extreme (p, α)The results are as follows > # result output > c(rows, sumDiff) # number of pairs and sum of paired diff from input data [1] 20 -2021 > c(nTot, nLess, nSame, nMore) # result numbers [1] 1048576 173010 262 875304 > c(pLess, pSame, pMore) # result proportions [1] 0.1649951935 0.0002498627 0.8347549438 > p # probability of sum(x) or more extreme (p, α) [1] 0.1652451 |