We wish to conduct a controlled trial on the effect of introducing additional fertiliser to the feeding ground of calves on their weight gain. As we cannot randomize the calves because fertilisers can only be applied to paddocks, we will use the cluster randomisation model, using each paddock as the cluster for randomisation.

Step 1 : Find ssiz_individual

We use the following parameters to determine the sample size based on individual randomisation

• Type I Error α = 0.05
• Power (1 - β) = 0.8
• We expect that the calves would gain some 15Kg over a 3 months period, with the Standard Deviation some 6Kg. Our research hypothesis is that with additional fertilisers, the calves would gain an extra 3 Kg during that period. The effect size is therefore roughly half a Standard Deviation (0.5)

We looked up the sample size requirement table in the sample size for unpaired means table page, and found that we will require 65 calves per group if we were to randomize on individual calves. In other words, ssiz_individual, s = 65

Step 2 : Find Intraclass Correlation Coefficient ρ

We decided to obtain the likely Intraclass Correlation Coefficient ρ for our experiment by a pilot study, where we placed some calves into a number of paddocks, and measure their growth (weight gain over 3 months), and found the results as in the table to the left.

We use the first program in the sample size panel of this page to obtain a workable Intraclass Correlation Coefficient, which is ρ = 0.0889. This is, as expected a very low level of correlation, as the calves are allocated at random.

Step 3 : Sample Size Adjustment

Using the third program in the sample size calculation panel, we can view the sample size required in each cluster for a range of cluster numbers, as shown in the table to the left.

It can be seen that, as the number of paddocks (clusters) to be used decreases, the number of calves per paddock (sample size per cluster) increases exponentially. The sample size reaches infinity when the number of cluster is below 6.

Although the sample size per cluster continues to decrease as the number of clusters increases, the changes in the total group sample size becomes increasingly minor.

From such an analysis, and depending on the costs (financial, time, and effort) of different aspects of the experiment, the most efficient combination with the same power can be selected. In this example, the best combinations would seem to be from 9 paddocks with 19 calves per paddock (total 171 calves per group) to 18 paddocks with 5 claves per paddock (total 90 calves per group), depending on whether managing calves or paddocks to be more costly.

Step 4 : Analysis of Results

Following calculations in the previous section, we decided to use 9 paddocks (clusters) per group, placing 20 calves in each paddock. Those in Grp 2 were controls, and additional fertilisers were added to the Grp 1 paddocks. The calves were weighed at the beginning of the experiment and 3 months later. The weight gain in Kg was the outcome. The table to the right shows means and SDs in weight gain from each paddock (cluster). The results are analysed using the two sample t test, with and without adjustments for intraclass correlation (ρ)

The overall mean and SD of the two groups are as in the table to the left. The difference between the two means is 3.7Kg, the Standard Error of the difference is 0.62, and probability of Type I error (α) p<0.0001. These results have not been adjusted by the intraclass correlation coefficient (ρ), so are references only, for comparisons with the final results.

Intraclass Correlation Coefficient, calculated from the data, is ρ = 0.0084.

The adjusted Standard Error of the difference is now 0.67, and the 95% confidence interval of the difference is 2.27 to 5.11 Kgs. This is the final results, showing a significant difference between the two groups, and we can conclude that adding fertilisers to the feeding paddocks increases the growth of calves.

Please note that, in this example, the correction is so small as to be trivial, because of low ρ value. This is expected as allocation to different paddockes are randomized, and there is not much difference between the paddocks, so there would not be much interaclass correlation.

Options to analyse the data using non-parametric statistical methods: The first and the third columns of the input data, group designations and cluster means in the table above and to the right, can be used for non-parametric statistical tests. These tests can be used as a check and comparison with the adjusted two sample t test, because of uncertainties of normal distribution, or very small sample size within the clusters.

StatsToDo provides these test is the unpaired difference programs page

• Wilcoxon Mann-Whitney Test shows z=3.4473, p=0.0003, confirming a significant
• The Robust Rank Order Test (Mann-Whitney U Test) results in U = 15.9153, p<0.05, confirminga significant difference
• Permutation Test shows Type I Error (α) of 0.0032 (1 tail) or 0.0065 (2 tail), confirming significant difference

We wish to conduct an experiment on the effect of introduce a student encouragement protocol into schools to reduce absenteeism, defined as having missed at least 1 scheduled class in a term. Given that such a protocol has to be introduced to a whole school, we decide to use the cluster randomization model.

Step 1 : Find ssiz_individual We use the following parameters to obtain the required sample size as if randomization is based on individuals.

• Type I Error α = 0.05
• Power (1 - β) = 0.8
• We estimate the average absenteeism rate in a school is about 30% (0.3), and expect the encouragement protocol to reduce this to 15% (0.15)

We looked up the sample size requirement table in the sample size for comparing two proportions page, and found that we will require 121 students per group. In other words, ssiz_individual s = 121

Step 2 : Find Intraclass Correlation Coefficient ρ

We decided to carry out a pilot simulation, by examining the absenteeism in a number of schools, and found the results as in the table to the left (Pos=number with absenteeism present, Neg = number with no absenteeism). Please note : In a real pilot study, many more clusters would be used to obtain a stable and robust ρ.

We used the second program in the sample size panel to calculate the Intraclass Correlation Coefficient ρ. The program firstly convert the number of positives and negatives into 1s and 0s, then calculate the means and SDs for each cluster, as shown in the table to the right. From this table, we estimate the Intraclass Correlation Coefficient ρ to be 0.197

Step 3 : Sample Size Adjustment

Although we can use the third or fourth programs of the sample size calculations to calculate one at a time the required number of individual within each cluster when the number of cluster in each group is pre-determined, or the number of individuals in each cluster when the number of clusters per group is pre-determined, we can also use either program to test a range of combination.

Using the third program to determine the number of individuals in each cluster, with a range of numbers of clusters, we produced the results as shown in the table to the left.

It can be seen that, as the number of schools (clusters) to be included decreases, the number of students per school (sample size per cluster) increases exponentially. The sample size reaches infinity when the number of cluster is below 24.

Although the sample size per cluster continues to decrease as the number of clusters increases, the changes in the total group sample size becomes increasingly minor.

As the main costs of such a program are related to selecting and introducing the encouragement protocol into schools, the decision can be based on finding the minimum number of schools (clusters) that contain sufficient number of individuals (students). As most schools have more than 84 students, a reasonable decision can be to have 25 schools (clusters) per group.

Step 4 : Analysis of Data

We recruited 50 schools that had at least 84 students each that are available for the study, and randomly divided the schools into 2 groups. Those in Grp 2 were controls and those in Grp 1 were introduced to the encouragement program. The number of absentees in one term were collated from each school and presented in the table to the right.

For comparing two proportions, 4 options are available, 3 of which adjusted for intraclass correlation coefficient (ρ), and can be used to draw conclusions from the data

• Option 1 : The Two Samples t Test

  Grp	Npos	Nneg	Ntotal	Proportion pos
  1	473	2793	3266	0.1448
  2	716	2407	3123	0.2293
  Total	1189	5200	6389	0.1861

  The summary table is as shown to the right. The proportions of positives are treated as if it is a measurement, and the mean and Standard Deviations of the two groups estimated. The difference (Mean Prop1 - Mean Prop2) = -0.086, SE = 0.0247, df = 48, t = -3.4817, p = 0.0011.

  Although this shows a significant difference between the two groups, the results should be considered only as preliminary as no allowance is made of the Intraclass Correlation.

• Option 2 : The unadjusted and adjusted Pearson's Chi Squares Test estimates from the numbers of positive and negative outcomes in all the clusters in the two groups.

  The Intraclass Correlation Coefficient ρ is calculated from the data. ρ = 0.0425

  Unadjusted Chi Sq = 49.7836 df = 1 p <0.0001
  Adjusted Chi Sq = 7.721 df = 1 p = 0.0055

  The adjusted (corrected by ρ) Chi Square should be used to interpret the results, and this indicates that the proportion of absenteeism in the two groups are significantly different

• Option 3 : Confidence Interval of the Difference Difference in proportions (Grp1 - Grp2) = -0.0844, Adjusted SE_diff = 0.0246
  95% CI = Diff ± 1.96SE = -0.1328 to -0.0361

  This shows that the 95% confidence interval of the difference in proportion of absenteeism in group 1 is 0.04 (4%) to 0.13 (13%) lower than in group 2. As this range does not cross the null value of 0, we can conclude that it is statistically significant

• Option 4 : Confidence Interval of Odds Ratio Between the Two Groups

  Odds Ratio = 0.5693, Log(Odds Ratio) LOR = -0.5633
  Adjusted (corrected for ρ) SE_LOR = 0.1665
  95% CI = Exp(LOR-SE) to Exp(LOR + SE) = 0.4108 to 0.789

  This is a different expression to Test 4, Odds Ratio instead of difference in proportions. The Odds of absenteeism from group 1 is 0.41 to 0.79 that from group 2. As this range does not cross the null value of 1, we can concluded the odds of absenteeism in group 1 is significantly lower than group 2.

• Options 2, 3, and 4 can all be used to draw conclusions on the data. They performs similar tests, except that the assumed distribution patterns used are different.

Options to analyse the data using non-parametric statistical methods: The group designation and proportion of positives in the table above and to the right, can be used for non-parametric statistical tests. These tests can be used as a check and comparison with the adjusted two sample t test, because of uncertainties of normal distribution, or very small sample size within the clusters.

StatsToDo provides these test is the unpaired difference programs page

• Wilcoxon Mann-Whitney Test shows z=-2.7456, p=0.0003, confirming a significant difference
• The Robust Rank Order Test (Mann-Whitney U Test) results in U = -3.0562, p = 0.0011, confirming significant difference
• Permutation Test aborted, the amount of data exceeded capacity of program

Estimate Intraclass Correlation Coefficient ρ

• ρ from normally distributed measurements: The data are summary of measurements, collected from a number of clusters, in a pilot study. It is a matrix where each row is from a cluster, and the 3 columns are n, mean, and SD of data in that cluster. The results are the within and between cluster variance, from which ρ is calulated
• ρ from binomially distributed numbers of negative and positive outcomes: The data are summary of counts, collected from a number of clusters, in a pilot study. It is a matrix where each row is from a cluster, and the 2 columns are numbers of positive and negative outcomes in that cluster. The program converts the binomial probability of positive outcome as mean and SD, from which ρ is calulated

Estimate Sample Size
Using pre-estimated sample size for comparing 2 groups
and Intraclass Correlation Coefficient ρ

Sample Size: for simple comparison between two groups are firstly estimated, using sample size for comparing two means and sample size for comparing two proportions

Intraclass Correlation Coefficient ρ: is used to adjust the sample size for using in cluster randomization. ρ can be assigned the value 0 if the researcher is convinced that all cases are truely randomly allocated to clusters and groups, and the environment in all clusters are the same. This however is seldom entirely true, and some sort of pilot study to estimate ρ is usually necessary, using the first two programs to estimate ρ from pilot data

Calculating sample size: Two programs are available.

• Calculate sample size from Intraclass Correlation Coefficient ρ and number of cluster. The data is in 3 columns, simple sample size for comparing two groups, ρ, and the number of clusters to use. The program estimate the number of cases to be allocated to each cluster, and therefore the total number of cases required in each group.

  Although only a single calculation (single row) is necessary, often placing a number of rows, with multiple options of different number of clusters may be useful to help select the best combination of cluster number and cluster size

• Calculate sample size from Intraclass Correlation Coefficient ρ and number of cluster. The data is in 3 columns, simple sample size for comparing two groups, ρ, and the number per clusters (number of cases in each cluster) to use. The program estimate the number of clusters in each group, and therefore the total number of cases required in each group

  Although only a single calculation (single row) is necessary, often placing a number of rows, with multiple options of different number in each clusters may be useful to help select the best combination of cluster number and cluster size

Cluster Analysis Comparing Two Means

• Each row is data from a cluster
• Col 1 is group designation. It is usually a number, but can be a name in text. Text must be a single word. Groups are assgned in alphabetical order
• Col 2 is sample size in the cluster
• Col 3 is the mean of the cluster
• Col 4 is the Standard Deviation of the cluster

The difference is mean(group 1) - mean (group 2), with the two groups in alphabetical order

α<0.05, or the 95% confidence interval not traversing the null value of 0, can be used to identify that the difference is statistically significant.

The program also produces a table with 2 columns, the group designation and mean for each cluster. This table can be used for non-parametric comparisons using programs in the unpaired difference programs page

Cluster Analysis Comparing Two Proportions

• Each row is data from a cluster
• Col 1 is group designation. It is usually a number, but can be a name in text. Text must be a single word. Groups are assgned in alphabetical order
• Col 2 is number of positive cases in the cluster
• Col 3 is the number of negative cases in the cluster

The program next performs 3 tests to compare the proportions in the two groups. The 3 tests are similar, but assumes different distribution of data.

• The chi squares test, before and after adjustment by intraclass correlation coefficient ρ
• The difference in proportions of positive cases, before and after adjustment by intraclass correlation coefficient ρ
• The odds ration of positive cases, before and after adjustment by intraclass correlation coefficient ρ
• Users can choose which results to present, but the odds ratio seems to be the easiest to understand, and are frequently presented.

The program also produces a table with 2 columns, the group designation and proportion of positive cases for each cluster. This table can be used for non-parametric comparisons using programs in the unpaired difference programs page