Related Links:
Receiver Operator Characteristics (ROC) Explained Page
Introduction
Example
References
This page supports the program from the . The algorithm is derived from publications
listed in the references panel
The Need for Curve Fitting the Receiver Operator Characteristic (ROC) Curve
The underlying assumptions of the ROC curve is that the Outcome is binary, but the Test is a continuous variable. In
establishing the ROC from a set of reference data, it is assumed that the sample size provided is large enough, and the
granularity of the values in the data set is fine enough, to produce a reproducible curve.
The reference data set unfortunately often failed these assumptions. A major problem is that test measurements are often
rounded to clinically meaningful intervals, so that the granularity of the data is coarse, and changes occurred in a
series of steps rather than in a smooth curve.
Furthermore, with rounding of values, often there are multiple cases with the same test value, and to include or exclude
values at the cut off makes a large difference to the subsequent use. Also the changes to the False Positive Rate (FPR) is not
always accompanied by any change to the True Positive Rate (TPR)
An example of this is shown in the plot to the left. If the cut off for decision making is 155 (the black line), data points
can be clearly separated into above or below that value. However, if the cut off value is set at 154, there are 3 data points
in the Outcome Positive group and 2 in the Outcome Negative group that have this exact value. Whether these are considered above
the cut off, below the cut off, or excluded from consideration, will have a large impact on subsequent results produced.
The resulting ROC plot would be as the black line in the plot to the right.
Curve fitting would result in a ROC curve that has the closest fit with the data, yet allows a continuous interpretation along
the whole range of the Test values, as shown in the red line in the plot to the right.
Test Value  FPR  TPR 
162.5  0.96  1 
162  0.92  1 
161.5  0.84  1 
160  0.84  0.96 
159.5  0.76  0.96 
159  0.68  0.96 
158.5  0.48  0.92 
157.5  0.4  0.88 
157  0.36  0.88 
156.5  0.36  0.8 
156  0.24  0.68 
155.5  0.2  0.6 
154  0.12  0.48 
153.5  0.04  0.4 
153  0.04  0.36 
152.5  0  0.28 
151.5  0  0.24 
151  0  0.2 
150.5  0  0.16 
150  0  0.12 
149  0  0.08 
This example uses the default input data from the program in the . This set of the data,
in turn is obtained from producing a Receiver Operator Characteristics (ROC) curve from the first 2 columns of data in the
Receiver Operator Characteristics (ROC) Program Page
. The original data are computer generated and do not represent any real information.
We wished to establish maternal height (cms) as the Test to predict the need for Caesarean Section as the Outcome. Using
the first 2 columns of default example data from Receiver Operator Characteristics (ROC) Program Page
, We produced a table of False and True Positive
Rates. A subset of columns from that table is used as the default example in the for curve
fitting. The data is as shown in the table to the right. Test value is maternal height in cms, FPR and TPR are False and True
Positive Rates for that value.
Curve Fitting
Two curve fitting procedures are carried out by the program, both use the False Positive Rate as the independent variable.
 Curve fitting for False Positive Rate.
 The independent variable (x) is the maternal height
 The y variable is the logit of the FPR, so that y = log(FPR(1FPR))
 The standard Regression analysis as in the Correlation and Regression Program Page
is carried out, so that the formula
y=a+bx is produced. In this example, a = 101.6295 and b = 0.6439
 The result is therefore Logit(FPR) = 101.6295 + (0.6439)(maternal height in cm), or
FPR = 1 / (1  exp((101.6295 + (0.6439)(maternal height in cm))))
 Curve fitting for True Positive Rate.
 The logit of FPR and TPR are firstly calculated, so that u = log(FPR(1FPR)) and v = Log(TPR(1TPR)
 The y (dependent) variable is then calculated as the Log Odds Ratio y = uv
 The x (independent variable is calculated as the Standard Error of the Log Odds Ratio x = u+v
 The standard Regression analysis as in the Correlation and Regression Program Page
is carried out, so that the formula
y=a+bx is produced. In this example, a = 2.2319 and b = 0.0445
 The formula to obtain TPR from FPR is TPR = 1 / (1 + exp(a / ( 1  b)) exp(log((1 FPR) / FPR) * ((1 + b) / (1  b))))
 The term (1 + exp(a / ( 1  b)) is a constant, and in this example = 0.118
 The term ((1 + b) / (1  b)) is also a constant, and in this example = 0.9148. Please note that b is changes in Standard
error, and in this curve fitting it contains mostly rounding errors, and is not significantly different from 0. As a
consequence, the second term ((1 + b) / (1  b)) is not significantly different from 1, and can be left out without
materially affecting the end results. However the program in the has included
this term.
 The result is therefore TPR = 1 / (1 + 0.118(exp(log((1FPR)/FPR)(0.9148))))
Data used in curve fitting
The program provides 3 options for the user to choose, each has its advantages and disadvantages
 A: All data included : These include all data presented, including where the True Positive Rate (TPR) or the False Positive Rate (FPR) have values of 0 or 1.
The advantage is that the fitted curve is a true reflection of all data. The disadvantages are that this includes numerous nonoverlapping values where the False Positive Rate is 0 (FPR=0) or where the True Positive Rate is 1 (TPR=1). Although the overall fitting is more accurate, the part that is of interest to the researcher, where values from the two groups overlapped, is distorted by extreme nonoverlapping values that provide no prediction insight.
 B: Only data with values >0 and <1 are included : This includes only those data points where values from the two groups overlapped.
The advantage is that the resulting curve is more representative of those data that are of interest to the researcher. The disadvantage is that not all the data available have been included.
As this option most closely reflect the researcher's needs, B: Only data with values >0 and <1 are included is the recommended option
 C: Only data with True Positive Rate >=0.5 and False Positive Rate <=0.5 are included : This option is provided so that the calculations are comparible with the original reference from Moses et.al.
The idea is that all data points that would lead to erroneous prediction should not be excluded. This makes sense when curve fitting is used in metaanalysis, but is not recommended for general use, as many researchers requires the prediction properties of a test through out where the values from the two outcomes overlapped.
Results and their Interpretations
The program in the produces one plot and two tables. Using the default example input data,
the plots produced from different options are shown to the left.
 The red curve uses algorithm A, inclusive of all data presented, providing 0.00001 to replace 0,
and 0.99999 to replace 1 to avoid division by zero. This produces the most accurate fit, but the
curve is dominated by the numerous data points that hugged the borders of the ROC space, so that
the fit is poor in the range where the two outcomes overlapped.
 The green curve uses algorithm C, including only data where TPR>=0.5 and FPR<=0.5. This was
the option suggested by Moses et.al. (see references) for metaanalysis of prediction studies.
Although this produced a very good fit for data in the top left quadrant, it fitted poorly for
the rest of the scale.
 The black curve uses algorithm B, including only data that have values >0 and <1. In other
words, it uses those data points where the values from both outcome groups overlapped. This curve
produced the best fit for that part of the data that are of interest to the researcher, and
therefore recommended.
The plot to the right and the two tables on this page show the relationship between original
data (black circular dots) and the curve fitted ROC curve using algorithm B, including data that
have values >0 and <1. The curve is the fitted line, and the short straight line joins the
coordinate of the original values to the values occupied on the fitted curve for each data point.
Data  Curve Fitted 
Test Value  FPR  TPR  FPR  TPR 
162.5  0.96  1  0.95  0.99 
162  0.92  1  0.94  0.99 
161.5  0.84  1  0.91  0.99 
160  0.84  0.96  0.8  0.97 
159.5  0.76  0.96  0.75  0.96 
159  0.68  0.96  0.68  0.94 
158.5  0.48  0.92  0.61  0.93 
157.5  0.4  0.88  0.45  0.87 
157  0.36  0.88  0.37  0.84 
156.5  0.36  0.8  0.3  0.8 
156  0.24  0.68  0.24  0.74 
155.5  0.2  0.6  0.18  0.68 
154  0.12  0.48  0.08  0.47 
153.5  0.04  0.4  0.06  0.4 
153  0.04  0.36  0.04  0.33 
152.5  0  0.28  0.03  0.27 
151.5  0  0.24  0.02  0.17 
151  0  0.2  0.01  0.13 
150.5  0  0.16  0.01  0.1 
150  0  0.12  0.01  0.08 
149  0  0.08  0  0.04 
Please note that the program produces results to 4 decimal point precision,
and these are rounded to 2 for most columns on this page.
Table of Transformed Data is the table on the left. The first 3 columns displays the original
data as they were entered, and the last two column the curve fitted FPR and TPR. This table is for
the user to review, to consider whether the algorithm chosen is appropriate, whether the
results reflect the inputted data, and whether the parameters of the curve fitted ROC can be
generalized for future clinical use.
It can be seen that, above a height of 160cms, all mothers delivered vaginally, and below 153cms all
mothers required a Caesarean Section The data overlapped between these two values were used for curve
fitting (in this example).
It can also be seen that, where the data overlapped, the curve fitted TPR and maternal height are
closed enough to the original for practical use.
Test Value  FPR  TPR  TNR  YI  TPR/TNR  TNR/TPR  LR+  LR 
150.69  0.01  0.11  0.99  0.10  0.11  8.81  11.24  0.90 
151.78  0.02  0.19  0.98  0.17  0.20  5.05  9.71  0.82 
152.43  0.03  0.26  0.97  0.23  0.27  3.72  8.68  0.76 
152.89  0.04  0.32  0.96  0.28  0.33  3.03  7.91  0.71 
153.25  0.05  0.36  0.95  0.31  0.38  2.61  7.29  0.67 
153.55  0.06  0.41  0.94  0.35  0.43  2.31  6.77  0.63 
153.81  0.07  0.44  0.93  0.37  0.48  2.10  6.33  0.60 
154.03  0.08  0.48  0.92  0.40  0.52  1.93  5.95  0.57 
154.23  0.09  0.51  0.91  0.42  0.56  1.80  5.61  0.54 
154.41  0.10  0.53  0.90  0.43  0.59  1.69  5.32  0.52 
154.58  0.11  0.56  0.89  0.45  0.62  1.60  5.05  0.50 
154.73  0.12  0.58  0.88  0.46  0.66  1.52  4.82  0.48 
154.87  0.13  0.60  0.87  0.47  0.69  1.45  4.60  0.46 
155.01  0.14  0.62  0.86  0.48  0.72  1.39  4.41  0.45 
155.13  0.15  0.63  0.85  0.48  0.75  1.34  4.23  0.43 
155.25  0.16  0.65  0.84  0.49  0.77  1.29  4.06  0.42 
155.36  0.17  0.67  0.83  0.50  0.80  1.25  3.91  0.40 
155.47  0.18  0.68  0.82  0.50  0.83  1.21  3.77  0.39 
155.57  0.19  0.69  0.81  0.50  0.85  1.17  3.64  0.38 
155.67  0.20  0.70  0.80  0.50  0.88  1.14  3.52  0.37 
155.77  0.21  0.72  0.79  0.51  0.91  1.10  3.41  0.36 
155.86  0.22  0.73  0.78  0.51  0.93  1.07  3.30  0.35 
155.95  0.23  0.74  0.77  0.51  0.96  1.04  3.21  0.34 
156.04  0.24  0.75  0.76  0.51  0.98  1.02  3.11  0.33 
156.12  0.25  0.76  0.75  0.51  1.01  0.99  3.02  0.33 
156.20  0.26  0.76  0.74  0.50  1.03  0.97  2.94  0.32 
156.28  0.27  0.77  0.73  0.50  1.06  0.94  2.86  0.31 
156.36  0.28  0.78  0.72  0.50  1.09  0.92  2.79  0.30 
156.44  0.29  0.79  0.71  0.50  1.11  0.90  2.72  0.30 
156.51  0.30  0.80  0.70  0.50  1.14  0.88  2.65  0.29 
156.58  0.31  0.80  0.69  0.49  1.16  0.86  2.59  0.29 
156.66  0.32  0.81  0.68  0.49  1.19  0.84  2.53  0.28 
156.73  0.33  0.82  0.67  0.49  1.22  0.82  2.47  0.27 
156.80  0.34  0.82  0.66  0.48  1.25  0.80  2.42  0.27 
156.86  0.35  0.83  0.65  0.48  1.27  0.79  2.37  0.26 
156.93  0.36  0.83  0.64  0.47  1.30  0.77  2.32  0.26 
157.00  0.37  0.84  0.63  0.47  1.33  0.75  2.27  0.26 
157.07  0.38  0.84  0.62  0.46  1.36  0.73  2.22  0.25 
157.13  0.39  0.85  0.61  0.46  1.39  0.72  2.18  0.25 
157.20  0.40  0.85  0.60  0.45  1.42  0.70  2.13  0.24 
157.26  0.41  0.86  0.59  0.45  1.46  0.69  2.09  0.24 
157.32  0.42  0.86  0.58  0.44  1.49  0.67  2.06  0.24 
157.39  0.43  0.87  0.57  0.44  1.52  0.66  2.02  0.23 
157.45  0.44  0.87  0.56  0.43  1.56  0.64  1.98  0.23 
157.51  0.45  0.88  0.55  0.43  1.59  0.63  1.95  0.23 
157.58  0.46  0.88  0.54  0.42  1.63  0.61  1.91  0.22 
157.64  0.47  0.88  0.53  0.41  1.67  0.60  1.88  0.22 
157.70  0.48  0.89  0.52  0.41  1.71  0.59  1.85  0.22 
157.76  0.49  0.89  0.51  0.40  1.75  0.57  1.82  0.21 
157.83  0.50  0.89  0.50  0.39  1.79  0.56  1.79  0.21 
157.89  0.51  0.90  0.49  0.39  1.83  0.55  1.76  0.21 
157.95  0.52  0.90  0.48  0.38  1.88  0.53  1.73  0.21 
158.01  0.53  0.90  0.47  0.37  1.92  0.52  1.71  0.20 
158.08  0.54  0.91  0.46  0.37  1.97  0.51  1.68  0.20 
158.14  0.55  0.91  0.45  0.36  2.02  0.49  1.66  0.20 
158.20  0.56  0.91  0.44  0.35  2.08  0.48  1.63  0.20 
158.26  0.57  0.92  0.43  0.35  2.13  0.47  1.61  0.19 
158.33  0.58  0.92  0.42  0.34  2.19  0.46  1.58  0.19 
158.39  0.59  0.92  0.41  0.33  2.25  0.44  1.56  0.19 
158.46  0.60  0.92  0.40  0.32  2.31  0.43  1.54  0.19 
158.52  0.61  0.93  0.39  0.32  2.38  0.42  1.52  0.19 
158.59  0.62  0.93  0.38  0.31  2.45  0.41  1.50  0.18 
158.65  0.63  0.93  0.37  0.30  2.52  0.40  1.48  0.18 
158.72  0.64  0.93  0.36  0.29  2.60  0.39  1.46  0.18 
158.79  0.65  0.94  0.35  0.29  2.68  0.37  1.44  0.18 
158.86  0.66  0.94  0.34  0.28  2.76  0.36  1.42  0.18 
158.93  0.67  0.94  0.33  0.27  2.85  0.35  1.41  0.18 
159.00  0.68  0.94  0.32  0.26  2.95  0.34  1.39  0.17 
159.07  0.69  0.95  0.31  0.26  3.05  0.33  1.37  0.17 
159.14  0.70  0.95  0.30  0.25  3.16  0.32  1.35  0.17 
159.22  0.71  0.95  0.29  0.24  3.28  0.31  1.34  0.17 
159.29  0.72  0.95  0.28  0.23  3.40  0.29  1.32  0.17 
159.37  0.73  0.95  0.27  0.22  3.54  0.28  1.31  0.17 
159.45  0.74  0.96  0.26  0.22  3.68  0.27  1.29  0.17 
159.53  0.75  0.96  0.25  0.21  3.83  0.26  1.28  0.17 
159.62  0.76  0.96  0.24  0.20  4.00  0.25  1.26  0.16 
159.70  0.77  0.96  0.23  0.19  4.18  0.24  1.25  0.16 
159.79  0.78  0.96  0.22  0.18  4.38  0.23  1.24  0.16 
159.88  0.79  0.97  0.21  0.18  4.60  0.22  1.22  0.16 
159.98  0.80  0.97  0.20  0.17  4.84  0.21  1.21  0.16 
160.08  0.81  0.97  0.19  0.16  5.10  0.20  1.20  0.16 
160.18  0.82  0.97  0.18  0.15  5.40  0.19  1.18  0.16 
160.29  0.83  0.97  0.17  0.14  5.72  0.17  1.17  0.16 
160.40  0.84  0.97  0.16  0.13  6.09  0.16  1.16  0.16 
160.52  0.85  0.98  0.15  0.13  6.51  0.15  1.15  0.16 
160.65  0.86  0.98  0.14  0.12  6.99  0.14  1.14  0.16 
160.78  0.87  0.98  0.13  0.11  7.54  0.13  1.13  0.16 
160.92  0.88  0.98  0.12  0.10  8.18  0.12  1.12  0.16 
161.07  0.89  0.98  0.11  0.09  8.94  0.11  1.10  0.16 
161.24  0.90  0.98  0.10  0.08  9.84  0.10  1.09  0.16 
161.42  0.91  0.99  0.09  0.08  10.96  0.09  1.08  0.16 
161.62  0.92  0.99  0.08  0.07  12.34  0.08  1.07  0.16 
161.84  0.93  0.99  0.07  0.06  14.13  0.07  1.06  0.16 
162.10  0.94  0.99  0.06  0.05  16.51  0.06  1.05  0.16 
162.40  0.95  0.99  0.05  0.04  19.84  0.05  1.04  0.16 
162.76  0.96  0.99  0.04  0.03  24.84  0.04  1.04  0.16 
163.22  0.97  1.00  0.03  0.03  33.17  0.03  1.03  0.16 
163.87  0.98  1.00  0.02  0.02  49.84  0.02  1.02  0.17 
164.96  0.99  1.00  0.01  0.01  99.82  0.01  1.01  0.18 
Table of Curve Fitted Values
The table to the right is intended to be downloaded and used as reference for clinical decision making, should the user
decides to accept the curve fitted results.
 Column 1 is the reversed calculated Test value for the FPR, and in this example it is the maternal height (cms).
 Column 2 is the False Positive Rate (FPR), from 0.01 to 0.99 at 0.01 intervals. This is the independent variable (x)
used to calculate the values in the remaining columns
 Column 3 is the curve fitted True Positive Rate, and Columns 2 and 3 define the curve fitted ROC.
 Column 4 is the True Negative Rate TNR = 1FPR. This is used in some of the subsequent calculations
 Column 5 is the Youden Index YI = TPR+TNR1, which represents the overall accuracy of the values. The Test value associated
with the maximum YI is sometimes used as the cut off value for clinical decision making, and in this example, the closest
is a maternal height of 156.04cms, with FPR=0.24, TPR=0.75, TNR=0.76, and Youden Index = 0.51
 Column 6 is the ratio of TPR/TNR. This is commonly used for two purposes
 An alternative to the Youden Index for selecting the best cut off value is Q*, suggested by Moses et.al (see references).
This is a point where TPR=TNR, and represents the intersection between the ROC and a line joining the top left and
bottom right corner of the plot. In this example, the closest is a maternal height of 156.12cms, with FPR=0.25, FNR=0.76,
TNR=0.75, Youden Index=0.51, and TPR/TNR=1.01
 If the test is being used for screening purposes, to create an early warning system, then a High TPR is preferred over a
high TNR. A TPR/TNR ratio of between 2 and 3 is therefore used for this purposes. In our example, the closest is a maternal
height of 158.65cms, where TPR/TNR=2.52. This can be set as an alert trigger, possibly making a rule that all
mothers admitted to the labour ward shorter than 158.65cms should be reviewed by an experienced obstetrician in early labour.
 Column 7 is a ratio of TNR/TPR, the inverse of column 5. This is commonly used for intervention decisions, especially if the
intervention is costly or risky. A high TNR is preferred over high TPR, and a TNR/TPR ratio of 2 to 3 can be used for
this purpose. In our example, the closet is a maternal height of 153.55cms, where TNR/TPR=2.31 . This can be used for
clinical decision, such as a rule that all mothers admitted to the labour ward shorter than 153.55cms be considered
for elective Caesarean Section in early labour.
 Column 8 and 9 are Likelihood Ratios for Test Positive and Test negative. Discussions on Likelihood Ratio are provided in
the Prediction Statistics Explanation Page
and will not be repeated here.
Hanley JA, McNeil BJ (1982) The meaning and use of the Area Under a Receiver
Operating Characteristic (ROC) curve. Radiology 143:2936
Simel D.L., Samsa G.P., Matchar D.B. (1991) Likelihood ratios with confidence:
sample size estimation for diagnostic test studies. J. Clin. Epidemiology
vol 44 No. 8 pp 763770
Moses LE, Shapiro D, and Littenberg B(1993) Combining independent studies of a
diagnostic test into a Summary ROC Curve: Data analysis approaches and some
additional considerations. Statistics in Medicine 12:p12931316.
