Wiki Sumber Informasi iSIKHNAS

<languages/>
<translate>
= Diagnostic tests = <!--T:1-->
In field epidemiology the term ''diagnosis'' generally means identification of a disease or condition that is affecting an animal. A diagnostic test is any procedure or process that may contribute to the development of a diagnosis. The term diagnostic test may be used to refer to a clinical examination of an individual animal or to a laboratory test performed on a sample collected from an animal (blood, faeces etc). The results of these tests are interpreted and used to determine if the animal has a disease or not.

<!--T:2-->
Diagnostic tests may be applied to an individual animal (as above) or to a group or aggregate of animals such as a mob, herd or farm. The presence of one or more disease-positive animals in a mob may mean that the mob is declared infected (mob-level diagnosis).

== Measures of diagnostic test performance == <!--T:3-->
=== Accuracy and precision ===
'''''Accuracy '''''relates to the ability of the test to provide a result that is close to the truth (the true value). Accuracy is generally assessed in the long run, meaning that it can be thought of as the average of multiple test results. A test is therefore considered to be accurate when the average of repeated tests is close to the true value. Any one test result may itself not be as accurate as the average of repeated tests performed on the same sample. 

<!--T:4-->
'''''Precision''''' refers to how repeatable the test is. If the test is repeated and the result from different runs is always the same then the test is precise (regardless of whether the result is accurate or not). 

<!--T:5-->
{{hlbox|''A '''precise''' test has a low level of random error i.e. a high level of '''repeatability'''.''

<!--T:6-->
''An '''accurate (valid)''' test has a low level of systematic error (bias).''}}

<!--T:7-->
A test can be precise without being accurate and vice versa. A good test is both precise and accurate. The concepts of precision and accuracy are most easily understood by thinking of shooting at a target as shown below. The term validity is often used to represent accuracy.

<!--T:8-->
[[Image:Diagram showing test accuracy (validity) and precision.jpg]]

<!--T:9-->
'''Figure 4.: Diagram showing test accuracy (validity) and precision'''

<!--T:10-->
There are lots of terms used to describe attributes of diagnostic tests.

<!--T:11-->
* ''Repeatability'' generally refers to results from repeated testing of the same sample performed in the same lab
* ''Reproducibility'' refers to results from testing the same sample in different labs (splitting the original sample in to multiple sub-samples and sending them to different labs)
* ''Agreement'' refers to how well two different tests agree

<!--T:12-->
Tests performed on identical material under apparently similar conditions do not, in general, yield identical results. This variation is attributed to random error inherent in every test procedure because factors that may influence the result of a test cannot all be completely controlled. When interpreting test results, this variability must be taken into account. There are many different factors which contribute to the variability of a test procedure, including:

<!--T:13-->
* uniformity of test material
* transport and storage of test material
* reagents
* equipment and its calibration
* operator
* environmental conditions - temperature, humidity, light, air pollution

=== Ability of a test to detect disease states === <!--T:14-->
All tests may produce errors in their results. To qualify as a test, the procedure should classify animals (diseased, not diseased) at least more accurately than a purely random procedure - such as tossing a coin. 

<!--T:15-->
The two types of '''errors''' that a test can make are: 

<!--T:16-->
* '''false positive''' - the test identifies an animal to have a disease when it does not
* '''false negative''' - the test identifies an animal not to have a disease when it does

<!--T:17-->
The '''validity''' of a test is the probability that it will get the classification correct. Validity is expressed in terms of '''sensitivity''' and '''specificity''':

<!--T:18-->
* '''Sensitivity''' (Se) is the probability that a positive animal will be identified as positive by the test (1&nbsp;-&nbsp;false negative rate) - this describes the test ability to detect a disease animal
* '''Specificity''' (Sp) is the probability that a negative animal will be correctly identified as negative by the test (1&nbsp;-&nbsp;false positive rate) - this describes the tests ability to determine an animal is not diseased 

<!--T:19-->
[[Image:Sensitivity and specificity in relation to true disease status and diagnostic test.jpg]]

<!--T:20-->
'''Figure 4.2: Diagram showing interpretation of sensitivity and specificity in relation to true disease status and diagnostic test outcome.'''


<!--T:21-->
Assessing Se and Sp often includes something called a gold standard test. A gold standard is a test that is absolutely accurate - it correctly detects all diseased animals and it correctly classifies all non-diseased animals as being disease free.

<!--T:22-->
Traditionally test performance was based on comparison to a gold standard. More recently methods have been developed that allow assessment of test performance in the absence of a gold standard but these methods are usually more complex.

<!--T:23-->
Diagnostic test performance is often described using a 2x2 table approach.

<!--T:24-->
'''Table 4.3: Table showing 2x2 layout used for assessment of diagnostic test performance. Columns display true disease status based on a gold standard test (+= positive, -=negative) and rows display the results from a diagnostic test.'''


<!--T:25-->
{| style="border-spacing:0;"
| style="border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| style="border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''Gold Std +'''</center>
| style="border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''Gold Std -'''</center>
| style="border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''Total'''</center>

<!--T:26-->
|-
| style="border-top:0.5pt solid #00000a;border-bottom:none;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Test +'''
| style="background-color:#ffffff;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>a</center>
| style="background-color:#ffffff;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>b</center>
| style="border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>a + b</center>

<!--T:27-->
|-
| style="border-top:none;border-bottom:0.5pt solid #00000a;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Test -'''
| style="background-color:#ffffff;border-top:none;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>c</center>
| style="background-color:#ffffff;border-top:none;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>d</center>
| style="border-top:none;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>c + d</center>

<!--T:28-->
|-
| style="border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Total'''
| style="border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>a + c</center>
| style="border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>b + d</center>
| style="border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>a + b + c + d</center>

<!--T:29-->
|}
<math>\mathit{Se}=\frac{a}{\left(a+c\right)}</math><math>\mathit{Sp}=\frac{d}{\left(b+d\right)}</math>


<!--T:30-->
If a new test were applied to 100 animals, made up of 60 healthy animals and 40 infected animals, the results in Table&nbsp;4.2 might be obtained.


<!--T:31-->
{| style="border-spacing:0;"
| style="background-color:#e7e6e6;border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| style="background-color:#e7e6e6;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Infected'''
| style="background-color:#e7e6e6;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Healthy'''
| style="background-color:#e7e6e6;border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Total'''

<!--T:32-->
|-
| style="background-color:#e7e6e6;border-top:0.5pt solid #00000a;border-bottom:none;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Test +'''
| style="background-color:#e7e6e6;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 36
| style="background-color:#e7e6e6;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 10
| style="background-color:#e7e6e6;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 46

<!--T:33-->
|-
| style="background-color:#e7e6e6;border-top:none;border-bottom:0.5pt solid #00000a;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Test -'''
| style="background-color:#e7e6e6;border-top:none;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 4
| style="background-color:#e7e6e6;border-top:none;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 50
| style="background-color:#e7e6e6;border-top:none;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 54

<!--T:34-->
|-
| style="background-color:#e7e6e6;border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Total'''
| style="background-color:#e7e6e6;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 40
| style="background-color:#e7e6e6;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 60
| style="background-color:#e7e6e6;border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 100

<!--T:35-->
|}
* the sensitivity of the test is 36/40&nbsp;=&nbsp; 90%
* the specificity of the test is 50/60&nbsp;=&nbsp;83.3%

=== Sensitivity === <!--T:36-->
''Sensitivity'' is the proportion of animals with the disease (or infection) of interest which test positive (i.e. proportion of true positives). 

<!--T:37-->
'''''Sensitivity''' (True Positive Fraction): The proportion of animals with the disease of interest that test positive. Sensitivity is also defined as the conditional probability that a test will correctly identify those animals that are infected (Pr T+|D+).''

<!--T:38-->
The ''false negative fraction'' is 1 - Se.

=== Specificity === <!--T:39-->
''Specificity'' is the proportion of animals without the disease of interest which test negative (i.e. proportion of true negatives).

<!--T:40-->
'''''Specificity''' (True Negative Fraction): The proportion of animals without the disease of interest that test negative. Specificity is also defined as the conditional probability that a test will correctly identify those animals that are not infected (Pr T-|D-).''

<!--T:41-->
The ''false positive fraction'' is 1 - Sp.

<!--T:42-->
One way to remember the difference between Se and Sp is to think:

<!--T:43-->
* Se = "e" = false negatives and true positives
* Sp= "p" = false positives and true negatives

<!--T:44-->
There is an inverse relationship between Se and Sp for most tests and particularly those that are based on a continuous measure, such as an ELISA. This means that tests that have a very high Se will often have a lower Sp and vice versa.

<!--T:45-->
Tests that produce a continuous measure (measuring antibody or enzyme concentration in blood for example) can have the cut-point altered to move test performance towards higher Se or higher Sp. 

<!--T:46-->
[[Image:Frequency measure of test results.jpg]]

<!--T:47-->
'''Figure 4.3: Plot showing a frequency measure of test results from application of a diagnostic test applied to healthy and diseased animals when the test output is measured on a continuous scale. The vertical line at C-C represents a cut-point to distinguish healthy animals (to the left of C-C) from diseased animals (to the right of C-C).'''

<!--T:48-->
Results for animals that are disease free (D-) usually overlap with the results in the diseased population (D+). Animals to the right of the cut-point (C-C) are classified as reactors (diseased or infected) and animals to the left are classified as negative (non-infected). If fewer false positives are required, C-C is moved to the right; specificity increases and sensitivity decreases. However, if fewer false negatives are required, C-C is moved to the left: sensitivity increases and specificity decreases.

<!--T:49-->
Selection of the appropriate cut-off value will depend on a number of issues including the relative cost of false positives and false negatives, the stage of an eradication program, if any, and the availability of other tests. An important consequence of imperfect specificity (i.e. ~< 100%) is that if a large number of animals are tested from a population free of the disease in question, there is a significant chance of abnormal results. For example, if 10 independent samples were tested using a test with 90% specificity the probability of at least 1 positive test result occurring is 65%.

=== Predictive Values === <!--T:50-->
Se and Sp are characteristics of the test when the test is applied to animals of known disease status. Se and Sp do not tell us how useful the test might be when applied to animals of unknown disease status.

<!--T:51-->
For most people in the field, the practical situation is that they have performed the test on an animal with unknown disease status and they wish to use the test results to better classify the animal as disease + or disease - Se and Sp are not helpful in this situation.

<!--T:52-->
Predictive values are useful in this situation. Predictive values allow us to answer the two related questions:

<!--T:53-->
* What proportion of the test positive animals are truly infected? 
* What proportion of the test negative animals are truly not infected?

<!--T:54-->
Predictive values are functions of ''prevalence'' and the test characteristics of ''sensitivity'' and ''specificity''. As prevalence declines so does the positive predictive value. The converse is true for negative predictive value. 


<!--T:55-->
{| style="border-spacing:0;"
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Positive predictive value = a/(a+b) ='''
| style="background-color:#f2f2f2;border-top:none;border-bottom:0.5pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Prev x Se'''

<!--T:56-->
|-
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| style="background-color:#f2f2f2;border-top:0.5pt solid #00000a;border-bottom:none;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Prev x Se + (1-Prev) x (1-Sp)'''

<!--T:57-->
|-
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 

<!--T:58-->
|-
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Negative predictive value = d/(c+d) = '''
| style="background-color:#f2f2f2;border-top:none;border-bottom:0.5pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''(1-Prev) x Sp'''

<!--T:59-->
|-
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| style="background-color:#f2f2f2;border-top:0.5pt solid #00000a;border-bottom:none;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''(1-Prev) x Sp + Prev x (1-Se)'''

<!--T:60-->
|}
With an understanding of the principles of predictive values, the following rules of thumb for using tests in the diagnostic process at the individual animal level can be recommended:

<!--T:61-->
* If the objective is ''to confirm a likely diagnosis'' (the "rule-in" situation), then choose a test which has ''high specificity'' (~>95%) and at least moderate sensitivity (~>75%). If a positive result is returned, then it is highly likely the individual has the disease in question (PPV's are high for tests with high specificity). If a negative result is returned, then further diagnostic work up is required.
* If the objective is to ''confirm that an individual is free from a particular disease'' (the "rule-out" situation), then choose a test with high sensitivity (~>95%) and at least moderate specificity (~>75%). If a negative result is returned, then it is highly likely the individual is free from the disease in question. If a positive result is returned, then further testing is required with more specific tests to ascertain whether or not the result was a false positive result or not.

== Multiple testing == <!--T:62-->
Two or more tests can be used either sequentially or simultaneously and results interpreted in series or parallel. In parallel interpretation, an animal is considered positive if it reacts positively to either or both tests - this increases sensitivity but tends to decrease the specificity of the combined tests. In series interpretation, an animal must be positive on both tests to be considered positive - this increases specificity at the expense of sensitivity.

<!--T:63-->
{{hlbox|'''''Parallel''' interpretation of tests means that both tests must give a positive result for the animal to be considered positive. Parallel testing increases sensitivity but tends to decrease the specificity, compared to using either test in isolation.''

<!--T:257-->
'''''Series''' interpretation of tests means that the animal is considered positive if either of the tests gives a positive result. Series testing increases specificity at the expense of sensitivity, compared to using either test in isolation.''}}

<!--T:65-->
In general, the greater the number of tests involved, the greater the increase in sensitivity or specificity, depending on the method of interpretation that is used. 

=== Sensitivity and specificity for multiple tests === <!--T:66-->
Overall values for sensitivity for interpretation of tests in series or parallel, assuming conditional independence of the tests, can be calculated using the following example.

<!--T:67-->
For this example the two tests are assumed to be independent and have the following characteristics: 

<!--T:258-->
Test 1 - Se = 50%; Sp = 98.7%     Test 2 - Se = 60%; Sp = 98.6%

<!--T:259-->
What are the theoretical sensitivities and specificities of the two tests used in parallel or series?

<!--T:70-->
For sensitivity, we assume an animal is infected and that it is tested with both Test 1 and Test 2. For Test 1, the probability of a positive test result (given that the animal is infected) is Se<sub>1</sub> = 0.5 and the corresponding probability that it will give a negative result is 1 - Se<sub>1</sub>, also = 0.5 for this example. For Test 2, the probability of a positive test result (given that the animal is infected) is Se<sub>2</sub> = 0.6 and the corresponding probability that it will give a negative result is 1 - Se<sub>2</sub> = 0.4.

<!--T:71-->
For series interpretation, both tests must be positive for it to be considered a positive result. From the scenario tree this is the result for the first limb on the left, which has probability P(+/+) = Se<sub>1</sub> ï‚´ Se<sub>2</sub> = 0.5 ï‚´ 0.6 = 0.3. Thus, the formula for sensitivity for series interpretation is Se<sub>series</sub> = Se<sub>1</sub> ï‚´ Se<sub>2</sub> and for this example is 0.3 or 30%. 

<!--T:72-->
For parallel interpretation, the result is considered positive if either of the individual test results is positive. Alternatively, for a result to be considered negative both test results must be negative. Again this can be determined from the scenario tree, where the limb on the right represents both tests having a negative result and the probability of both negative results is P(-/-) = (1 - Se<sub>1</sub>) ï‚´ (1 - Se<sub>2</sub>). Therefore the probability of an overall positive result for parallel interpretation is Se<sub>parallel</sub> = 1 - (1 - Se<sub>1</sub>) ï‚´ (1 - Se<sub>2</sub>) = 0.8 (80%) for this example.

<!--T:73-->
Similar logic can be applied to the example of an uninfected animal to derive formulae for specificity for series and parallel interpretation as shown below:

<!--T:74-->
Sp<sub>parallel</sub> = Sp<sub>1</sub> ï‚´ Sp<sub>2</sub> = 0.973 or 97.3% for this example and 

<!--T:75-->
Sp<sub>series</sub> = 1 - (1 - Sp<sub>1</sub>) ï‚´ (1 - Sp<sub>2</sub>) = 0.999 or 99.9% for our example

<!--T:260-->
[[Image:Scenario tree 1.jpg]]


<!--T:76-->
'''Figure 4.4: Scenario tree for calculating overall sensitivity for two tests interpreted in series or parallel'''

=== Conditional independence of tests === <!--T:77-->
An important assumption of series and parallel interpretation of tests is that the tests being considered are conditionally independent. Conditional independence means that test sensitivity (specificity) remains the same regardless of the result of the comparison test, depending on the infection status of the individual. 

<!--T:78-->
If the assumption of conditional independence is violated then combined sensitivity (or specificity) will be biased. The "conditional" term relates to the fact that the independence (or lack of independence) is conditional on the disease status of the animal. Therefore sensitivities may be conditionally independent (or not) in diseased animals, while specificities may be conditionally independent (or not) in non-diseased animals. 

<!--T:79-->
{{hlbox|''Two tests are '''conditionally independent''' if test sensitivity or specificity (depending on disease status) of one test remains the same regardless of the result of the other (comparison) test''}}

<!--T:80-->
If tests are not independent (are correlated), the overall sensitivity or specificity improvements may not be as good as the theoretical estimates, because two tests will tend to give similar results on samples from the same animal.

<!--T:81-->
For example, let us assume that the two tests described above were applied to 200 infected and 7,800 uninfected animals with the following results. What are the actual sensitivities and specificities for parallel and series interpretations and how do they compare to the theoretical values?


<!--T:82-->
{| style="border-spacing:0;"
| style="background-color:#d9d9d9;border:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>'''Test 1'''</center>
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>'''Test 2'''</center>
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>'''Infected'''</center>
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>'''Uninfected'''</center>

<!--T:261-->
|-
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:0.75pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>+</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>-</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 30
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 70

<!--T:262-->
|-
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:0.75pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>-</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>+</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 50
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 80

<!--T:263-->
|-
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:0.75pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>+</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>+</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 70
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 30

<!--T:264-->
|-
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:0.75pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>-</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>-</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 50
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 7620

<!--T:265-->
|-
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:none;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:none;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| <center>Total</center>
| style="background-color:#d9d9d9;border:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 200
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.203cm;padding-right:0.191cm;"| 7800

<!--T:266-->
|}

<!--T:267-->
Observed sensitivities and specificities of the two tests used in parallel or series are:

<!--T:268-->
{|
|-
| Se<sub>series</sub> = 70/200 = 35%||  Se<sub>parallel</sub> = 150/200 = 75%
|-
| Sp<sub>series</sub> = 7770/7800 = 99.6%||  Sp<sub>parallel</sub> = 7620/7800 = 97.7%
|}
          

<!--T:91-->
Sensitivity in series has dropped less than predicted (35% instead of 30% predicted), and sensitivity of parallel testing has increased less than predicted (75% compared to 80% predicted). The apparent difference between calculated and observed values for combined sensitivities suggests that these tests are in fact correlated. 

<!--T:92-->
This difference is due to correlation of the test sensitivities, so that infected animals that are positive to Test 1 are also more likely to be positive in Test 2, as shown by the substantial difference in sensitivity of Test 2 in animals positive to Test 1 (70/100 or 70%) compared to those negative to Test 1 (30/100 or 30%).

<!--T:93-->
The differences in observed and predicted specificities are much smaller and in this case probably due to random variation.


<!--T:94-->
Lack of conditional independence of tests is particularly likely if two tests are measuring the same (or similar outcome).

<!--T:95-->
For example: ELISA and AGID are two serological tests for Johne's disease in sheep. Both tests measure antibody levels in serum. Therefore, in an infected animal, the ELISA is more likely to be positive in AGID-positive animals than in AGID-negative animals, so that the sensitivities of the two tests are correlated (not independent). This is illustrated in Table 4.5, where the sensitivities of both tests vary markedly, depending on the result of the other test. In contrast, serological tests such as ELISA and AGID are likely to be less correlated with agent-detection tests, such as faecal culture.


<!--T:269-->
{| style="border-spacing:0;"
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| colspan="2"  style="background-color:#d9d9d9;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>ELISA</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 

<!--T:270-->
|-
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:none;border-left:0.75pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| AGID
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.75pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>+</center>
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.75pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>-</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:0.75pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>Total</center>

<!--T:271-->
|-
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.75pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| +
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''34'''</center>
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''21'''</center>
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>55</center>

<!--T:272-->
|-
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.75pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| -
| style="background-color:#d9d9d9;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''13'''</center>
| style="background-color:#d9d9d9;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''156'''</center>
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>169</center>

<!--T:273-->
|-
| style="background-color:#d9d9d9;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Total 
| style="background-color:#d9d9d9;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>47</center>
| style="background-color:#d9d9d9;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>177</center>
| style="background-color:#d9d9d9;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>224</center>

<!--T:274-->
|}
All 224 sheep are infected, so we can calculate sensitivities of both ELISA and AGID as follows:

<!--T:102-->
{| style="border-spacing:0;"
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>ELISA Se overall</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>47/224 = 21.0%</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>AGID Se overall</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>55/224 = 24.6%</center>

<!--T:275-->
|-
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>ELISA Se in AGID +</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>34/55 = 61.8%</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>AGID Se in ELISA +</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>34/47 = 72.3%</center>

<!--T:276-->
|-
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>ELISA Se in AGID -</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>13/169 = 7.7%</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>AGID Se in ELISA -</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>21/177 = 11.9%</center>

<!--T:277-->
|}

=== Application of series and parallel testing === <!--T:105-->
Series testing is commonly used to improve the specificity, and hence the positive predictive value, of a testing regimen (at the expense of reduced sensitivity). 

<!--T:106-->
For example, in large-scale screening programs, such as for disease control or eradication, a relatively cheap, high-throughput test with relatively high sensitivity and precision but only only modest specificity may be used for initial screening. This sort of test can be applied to large numbers of animals (entire population) where the purpose is to be very confident that those animals that test negative are in fact disease free.

<!--T:107-->
Any positives to the initial screening test are then tested using a highly specific (and usually more expensive) confirmatory test to minimise the overall number of false positives at the end of the testing process. For an animal to be considered positive it must be positive to both the initial screening test and the confirmatory follow-up test. 

<!--T:108-->
A good example of series testing is in eradication programs for bovine tuberculosis, where the initial screening test is often either a caudal fold or comparative cervical intradermal tuberculin test, which is followed up in any positives by a range of possible tests including additional skin tests, a gamma interferon immunological test or even euthanasia and lymph node culture, depending on circumstances. 

<!--T:109-->
In the above situation it is important to realise that even though the follow-up test is only applied to those that are positive on the first test, this is still an example of series interpretation. Because an animal must test positive to both tests for a positive overall result, the result of the second test in animals negative to the first test is irrelevant, so that the test doesnâ€™t actually need to be done. This is an important consideration in control or eradication programs, where testing costs are usually a major budget constraint and significant savings can be made by using a cheap, high-throughput screening test followed by a more expensive but highly specific follow-up test.

<!--T:110-->
Parallel testing is less commonly used, but is primarily directed at improving overall sensitivity and hence negative predictive value of the testing regimen. Parallel testing is mainly applied where minimising false negatives is imperative, for example in public health programs or for zoonoses, where the consequences of failing to detect a case can be extremely serious. In contrast to series testing, every sample must be tested with both tests for parallel testing to be effective, so that testing costs can be quite high.

<!--T:111-->
For example, in some countries testing for highly pathogenic avian influenza virus may rely on using a combination of virus isolation and PCR for detection of virus, with birds that are positive to either test being considered infected.

== Measuring agreement between tests == <!--T:112-->
There is often interest in comparing the diagnostic performance of two tests (new test compared to an existing test) to see if the new test produces similar results.

<!--T:113-->
For the same specimens submitted to each of the two tests, the investigator records the appropriate frequency data into the 4 cells of a 2x2 table, a (both tests positive), b (test 1 positive and test 2 negative), c (test 1 negative and test 2 positive), and d (both tests negative). The value ''kappa'' (''k''), a measure of relative agreement beyond chance, can then be calculated using software such as EpiTools or using formulae in standard epidemiology texts.

<!--T:114-->
''Kappa'' has many similarities to a correlation coefficient and is interpreted along similar lines. It can have values between -1 and +1. Suggested criteria for evaluating agreement are (Everitt, 1989, cited by Thrusfield, 1995):

<!--T:115-->
'''Table 4.2: Table showing interpretation of kappa values'''


<!--T:116-->
{| style="border-spacing:0;"
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''''kappa'' '''
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Evaluation'''

<!--T:117-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"|  >0.8 - 1
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Excellent agreement

<!--T:118-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| >0.6 - 0.8
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Substantial agreement

<!--T:119-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| >0.4 - 0.6
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Moderate agreement

<!--T:120-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| >0.2 - 0.4
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Fair agreement

<!--T:121-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| >0 - 0.2
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Slight agreement

<!--T:122-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 0
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Poor agreement

<!--T:123-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <0
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Disagreement
|}

<!--T:124-->
Care must be taken in interpreting ''kappa'' - if two tests agree well, they could be equally good or equally bad! However, it may be possible to justify use of a newly developed test if it agrees well with a standard test and if it is cheaper to run in the laboratory. 

<!--T:125-->
Conversely, if two tests disagree, one test is likely to be better than the other although there may no way to tell which is better! The exception to this is where both tests have close to 100% specificity (i.e. no or few false positives). In this case the test with the larger number of positive results is likely to be more sensitive. McNemar's Chi-squared test for paired data can also be used to test for significant differences between the discordant cells (b & c).

<!--T:126-->
'''An example of kappa and agreement between tests'''

<!--T:127-->
A comparison of two herd-tests for Johne's disease in sheep yields the following results (from Sergeant et al., ([#18 2002])):

<!--T:278-->
{| style="border-spacing:0;"
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:none;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| colspan="2"  style="background-color:#d9d9d9;border:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''Test 2 results'''</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:0.5pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 

<!--T:279-->
|-
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:none;border-left:0.75pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Test 1 results'''
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.75pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>+</center>
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:none;border-left:0.75pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>-</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:none;border-left:0.75pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Total

<!--T:280-->
|-
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.75pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>+</center>
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''58'''</center>
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''37'''</center>
| style="background-color:#d9d9d9;border-top:0.75pt solid #00000a;border-bottom:0.5pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 95

<!--T:281-->
|-
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:0.75pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>-</center>
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''5'''</center>
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.5pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>'''196'''</center>
| style="background-color:#d9d9d9;border-top:0.5pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:0.5pt solid #00000a;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 201

<!--T:282-->
|-
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Total 
| style="background-color:#d9d9d9;border-top:none;border-bottom:0.75pt solid #00000a;border-left:0.75pt solid #00000a;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>63</center>
| style="background-color:#d9d9d9;border-top:none;border-bottom:0.75pt solid #00000a;border-left:none;border-right:0.75pt solid #00000a;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>233</center>
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 296

<!--T:133-->
|}
How well do the two tests agree, and can you determine which test is better?

<!--T:134-->
For these tests, kappa is 0.64, suggesting moderate-substantial agreement. However, McNemarâ€™s chi-squared is 22.88, with 1 degree of freedom and P < 0.001. This means that the discordant cells (37 and 5) are significantly different. From the data available it is not possible to say which test is better - the additional positives on Test 1 could be either true or false positives, depending on test specificity.

<!--T:135-->
In this case, Test 1 was pooled faecal culture (specificity assumed to be 100%) and Test 2 was the agar gel-diffusion test with follow-up of positives by autopsy and histopathology (specificity also assumed to be 100%). How does this change the assessment of the two tests?

<!--T:136-->
Considering that both tests have specificity equal (or very close) to 100%, there are likely to be very few false-positives. Therefore it appears that the sensitivity of Test 1 (pooled faecal culture) is considerably higher than that for Test 2 (serology), since Test 1 detected a greater number of positives overall.

=== Proportional agreement of positive and negative results ===  <!--T:137-->
In some circumstances, particularly where the marginal totals of the 2-by-2 table are not balanced, ''kappa'' is not always a good measure of the true level of agreement between two tests ([#6 Feinstein and Cicchetti, 1990]). For example, in the first example above, kappa was only 0.74, compared to an overall proportion of agreement of 0.94 In these situations, the proportions of positive and negative agreement have been proposed as useful alternatives to ''kappa'' ([#3 Cicchetti and Feinstein, 1990]). For this example, the proportion of positive agreement was 0.78, compared to 0.96 for the proportion of negative agreement, suggesting that the main area of disagreement between the tests is in positive results and that agreement among negatives is very high.

== Estimation of true prevalence from apparent prevalence == <!--T:138-->
When we apply a test in a population, the proportion of positive results observed is the apparent prevalence. However, depending on test performance, apparent prevalence may not be a good indicator of the true level of disease in the population (the true prevalence). However, if we can estimate the sensitivity and specificity of the test, we can also estimate the true prevalence from the apparent (test-positive) prevalence (AP) using the formula ([#16 Rogan and Gladen, 1978]):


<!--T:139-->
{| style="border-spacing:0;"
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''True prevalence''' =
| style="background-color:#d9d9d9;border-top:none;border-bottom:0.75pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| AP + Sp - 1

<!--T:140-->
|-
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| style="background-color:#d9d9d9;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| Se + Sp - 1

<!--T:141-->
|}
which has a solution for situations other than when Se + Sp = 1. All values are expressed as proportions (between 0 and 1) rather than percentages for these calculations. Confidence limits can be calculated for the estimate using a variety of methods implemented in EpiTools. When true prevalence is 0, apparent prevalence = 1 - Sp, the false positive test rate. 

<!--T:142-->
For example: Say we have conducted a survey with a test whose sensitivity is 90% (0.9) and specificity is 95% (0.95) and we find a reactor rate (apparent prevalence) of 15% (0.15). By using the formula, we can estimate the true prevalence to be 11.8% (0.118).

<!--T:143-->
'''Another example'''

<!--T:144-->
Suppose we have conducted a survey of white spot disease in a shrimp farm, using a test with sensitivity of 80% (0.8) and specificity of 100% (1.0). We have tested 150 shrimp, and 6 shrimp tested positive. What is the estimated true prevalence?

<!--T:145-->
The apparent prevalence is 6/150 = 0.04 or 4% (Wilson 95% CI: 1.8% - 8.5%)

<!--T:146-->
Therefore, true prevalence = (0.04 + 1 - 1)/(0.8 + 1 - 1) = 0.04/0.8 = 0.05 or 5% (95% CI: 1.1 - 8.9%)

<!--T:147-->
What happens if we assume that sensitivity and specificity are both 90%?

<!--T:148-->
If Se = 0.9 and Sp = 0.9:

<!--T:149-->
Therefore, true prevalence = (0.04 + 0.9 - 1)/(0.9 + 0.9 - 1) <nowiki>= -0.06/0.8 = -0.0625.</nowiki>

<!--T:150-->
<nowiki>The above example illustrates one potential problem with Rogan and Gladen formula, which is that in some circumstances negative estimates can be produced. However, a negative (<0) prevalence is clearly impossible, so for this scenario the assumptions about sensitivity and specificity must be incorrect. For example, if specificity was 90% (0.9), and you tested 150 </nowiki>animals, you would expect to have 0.1*150 or on average about 15 false positive results (even in an uninfected population). Therefore if only 4 positives were recorded, the specificity of the test must be much higher than 90% (a minimum estimate would be to assume all of the positives are false positives, so that specificity = 1 - apparent prevalence = 1 - 4% or 96%).

<!--T:151-->
Because prevalence estimates are proportions we should also calculate and present confidence intervals for the estimate. 


== Group (aggregate) diagnostic tests == <!--T:152-->
The previous discussion describes the testing of individual animals. However, in epidemiological investigations, the study unit can often comprise a group of animals such as a herd of cattle, a flock of sheep, or a cage or pond of fish. For example, it is common practice to determine herd or flock status for some diseases based on the results of testing of a sample of animals, rather than testing the whole herd or flock.

<!--T:153-->
In this situation, it is important to realise that testing for disease at the group or aggregate level incorporates a number of factors additional to those relevant to testing at the individual animal level. Thus, tests which may be highly sensitive and specific at the individual animal level can still result in misclassification of a high proportion of groups where only a small number of animals in each group are tested.

<!--T:154-->
At the individual animal level, diagnostic test performance is determined by its sensitivity and specificity. The corresponding group-level measures are ''herd sensitivity'' and ''herd specificity''. Herd sensitivity and herd specificity are affected by animal-level sensitivity and specificity, as well as the number of animals tested, the prevalence of disease in the group and the number of individual animal positive results (1, 2, 3 etc) used to classify the group as positive. Just as we do for individuals, we also want high sensitivity and high specificity in our group level interpretation.

<!--T:155-->
{{hlbox|'''''Herd sensitivity''' (SeH) is the probability that an infected herd will give a positive result to a particular testing protocol, given that it is infected at a prevalence equal to or greater than the specified design prevalence.''

<!--T:283-->
'''''Herd specificity''' (SpH) is the probability that an uninfected herd will give a negative result to a particular testing protocol (HSP)''}}


=== Calculating herd sensitivity and herd specificity === <!--T:157-->
The herd-level sensitivity (''SeH'') and specificity (''SpH'') with a cut-off of 1 reactor to declare a herd infected can be calculated as ([#13 Martin et al., 1992]):

<!--T:158-->
'''SeH''' = 1 - (1 - (Prev x Se + (1 - Prev) x (1-Sp)))<sup>m</sup> and

<!--T:159-->
'''SpH''' = Sp<sup>m</sup>

<!--T:160-->
Where '''''Se''''' and '''''Sp''''' are animal-level sensitivity and specificity respectively, '''''Prev''''' is true disease prevalence and '''''m''''' is the number of animals tested. '''''SeH''''' is equivalent to the level of confidence of detecting infection in herds or flocks with the specified prevalence of infection. '''''SeH''''' and '''''SpH''''' can be easily calculated using EpiTools or other epidemiological calculators.

<!--T:161-->
If test specificity is 100% (i.e. any reactors are followed up to confirm their status) calculation of SeH is simplified:

<!--T:162-->
'''SeH ''' = 1 - (1 - Prev</nowiki> x Se)<sup>m</sup> 

<!--T:163-->
An example

<!--T:164-->
For example, assuming that we have tested 100 animals in a herd with a test that has Se = 0.9 and Sp = 0.99, what is the herd-sensitivity for an assumed prevalence of 5%? 

<!--T:165-->
SeH = 1 - (1 - (0.05*0.9 + (1 - 0.05)*(1 - 0.99)))<sup>100</sup>

<!--T:166-->
<nowiki>= 0.996 or 99.6%</nowiki>

<!--T:167-->
This means that if disease is present at a prevalence of 5% or more, there is a 99.6% chance that one or more animals in the sample will test positively.

<!--T:168-->
For this scenario, herd-specificity is:

<!--T:169-->
SpH = 0.99<sup>100</sup> = 0.37 or 37%

<!--T:170-->
This means that there is a 37% chance that an uninfected herd will also have one or more animals test positively.

<!--T:171-->
What happens if we assume that the prevalence of infection is 2% instead of 5%?

<!--T:172-->
'''Herd-sensitivity:'''

<!--T:173-->
SeH = 1 - (1 - (0.02*0.9 + (1 - 0.02)*(1 - 0.99)))<sup>100</sup>

<!--T:174-->
<nowiki>= 0.94 or 94%</nowiki>

<!--T:175-->
SeH decreases as prevalence decreases.

<!--T:176-->
'''Herd-specificity:'''

<!--T:177-->
SpH = 0.99<sup>100</sup>

<!--T:178-->
<nowiki>= 0.37 or 37%</nowiki>

<!--T:179-->
SpH is unaffected by prevalence because, by definition, SpH applies only to herds with zero prevalence (uninfected). 

<!--T:180-->
In the above example, increasing the cut-point number of reactors for a positive result from 1 to 2 (i.e. if there are 0 or 1 animals test positive the group is considered "uninfected" while if 2 or more test positive it is infected) results in an increase in SpH to 74% but a reduction in SeH to 77% (from EpiTools: [http://epitools.ausvet.com.au/content.php?page=HerdSens3 http://epitools.ausvet.com.au/content.php?page=HerdSens3]).

<!--T:181-->
The formulae above assume that sample size is small relative to population size (or that the population is large). Similar formulae are also available for small populations or where the sample size is large relative to population size.


=== Risk of infection in test-negative animals === <!--T:182-->
The only way to be 100% confident that no animals comprising a particular group are infected with a particular agent is to test every animal in the group with a diagnostic test which has perfect sensitivity and specificity. However, if only a low proportion of individual animals in the group are infected and only a small number are tested there can be quite a high chance that infected groups will be misclassified as uninfected. The following table shows the number of infected animals which may be present but undetected in a population of 100,000, despite a sample testing negative using a test with perfect sensitivity and specificity at the individual animal level.

<!--T:183-->
'''Table 4.3: Number of diseased or infected animals which could remain in a group of 100,000 after a small number are tested and found to be negative using a test which has perfect sensitivity and specificity at the individual animal level for 95% and 99% confidence levels'''


<!--T:184-->
{| style="border-spacing:0;"
| style="background-color:#ffffff;border-top:0.75pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''No. of animals in sample tested from group of 100,000 and found negative'''
| style="border-top:0.75pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>95%</center>
| style="border-top:0.75pt solid #00000a;border-bottom:0.75pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>99%</center>

<!--T:284-->
|-
| style="border-top:0.75pt solid #00000a;border-bottom:none;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>100</center>
| style="border-top:0.75pt solid #00000a;border-bottom:none;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>2,950</center>
| style="border-top:0.75pt solid #00000a;border-bottom:none;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>4,499</center>

<!--T:285-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>500</center>
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>596</center>
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>915</center>

<!--T:286-->
|-
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>1,000</center>
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>298</center>
| style="border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>458</center>

<!--T:287-->
|-
| style="border-top:none;border-bottom:0.75pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>10,000</center>
| style="border-top:none;border-bottom:0.75pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>29</center>
| style="border-top:none;border-bottom:0.75pt solid #00000a;border-left:none;border-right:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| <center>44</center>

<!--T:288-->
|}

<!--T:189-->
The situation is further complicated where the test procedure being used has poor sensitivity, which is the case for many tests in regular use.

<!--T:190-->
The probability of introducing infection in a group of tested-negative animals is the same as the probability that one or more animals in the group are infected but tests negative. This probability can be calculated as:


<!--T:289-->
{| style="border-spacing:0;"
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| '''Probability '''
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| = 1 - NPV<sup>m</sup> 

<!--T:290-->
|-
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| 
| style="background-color:#f2f2f2;border:none;padding-top:0cm;padding-bottom:0cm;padding-left:0.191cm;padding-right:0.191cm;"| = 1 - [(1-Prev)  x  Sp/((1-Prev)  x  Sp + Prev  x  (1-Se))]<sup>m</sup>

<!--T:291-->
|}

<!--T:193-->
Where '''''NPV'' '''is the negative predictive value of the test in the population of origin, '''''Se''''' and '''''Sp '''''are animal-level sensitivity and specificity respectively, '''''Prev''''' is true disease prevalence and '''''m '''''is the number of animals tested. As sample size increases the probability that the group will all test negative decreases, so that the overall risk associated with a group can be reduced by increasing the sample size. However, if all animals do test negatively the probability that one or more are actually infected increases (assuming that they are from an infected population), as shown in Figure 4.6.

<!--T:194-->
For example: If 20 animals are selected from a herd or flock with a true prevalence of 0.05 (5%) and are tested using a test with Se=0.9 and Sp=0.99, and all 20 have a negative result, the probability that there are one or more infected animals in the group is about 0.1 (10%). In addition, the probability that all 20 animals will have a negative test result is about 0.33 (33%). 

<!--T:195-->
In simple language, there is a 1 in 3 chance that all animals test negative and also a 1 in 10 chance that there is one or more infected animals in the group, even if they have all tested negative.

<!--T:196-->
Increasing sample size from 20 to 40 reduces the probability that all will test negatively from 33% to about 10%, but for those that are all negative, increases the probability that one or more are infected from 10% to 20% (1 in 5).


<!--T:197-->
[[Image:Effect of sample size on the probability.jpg]] 

<!--T:198-->
'''Figure 4.: Effect of sample size on the probability that a group of test-negative animals will include one or more infected (but test-negative) animals, and the probability that this will occur, for an assumed Se=0.9, Sp-0.99 and true prevalence=0.05 (5%) in the herd/flock of origin.'''


=== Demonstrate freedom of detecting disease? === <!--T:199-->
It is impossible to prove that a population is free from a particular disease without testing every individual with a perfect test. However, demonstrating "freedom" from disease in a population is essentially the same as sampling to provide a high level of confidence of detecting disease at specified (design) prevalence. If we don't detect disease, then we can state that we have the appropriate level of confidence that (if the disease is present) it is at prevalence lower than the ''design prevalence''. Provided we have selected appropriate design prevalence, it can then be argued that if the disease were present it would more than likely be at a higher level than the design prevalence, and therefore we can be confident that the population is probably free of the disease.

<!--T:200-->
The selection of appropriate design prevalence is obviously critical if it is too low sample sizes will be excessive, while if it is too high the argument that it is an appropriate threshold for detection of disease is weaker. For infectious diseases it is common to use a value equal to or lower than values observes in endemic or outbreak situations.

=== Important factors to consider in group testing === <!--T:201-->
When testing a group of animals for the presence of disease, there are a number of important points to keep in mind:

<!--T:202-->
* Individual and group level test characteristics (sensitivity and specificity) are not equivalent.
* The number of animals to be tested in the group (sample size) is relatively independent of group size except for small groups (<~1000) or where sample size is more than about 10% of the group size. Alternative methods are available for small populations or where sample size is large relative to group size.
* The number of animals required to be tested in the group depends much more on individual animal specificity than it does on sensitivity.
* The number of animals to be tested in the group is linearly and inversely related to the expected prevalence of infected animals in the group.
* As the required level of statistical confidence increases, so the required sample size increases. The usual level is 95%. If this is increased to 99%, there is an approximate increase of 50% in the required sample size. For a reduction from 95% to 90% confidence, there is a decrease in sample size by 25%.
* As the sample size increases, group level sensitivity increases.
* As the number of animals used to classify the group as positive is increased, there is a corresponding increase in specificity.
* As group level sensitivity increases, group level specificity decreases.
* When specificity = 100% at the individual animal level, all uninfected groups are correctly classified i.e. group level specificity also equals 100%.

== Estimating test sensitivity and specificity == <!--T:203-->
There are two broad approaches to estimating test sensitivities and specificities. 

<!--T:204-->
"Gold standard" methods rely on the classification of individuals using a reference test (or tests) with perfect sensitivity and/or specificity to identify groups of diseased and non-diseased individuals in which the test can be evaluated. In contrast, "Non-gold-standard" methods are used in situations where determination of the true infection status of each individual is not possible or economically feasible.

<!--T:205-->
Regardless of the methods used for estimating sensitivity and specificity, a number of important principles must be considered when evaluating tests, as for any other epidemiological study ([#8 Greiner and Gardner, 2000]):

<!--T:206-->
* The study population from which the sample is drawn should be representative of the population in which the test is to be applied;
* The sample of individuals to which the test is applied must be selected in a manner to ensure that it is representative of the study population;
* The sample should include animals in all stages of the infection/disease process;
* The sample size must be sufficient to provide adequate precision (confidence limits) about the estimate; and
* Testing should be undertaken with blinding as to the true status of the individual and to other test results.

=== Gold-standard methods === <!--T:207-->
Gold standard methods have the advantage of using a known disease status as the reference test. This allows for relatively simple calculations to estimate sensitivity and specificity of the test being evaluated, using a simple two-by-two cross-tabulation of the test against disease status. However, for many conditions a gold-standard test either does not exist or is prohibitively expensive to use (for example may require slaughter and detailed examination and testing of multiple tissues for a definitive result). In such cases the best available test is often used as if it were a gold standard, resulting in biased estimates of sensitivity and specificity. Alternatively, it may only be possible to use a small sample size due to financial limitations or the nature of the disease, resulting in imprecise estimates.

<!--T:208-->
{{hlbox|'''''Gold standard '''test evaluation assumes comparison with the true disease status of an animal based on the results of a test (or tests) with perfect sensitivity and/or specificity''}}

<!--T:209-->
For example: The "gold-standard" test for bovine spongiform encephalopathy (BSE) is the demonstration of typical histological lesions in the brain of affected animals. However, false-negative results on histology will occur in animals in an early stage of infection. Therefore, if a screening test is evaluated by comparison with histology, specificity will be underestimated because some infected animals could react to the screening test but be histologically negative, resulting in mis-classification as false-positives. In addition, any infected but histologically-negative animals that are negative on the screening test will be mis-classified as true-negatives, resulting in over-estimation of the sensitivity.

<!--T:210-->
If a disease is rare, or if the "gold standard" test is complex and expensive to perform, sample sizes for estimation of sensitivity are likely to be small, leading to imprecise estimates of sensitivity. If a disease does not occur in a country it is impossible to estimate sensitivity in a sample that is representative of the population in which it is to be applied. Conversely, if a disease does not occur in a country or region, it is relatively easy to estimate test specificity, based on a representative sample of animals from the population, because if the population is free of disease all animals in the population must also be disease-free.

<!--T:211-->
Sometimes a new test may appear to be more sensitive (or specific) than the existing "gold standard" test (for example, new DNA-based tests compared to conventional culture). In this situation, the new test will find more (or fewer) positives than the reference test and careful analysis is required to determine whether this is because it is more sensitive or less specific. Even then, it is often not possible to reliably estimate sensitivity or specificity because there is no fixed reference point, so it may only be possible to say that the new test is more sensitive (or specific) than the old test, without specifying a value.

<!--T:212-->
Gold-standard methods for estimating sensitivity and specificity of diagnostic tests and their limitations are discussed in more detail by Greiner and Gardner ([#8 2000]).


==== Estimating specificity in uninfected populations ==== <!--T:213-->
One special case of a gold standard comparison is for estimating test specificity in an uninfected population. In this case either historical information or other testing can be used to determine that a defined population is free of the disease of concern. This can be based on either a geographic region which is known to be free, or on intensive testing of a herd or herds over a period of time to provide a high level of confidence of freedom. If the population is assumed to be free, by definition all animals in the population are uninfected. Therefore, if a sample of animals from the population is tested with the new test, any positives are assumed to be false positives and the test specificity is estimated as the proportion of samples that test negatively.

<!--T:214-->
For example, to evaluate the specificity of a new test for foot-and-mouth disease you could collect samples from an appropriate number of animals in a FMD-free country and use these as your reference panel. 

<!--T:215-->
Two drawbacks of this approach are: firstly that you cannot estimate sensitivity in this sample, since none of the animals are infected; and secondly that by using a defined (often geographically isolated) population there is a risk that specificity may be different in this population to what might be the case in the target population where the test is to be used.

=== Non-gold-standard methods === <!--T:216-->
Non-gold-standard methods for test evaluation can often be used in situations where the traditional gold-standard approaches are not possible or feasible. These methods do not depend on determining the true infection status of each individual. Instead, they use statistical approaches to calculate the values of sensitivity and specificity that best fit the available data. 

<!--T:217-->
{{hlbox|'''''Non gold-standard '''test evaluation makes no explicit assumptions about the disease state of the animals tested and relies on statistical methods to determine the most likely values for test sensitivity and/or specificity''}}

<!--T:218-->
Although these methods don't rely on a gold standard for comparison, they do depend on a number of important assumptions. Violation of these assumptions could render the resulting estimates invalid. Non-gold-standard methods for estimating sensitivity and specificity of diagnostic tests have been described in more detail by Hui and Walter ([#9 1980]), Staquet et al. ([#19 1981]) and Enøe et al. ([#4 2000]).

<!--T:219-->
Available non-gold-standard methods include:

==== Maximum likelihood estimation  ==== <!--T:220-->
Maximum likelihood methods use standard statistical methods to estimate sensitivity and specificity of multiple tests from a comparison of the results of multiple tests applied to the same individuals in multiple populations with different prevalence levels ([#9 Hui and Walter, 1980]; [#4 EnÃ¸e et al., 2000]; [#15 Pouillot et al., 2002]). Key assumptions for this approach are:

<!--T:221-->
* The tests are independent, conditional on disease status (the sensitivity [specificity] of one test is the same, regardless of the result of the other test, as discussed in more detail in the section on series and parallel interpretation of tests);
* Test sensitivity and specificity are constant across populations;
* The tests are compared in two or more populations with different prevalence between populations; and 
* There are at least as many populations as there are tests being evaluated.

==== Bayesian estimation ==== <!--T:222-->
Bayesian methods have been developed that allow the estimation of sensitivity and specificity of one or two tests that are compared in single or multiple populations ([#11 Joseph et al., 1995]; [#4 Enøe et al., 2000]; [#10 Johnson et al., 2001]; [#2 Branscum et al., 2005]). These methods allow incorporation of any prior knowledge on the likely sensitivity and specificity of the test(s) and of disease prevalence as probability distributions, expressing any uncertainty about the assumed prior values. Methods are also available for evaluation of correlated tests, but these require inclusion of additional tests and/or populations to ensure that the Bayesian model works properly ([#7 Georgiadis et al., 2003]). 

<!--T:223-->
Bayesian methods rely on the same assumptions as the maximum likelihood methods. In addition, Bayesian methods also assume that appropriate and reasonable distributions have been used for prior estimates for sensitivity and specificity of the tests being evaluated and prevalence in the population(s). For critical distributions where prior knowledge is lacking it may be appropriate to use an uninformative (uniform) prior distribution.

==== Comparison with a known reference test ==== <!--T:224-->
Sensitivity and specificity can also be estimated by comparison with a reference test of known sensitivity and/or specificity ([#19 Staquet et al., 1981]). These methods cover a variety of circumstances, depending on whether sensitivity or specificity or both are known for the reference test. Key assumptions are conditional independence of tests, and that the sensitivity and/or specificity of the reference test is known.

<!--T:225-->
In the special situation where the reference test is known to be close to 100% specific (for example culture or PCR-based tests), the sensitivity of the new test can be estimated in those animals that test positive to the reference test:

<!--T:226-->
Se(new test) = Number positive to both tests / Total number positive to the comparison test

<!--T:227-->
However, the specificity of the new test cannot be reliably estimated in this way, and will generally be under-estimated. 

==== Estimation from routine testing data ==== <!--T:228-->
Where a disease is rare, and truly infected animals can be eliminated from the data, it is possible to estimate test specificity from routine testing results, such as in a disease control program ([#17 Seiler, 1979]). In this situation, test-positives are routinely subject to follow-up, so that truly infected animals are identified and removed from the population. It is also possible to identify and exclude tests from known infected herds or flocks. Specificity can then be estimated as:

<!--T:229-->
Sp = 1 - (Number of reactors / Total number tested)

<!--T:230-->
In fact, this is an under-estimate of the true specificity, because there may be some unidentified but infected animals remaining in the data after exclusion of tests from known infected animals or herds/flocks.

<!--T:231-->
For example: The flock-specificity of pooled faecal culture for the detection of ovine Johne's disease was estimated from laboratory testing records in New South Wales ([#18 Sergeant et al., 2002]). In this analysis, there were nine test-positive flocks out of 227 flocks eligible for inclusion in the analysis. After exclusion of results for seven known infected flocks, there were 2/220 flocks positive, resulting in an estimated minimum flock-specificity of 99.1% (95% Binomial CI: 96.9% - 99.9%). In fact one or both of these flocks could have been infected, and the true flock-specificity could be higher than the estimate of 99.1%.

==== Modelling approaches ==== <!--T:232-->
Several novel approaches using modelling have also been used to estimate test sensitivity and/or specificity without having to rely on a comparison with either a gold standard or an alternative, independent test. 

==== Mixture modelling ==== <!--T:233-->
One approach to estimating test sensitivity and specificity in the absence of a gold standard is that of mixture population modelling. This approach is based on the assumption that the observed distribution of test results (for a test with a continuous outcome reading such as an ELISA) is actually a mixture of two frequency distributions, one for infected individuals and one for uninfected individuals. 

<!--T:234-->
Using mixture population modelling methods, it is possible to determine the theoretical probability distributions for uninfected and infected sub-populations that best fit the observed data, and from these distributions to estimate sensitivity and specificity for any cut-point.

<!--T:235-->
For example, this approach was used to estimate sensitivity and specificity for ELISA for ''Toxoplasma gondii'' infection in Dutch sheep ([#14 Opsteegh et al., 2010]). ELISA results from 1,179 serum samples collected from sheep at slaughterhouses in the Netherlands were log transformed and normal distributions fitted to the infected and uninfected components. The resulting theoretical distributions allowed determination of a suitable cut-point with estimated sensitivity of 97.8% and specificity of 96.4%.

<!--T:236-->
While this is a useful approach for estimating sensitivity and specificity in the absence of suitable comparative test data, it does depend on the assumptions that the test results follow the theoretical distributions calculated and that the sample tested is representative of the population at large. If the actual results deviate significantly from the theoretical distributions, or the sample is biased, estimates will also be biased. 

==== Simulation modelling of longitudinal testing results ==== <!--T:237-->
An alternative approach, using simulation modelling, has been used where no comparative test data was available, but results of repeated testing over time were available. In this example, the sensitivity of an ELISA for bovine Johne's disease was estimated from repeated herd-testing results over a 10-year period using a simulation model. Age-specific data from up to 7 annual tests in 542 dairy herds were used to estimate ELISA sensitivity at the first-round test. The total number of infected animals present at the first test was estimated from the number of reactors detected at that test, plus the estimated number of animals that failed to react at that test, but reacted (or would have reacted if they had not died or been previously culled) at a subsequent test, based on reactor rates at subsequent tests. Reactor rates were adjusted for an assumed ELISA specificity of 99.8% to ensure estimates were not biased by imperfect ELISA specificity ([#12 Jubb et al., 2004]). Age-specific estimates of ELISA sensitivity ranged from 1.2% in 2-year-old cattle to 30.8% in 10-year-old cattle, with an overall age-weighted average of 13.5%. 

<!--T:238-->
This approach depends on the assumption that most JD-infected animals become infected at a young age, and that all animals that subsequently reacted to the ELISA were in fact infected at the time of the first test. If adult infection occurred in these animals the estimated sensitivity could have substantially under-estimated the true value.

== References - diagnostic testing == <!--T:239-->
Branscum, A. J., Gardner, I. A. & Johnson, W. O. 2005. Estimation of diagnostic-test semnsitivity and specificity through Bayesian modelling. Preventive Veterinary Medicine, 68:145-163.

<!--T:240-->
Cicchetti, D. V. & Feinstein, A. R. 1990. High agreement but low kappa: II. Resolving the paradoxes. Journal-of-Clinical-Epidemiology, 43:551-558.

<!--T:241-->
EnØe, C., Georgiadis, M. P. & Johnson, W. O. 2000. Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown. Preventive Veterinary Medicine, 45:61-81.

<!--T:242-->
Everitt, R. S. 1989. Statistical Methods for Medical Investigation, New York, Oxford University Press.

<!--T:243-->
Feinstein, A. R. & Cicchetti, D. V. 1990. <nowiki>High agreement but low kappa: I. The problems of two paradoxes. [see comments.]. Journal of Clinical Epidemiology., 43:43-9.</nowiki>

<!--T:244-->
Georgiadis, M. P., Johnson, W. O., Singh, R. & Gardner, I. A. 2003. Correlation-adjusted estimation of sensitivity and specificity of two diagnostic tests. Applied statistics, 52:63-76.

<!--T:245-->
Greiner, M. & Gardner, I. A. 2000. Epidemiologic issues in the validation of veterinary diagnostic tests. Preventive Veterinary Medicine, 45:3-22.

<!--T:246-->
Hui, S. L. & Walter, S. D. 1980. Estimating the error rates of diagnostic tests. Biometrics, 36:167-171.

<!--T:247-->
Johnson, W. O., Gastwirth, J. L. & Pearson, L. M. 2001. Screening without a "gold standard": the Hui-Walter paradigm revisited. American Journal of Epidemiology, 153:921-924.

<!--T:248-->
Joseph, L., Gyorkos, T. W. & Coupal, L. 1995. Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. American-Journal-of-Epidemiology, 141:263-272.

<!--T:249-->
Jubb, T. F., Sergeant, E. S. G., Callinan, A. P. L. & Galvin, J. W. 2004. Estimate of sensitivity of an ELISA used to detect Johne's disease in Victorian dairy herds. Australian Veterinary Journal, 82:569-573.

<!--T:250-->
Martin, S. W., Shoukri, M. & Thorburn, M. A. 1992. Evaluating the health status of herds based on tests applied to individuals. Preventive-Veterinary-Medicine, 14:33-43.

<!--T:251-->
Opsteegh, M., Teunis, P., Mensink, M., Zuchner, L., Titilincu, A., Langelaar, M. & van der Giessen, J. 2010. Evaluation of ELISA test characteristics and estimation of Toxoplasma gondii seroprevalence in Dutch sheep using mixture models. Preventive Veterinary Medicine.

<!--T:252-->
Pouillot, R., Gerbier, G. & Gardner, I. A. 2002. "TAGS", a program for the evaluation of test accuracy in the absence of a gold standard. Preventive Veterinary Medicine, 53:67-81.

<!--T:253-->
Rogan, W. J. & Gladen, B. 1978. Estimating prevalence from the results of a screening test. American Journal of Epidemiology, 107:71-76.

<!--T:254-->
Seiler, R. J. 1979. The non-diseased reactor: considerations on the interpretation of screening test results. Veterinary-Record, 105:226-228.

<!--T:255-->
Sergeant, E. S. G., Whittington, R. J. & More, S. J. 2002. Sensitivity and specificity of pooled faecal culture and serology as flock-screening tests for detection of ovine paratuberculosis in Australia. Preventive-Veterinary-Medicine, 52:199-211.

<!--T:256-->
Staquet, M., Rozencweig, M., Lee, Y. J. & Muggia, F. M. 1981. Methodology for the assessment of new dichotomous diagnostic tests. Journal-of-Chronic-Diseases, 34:599-610.
</translate>