NR ANWF
AU Moynagh,J.; Schimmel,H.; Kramer,G.N.
TI THE EVALUATION OF TESTS FOR THE DIAGNOSIS OF TRANSMISSIBLE SPONGIFORM ENCEPHALOPATHY IN BOVINES (PRELIMINARY REPORT)
QU EUROPEAN COMMISSION, DIRECTORATE-GENERAL XXIV, CONSUMER POLICY AND CONSUMER HEALTH PROTECTION, Directorate B - Scientific Health Opinions, Unit B3 - Management of scientific committees II, 8 JULY 1999, Vers 1.2
IA http://europa.eu.int/comm/dg24/health/bse/bse12_en.html
VT
Opinions expressed in this report do not necessarily reflect the views of the European
Commission.
1. Background
1.1 Objective
It has been recognised that the availability of a rapid and accurate diagnostic test would be a major advance in dealing with the problem of BSE and TSEs in general. In particular this problem was recognised by the European Parliament when reporting on the handling of the BSE crisis. The need for reliable rapid diagnostic tests for TSEs has not diminished despite the decrease in reported cases of BSE
The objective of the evaluation exercise is to evaluate diagnostic tests for TSE in bovines. This evaluation is designed to obtain concrete information on the performance of the tests which are either currently available or at an advanced stage of development. The availability of this information will allow decisions to be taken on possible future uses of such tests.
This test evaluation has been designed and managed by Directorate General 24 of the European Commission, Consumer Policy and Consumer Health Protection in collaboration with an expert group and the Institute for Reference Materials and Measurements (IRMM), where all practical aspects of the sample preparation were carried out. The IRMM is an institute of the Joint Research Centre of the European Commission and is based in Geel, Belgium.
1.2 Parameters Examined
No diagnostic test is 100% perfect, in respect of its ability to correctly detect infected and uninfected animals. However before a test is used it is essential, particularly where its use impinges on decisions relating to public health, to know how it performs.
All diagnostic tests have a level of false positives and false negatives. The best tests minimise these factors. This is particularly important in considering a rare disease which has public health implications such as BSE. Because the disease is rare, at least in most animal populations, even a low level of false positives can greatly exceed the true positives detected by the test. Unless a good confirmatory test is available, this situation may simply cause confusion and be economically wasteful. On the other hand, low sensitivity, resulting in false negative reactions will result in diseased animals reaching the market with possible public health consequences. The sensitivity and the specificity of diagnostic tests can be adjusted by varying the cut-off point, the point taken as the divider between the positive and negative readings.
Usually increasing the sensitivity will reduce the specificity and vice versa. For the initial assessment of the tests the cut-off point used was that proposed by the test developers. These values were sometimes arrived at on the basis of limited small scale trials. Following this assessment consideration can be given to selecting the optimum cut-off value or indeed to whether the inclusion of an inconclusive or doubtful category is useful.
The following parameters were examined in the evaluation:
- Diagnostic specificity: the ability to correctly identify non-infected animals;
- Diagnostic sensitivity: the ability to correctly identify infected animals with clinical symptoms;
- The ability of the tests to detect small quantities of PrP, the substance implicated in the causation and progression of TSE diseases. This could give an indication of the capability of the tests to detect the presence of pre-clinical BSE.
1.3 Planning
The organisation of the evaluation was divided into a number of phases.
1. Initial investigation to confirm the feasibility of carrying out the evaluation
2. Design of experimental protocol
3. Call for expression of interests to identify the most advanced rapid diagnostic tests
4. Selection of the most advanced tests for evaluation.
5. Sourcing of large quantities of both BSE infected nervous tissue and uninfected tissue.
6. Preparation of tissue as blind coded samples in the P3 facilities at IRMM Belgium.
7 Testing of the blind samples by the participants
8. Analysis of the results.
2. Selection of participants
2.1 Call for expression of interest
Expressions of interest were invited from those who had tests in advanced stages of development or available for use. This call was published in the Official Journal of the European Communities No. S96 of 19 May 1998. Thirty potential applicants requested additional information and were supplied with a detailed document setting out general conditions and detailed specifications. Applications were received from nine organisations which covered ten tests.
In order to ensure that useful tests would be widely available, applicants were also required to give assurances that they were prepared to make their tests available on a non discriminatory basis following the evaluation.
2.2 Assessment of applications
The applications were assessed by a scientific expert panel comprising four external scientists based on criteria covering the scientific basis for the test, available experimental evidence, practicality of the sampling and testing procedures and stage of development of the test. Based on this assessment, four tests were selected for inclusion in the test evaluation exercise. These tests are listed in section 3 below.
3. Tests Selected for Evaluation
Tests from the following four organisations were selected for participation in the evaluation exercise on the basis of the information submitted by them in response to the call for expression of interest.
Test A E. G. & G. Wallac Ltd., United Kingdom
Test B Prionics A.G., Switzerland
Test C Enfer Technology Ltd., Ireland
Test D Commissariat à l'Energie Atomique (CEA), France
4. Evaluation Protocol
The evaluation exercise was designed primarily to evaluate the sensitivity, specificity and detection limits associated with each of the selected tests. All samples were presented for testing in a coded 'blind' format. The protocol used is attached in Annex 2.
4.1 Sensitivity
The sensitivity of a test is the proportion of infected reference animals that test positive in the assay. 336 samples from 300 individual animals were used to assess this element.
4.2 Specificity
The specificity of a test is the proportion of uninfected reference animals that test negative in the assay. A total of 1064 samples from 1000 individual animals were included in the exercise for this purpose
4.3 Detection limits and Titre
The material used to prepare the dilutions was nervous tissue that had been titrated in mice, yielding a titre of 10^3.1 mouse i.c./i.p LD50/g of tissue.
To assess the test detection limits, various dilutions of the positive brain homogenate were used. The sample was presented in various dilutions down to 10 -5 .The objective of this was to determine the test detection limits and also to gain a perspective on the behaviour of the test in pre clinical animals. In order to achieve acceptable viscosity, 20% aequous solution containing 5% sucrose was added to the homogenate. The particle size of both the positive material and the negative "diluent" was measured in order to ensure what adequate mixing would occur at the various dilutions.
The number of samples examined by each test is set out below.
-----------------------------------------
Dilution Number of Dilution Number of
samples samples
Undiluted 06 10^-3.0 20
10^-1.0 20 10^-3.5 20
10^-1.5 20 10^-4.0 20
10^-2.0 20 10^-4.5 20
10^-2.5 20 10^-5.0 20
-----------------------------------------
4.4 Additional features
Duplicated whole brain material was also included, i.e. two samples from the same animal. In all 36 positive and 64 negative samples were duplicated. This was done to assess the repeatability of the tests in whole tissue and to ascertain the variability of results dependant on the location of the sampling site in the brain stem or spinal cord.
----------------------------------------------------------------
Table 1 - Breakdown of samples tested by the participants
Tissue slices Tissue homogenate
Negative tissue 1000 Negative 20
Positive tissue 300 Undiluted positive 6
Duplicate positive 36 Diluted positive 180 (9x20)
Duplicate negative 64
----------------------------------------------------------------
5. Selection of Positive and Negative Tissue
As stated above sections of both CNS tissue and tissue homogenates were used in the evaluation of each test.
1. The whole central nervous tissue was of the type which was used routinely in each test and on which most of the test development work had been carried out. Three participants received brain stem tissue and the fourth (test C) received cervical spinal cord.
2. All participants received identical samples of the homogenised central nervous tissue.
The known positive and known negative material was obtained as described below.
5.1 Tissue from Infected Reference Animals
5.1.1 Criteria for selection
Brainstem and spinal cord samples were selected from bovines showing clinical signs of BSE and were included in the project following confirmatory tests as detailed below. These samples were supplied by the Central Veterinary Laboratory, Weybridge, United Kingdom.
5.1.2 Collection methods
Following euthanasia, the brainstem was removed from each suspect animal by extraction via the foramen magnum. Spinal cord samples were removed by cutting the neck and pushing the spinal cord out of the vertebral column with a suitable piece of PVC tubing.
5.1.2 Confirmatory tests
A slice of tissue containing the obex with a thickness of about 1cm was removed for diagnosis. This tissue was fixed in 10% formal saline, followed by standard histopathological processing and embedding in paraffin wax, and staining of 5µm sections with a modified HE stain. The diagnosis was based on light microscopic examination of the major BSE target sites at the obex i.e. the nucleus of solitary tract and nucleus of the spinal tract of the trigeminal nerve. Samples were declared positive upon observation of characteristic vacuolation (spongiform change) affecting grey matter neuropil with a systematic and usually bilaterally symmetrical distribution in either the nucleus of the solitary tract or the nucleus of the spinal tract of the trigeminal nerve. At least three neurophil vacuoles were noted in a lesion target site.
5.2 Tissue from Uninfected Reference Animals
5.2.1 Criteria for selection
Samples were collected in New Zealand from healthy adult bovines (at least 4 years old) of mixed breeds. New Zealand was selected as the source for negative material because of its widely recognised high status as regards freedom from TSE diseases. No cases of BSE have ever been reported in New Zealand. In addition its cautious importation policy has minimised the risk of importing animals or feed that might carry the disease.
5.2.2 Collection methods
Following slaughter, the skull was opened to allow the removal of the brain. 10 cm of anterior cervical spinal cord was removed from the carcass by extrusion.
5.2.3 Confirmatory tests
Animals were healthy and showed no clinical signs suggestive of nervous disease. As an additional check, the obex from each sample was removed and subjected to histological examination as described in 5.1.2 above. No samples were removed from the study as a results of this examination.
6 Sample Preparation and Testing Procedure
The positive and negative material was delivered to the EU IRMM at Geel Belgium where the samples preparation took place.
Prior to the arrival of the samples the sample tissue containers were prepared and pre-labelled according to a randomised formula. The containers were then separated into two separate batches, those destined for positive and negative material.
Before being processed each sample was inspected visually and the code given by the suppliers was checked. Inappropriate samples or samples with inconsistencies concerning the code were not processed. The samples were then cut according to one of three schemes depending on the anatomical presentation of each sample. The presentation varied due to the natural fluctuation of the sample size and the difficulties in achieving fully reproducible sampling conditions in the abattoirs (NZ) and the veterinary centres (UK). Fig. 1 below shows the most frequently applied cutting scheme and all three schemes used are shown in Annex 3. The following criteria were applied for the selection of the sampling scheme:
---------------------------------------------------------
Minimum length Minimum length of the
of the brain depression between the
stem posterior cerebellar penduncles
to the obex (Ventricle) on the brain
(L2) stem anterior to the obex (L1)
Scheme 1 2.1 to 3 cm 1.5 cm
Scheme 2 1.4 to 2 cm 2.0 cm
Scheme 3 0.7 to 1 cm 2.5 cm
---------------------------------------------------------
Samples which did not fulfil these criteria were rejected. Of the 1064 negative samples 8 have been processed according to scheme 1, 899 to scheme 2 and 157 to scheme 3. 10 of the 336 positive samples were cut according to scheme 1, 283 according to scheme 2 and 43 according to scheme 3. Six sets consisting of brain stem slices and three sets consisting of spinal cord slices of 1400 samples each have been prepared. As an average 16 % each of the negative brain stem samples were taken from positions 1.1, 1.2, 1.3, 1.4, 1.5 and 1.6. The slice 1.1 of the positive samples has been always reserved for the set which was kept by IRMM for confirmation purposes. This means that 20 % each of the positive brain stem samples were taken from positions 1.2, 1.3, 1.4, 1.5 and 1.6. The sets consisting of spinal cord contained each 33 % each of samples 3.1, 3.2 and 3.3 for negative and positive samples.
The sampling scheme ensured that the brain stem samples supplied to the companies were taken at maximum2 to 2.5 cm away from the obex.
The schemes allowed the recording of the anatomical location of each individual sample to allow analysis of any relationship between the location and the test response. The corresponding duplicate samples were always taken from the opposite side of the brain stem or spinal cord. Approximately 1 g of tissue was used for each sample.
Figure 1: Typical cutting scheme used to prepare the samples
The preparation of the positive and negative samples was separated both in location and time to avoid the possibility of transfer of infectivity or mix up of samples. After addition of 20 % w/w of an aquous solution of 5 % sucrose 20 negative brain stem samples were individually homogenised with an Ultraturrax mixer and dispensed into the vials foreseen. The residues were pooled with another 34 homogenised individual brain stem samples. The pool was filtered and used for the production of the dilution series.
The positive titrated homogenate (see chapter4.4)was also diluted with 20%w/w of aquous 5 % sucrose solution and homogenised with an Ultraturrax. The 10^-1 and 10^-1.5 dilution was prepared by gravimetrical mixing of appropriate amounts of the pooled negative material and the titrated homogenate, both containing 20 % w/w of aquous 5 % sucrose solution. A propeller stirrer was used as mixing device. The lower dilutions were each prepared by 1 in 10 dilution of the corresponding higher concentrated homogenate.
P3 facilities were used for the positive material. Samples were then air freighted in dry ice to each participant.
The testing of these samples took place in the participants' premises and was supervised by Commission officials. The following additional features were put in place to ensure confidence in the exercise. All samples were sealed for delivery with special numbered seals. Participants were required to test the samples in a particular order and to inform the commission on the opening of each container as well as forwarding each evening the results of that days testing. Following use sample residues and intermediate solutions and homogenates were also sealed. Further details of these requirements are contained in the test protocol in Annex 2.
IN Das Generaldirektorat 24 der Europäischen Kommission, zuständig für Verbraucherpolitik und Gesundheitsschutz für Konsumenten, hat zusammen mit einer nicht näher beschriebenen Expertengruppe und dem Institute for Reference Materials and Measurements (IRMM) der EU-Kommission im belgischen Geel 4 BSE-Schnelltests validiert. Auf einen Aufruf im Official Journal of the European Communities Nummer S96 am 19. Mai 1998 hin meldeten sich 30 potentiell an der Evaluierung interessierte Testentwickler. Nachdem diesen die detailierten Evaluierungsbedingungen zugeschickt worden waren, bewarben sich 9 Hersteller mit 10 Tests um die Teilnahme. Diese sollten zusichern, dass sie ihre Tests nach der Evaluierung allgemein verfügbar machen würden. Von 4 nicht namentlich genannten, angeblich sachkundigen externen Wissenschaftlern wurden die "wissenschaftliche Plausibilität", die Entwicklungsstände und die Praktikabilität der 10 Testverfahren überprüft und die 4 angeblich geeignetesten wurden für die Evaluierung ausgewählt. Dies waren die Tests der E. G. & G. Wallac Ltd., United Kingdom, der Prionics A.G., Switzerland, der Enfer Technology Ltd., Ireland, sowie des Commissariat à l'Energie Atomique (CEA), France.
Für die Evaluierung wurden 336 Proben von 300 an BSE erkrankten und histopathologisch bestätigten Rindern und 1064 Proben von scheinbar gesunden Rindern aus Neuseeland hergestellt. Die positiven Proben wurden vom britischen Central Veterinary Laboratory in Weybridge geliefert. Bei 36 BSE-Tieren und bei 64 "gesunden" Rindern wurden je zwei Proben entnommen, um die Abhängigkeit des Nachweises vom Ort der Probenentnahme zu testen. Um den Nachweis von BSE bei noch nicht erkrankten Tieren zu simulieren, wurde homogenisiertes Nervengewebe mit einem Titer von 10 hoch 3,1 für die Hälfte der intrazerebral und intraperitoneal infizierten Mäuse tödlichen Einheiten pro Gramm bis zu 100.000-fach mit homogenisiertem Hirngewebe BSE-freier Rinder verdünnt. Reste von 35 Negativkontrollen wurden vereinigt, gefiltert und für Negativkontrollverdünnungen verwendet.
Offene Fragen:
Who where the four external scientists of your scientific expert panel that selected the four evaluated tests?
AD
Jim Moynagh, Unit B3 Management of Scientific Committees II, Directorate General 24, Consumer Policy and Health Protection, Rue Belliard 232, 1049 Bruxelles, Belgium, James.moynagh@dg24.cec.be
Heinz Schimmel and G.N. Kramer, Reference Materials Unit, Institute for Reference Materials and Measurements, Joint Research Centre, Geel, Belgium, Schimmel@irmm.jrc.be
SP englisch
OR Prion-Krankheiten M