18 F-FDG PET in Squamous Cell Carcinoma of the Oral Cavity and Oropharynx: A Study on Inter- and
Christiaan A. Krabbe, MD, DDS,* Jan Pruim, MD, PhD,†
Asbjørn M. Scholtens, MD,‡ Jan L.N. Roodenburg, DDS, PhD,§
Adrienne H. Brouwers, MD, PhD,储 T.T. Ha Phan, MD, PhD,¶
Ali Agool, MD,# and Pieter U. Dijkstra, PhD**
Purpose: Good observer agreement is mandatory for an effective imaging technique. However, little is known about the observer agreement of fluorine-18 fluorodeoxyglucose (18F-FDG) positron emission tomography (PET) in head and neck squamous cell carcinoma. The aim of the present study was to evaluate the inter- and intraobserver agreement of interpretations of18F-FDG PET in head and neck SCC and to assess the influence of observer experience, tumor localizing, and tumor size on the agreement.
Patients and Methods: 18F-FDG PET scans of 80 patients with oral and oropharyngeal SCC were reassessed twice by 2 experienced nuclear medicine physicians and 2 residents in nuclear medicine. The absolute agreement and Cohen’s kappa were calculated by comparing the results of the 4 observers for the primary tumor, cervical metastases, and distant metastases/second primary tumor. To analyze the sensitivity and specificity, the results were compared with the findings from the histologic specimens or the follow-up data.
Results: The interobserver agreement of the nuclear medicine physicians revealed an absolute agree- ment and kappa of 0.91 and 0.58 for detecting the primary tumor, 0.94 and 0.83 for detecting cervical metastases, and 0.85 and 0.53 for detecting distant metastases/second primary tumors, respectively. The intraobserver agreement was greater overall than the interobserver agreement. Compared with the nuclear medicine physicians, the residents scored lower in interobserver agreement. The interobserver agreement decreased when localizing the malignancy more precisely. The agreement and sensitivity increased with tumor size. However, for small metastases, a high observer agreement was found owing to the nondetection of these malignancies.
Conclusions: Good inter- and intraobserver agreement in SCC in the oral cavity or oropharynx with
18F-FDG PET was found. Observer experience had limited influence on observer agreement. However, the agreement level decreased when a more precise anatomic tumor localization was required.
©2010 American Association of Oral and Maxillofacial Surgeons J Oral Maxillofac Surg 68:21-27, 2010
*Resident, Department of Oral and Maxillofacial Surgery, Univer- sity Medical Center Groningen, Groningen, The Netherlands.
†Assistant Professor, Department of Nuclear Medicine and Mo- lecular Imaging, University Medical Center Groningen and Univer- sity of Groningen, Groningen, The Netherlands.
‡Research Fellow, Department of Nuclear Medicine and Molec- ular Imaging, University Medical Center Groningen and University of Groningen, Groningen, The Netherlands.
§Professor, Department of Oral and Maxillofacial Surgery, Uni- versity Medical Center Groningen and University of Groningen, Groningen, The Netherlands.
储Assistant Professor, Department of Nuclear Medicine and Mo- lecular Imaging, University Medical Center Groningen and Univer- sity of Groningen, Groningen, The Netherlands.
¶Resident, Department of Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, Groningen, The Netherlands.
#Resident, Department of Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, Groningen, The Netherlands.
**Associate Professor, Center for Rehabilitation and Depart- ment of Oral and Maxillofacial Surgery, University Medical Cen- ter Groningen and University of Groningen, Groningen, The Netherlands.
Address correspondence and reprint requests to Dr Krabbe:
Department of Oral and Maxillofacial Surgery, University Medical Center Groningen, PO Box 30.001, Groningen 9700 RB, The Neth- erlands; e-mail:email@example.com
©2010 American Association of Oral and Maxillofacial Surgeons 0278-2391/10/6801-0005$36.00/0
The management of oral and oropharyngeal squa- mous cell carcinoma (SCC) varies according to the tumor size, infiltration of surrounding tissues, and the absence or presence of metastases. Staging the tumor correctly is essential to ensure that the patient is treated optimally with the least possible amount of treatment-related morbidity. Usually, staging of oral and oropharyngeal SCC relies on clinical examination, computed tomography (CT), magnetic resonance im- aging (MRI), and/or ultrasonography (US), with or without guided fine needle aspiration cytology.1-6
Additionally, fluorine-18 fluorodeoxyglucose (18F- FDG) positron emission tomography (PET) has been shown to be an effective imaging technique in the diagnostic workup of oral and oropharyngeal SCC, especially in detecting locoregional and distant metas- tases.7-9 Recently, the support for 18F-FDG PET as a primary imaging technique of oral and oropharyngeal SCC has grown considerably. The sensitivity and spec- ificity of18F-FDG PET were both shown to be com- parable or superior to conventional imaging.10-16The advantage of PET lies in its ability to assess both locoregional and distant metastases in a single func- tional imaging modality.17The greatest drawback of
18F-FDG PET is its lack of anatomic detail and its relatively low resolution.18 This drawback has been eliminated by the development of the combined PET/CT imaging technique, a technique that has be- come more or less the standard.19
To be effective as a primary imaging technique, not only is good sensitivity and specificity required, but also consistency in the interpretations between the same observer at different times and between differ- ent observers is mandatory. This quality is independent of whether PET is used alone or combined with CT. In addition, it is important to know whether a certain level of expertise in evaluating18F-FDG PET images for oral and oropharyngeal SCC is required to interpret the findings effectively and consistently. Surprisingly little is known about the inter- and intraobserver agreement of18F-FDG PET images in head and neck SCC. Technological develop- ments seem to develop more quickly than the evaluation of observer properties and their influence on the interpreta- tions of18F-FDG PET images. Therefore, the aim of the present study was to evaluate the inter- and intraobserver agreement of the interpretations of18F-FDG PET images in oral and oropharyngeal SCC and to assess the influence of observer experience, tumor localization, and tumor size on the agreement and sensitivity and specificity.
Patients and Methods
The18F-FDG PET scans of 80 patients (31 women and 49 men with a mean age of 61.3 years, SD 11.3)
with newly diagnosed SCC of the oral cavity and/or oropharynx who had undergone 18F-FDG PET from 1999 to 2004 were retrieved. In all patients,18F-FDG PET scans were acquired for tumor staging before treatment. The diagnosis of SCC was confirmed histo- logically before 18F-FDG PET scanning. Of the 80 tumors, 62 were located in the oral cavity and 18 in the oropharynx. The T stage was determined from the histologic findings. The N stage was also determined from the histologic findings when available (n⫽ 50).
If no neck dissection had been performed, the N stage was determined from the cytologic finding (n⫽ 10) or, if also not available, the results of the diagnostic examinations (CT, MRI, and/or US) and clinical fol- low-up of at least 1.5 years (n⫽ 20). In 39 patients, cervical metastases (stage N⫹) were present in 51 neck sides; no cervical metastases were found (stage N0) in the other 41 patients. In 23 patients (28 neck sides), the cervical metastases were diagnosed by his- tologic examination, in 5 patients (5 neck sides), the metastases were diagnosed by cytologic examination, and in 11 patients (18 neck sides) by the clinical findings. The TN classification of the tumors is listed in Table 1. In 8 patients, a malignancy outside the head/neck region was diagnosed: 4 cases of lung carcinoma, 1 thyroid tumor, 1 skeletal and 1 infracla- vicular metastasis, and 1 esophagus carcinoma.
Of the 80 patients, 56 were treated with primary surgery, of whom 38 received supplementary radio- therapy. A total of 68 neck dissections in 50 patients were performed, of which 39 were supraomohyoid- eal, 28 were modified, and 1 was radical. A total of 19 patients were treated with primary radiotherapy, 7 of whom also received chemotherapy. On the neck sides without clinical evidence of cervical metastases but without cytologic or histologic findings obtained, no radiotherapy was applied. Finally, 5 patients re- ceived no therapy other than palliation.
Table 1. POST-TREATMENT TN CLASSIFICATION OF PRIMARY TUMORS
Stage N0 N1 N2 Total
T1 16 1 0 17*
T2 11 4 4 19
T3 5 3 2 10
T4 9 10 15 34
Total 41 18 21 80
Presence or absence of nodular involvement determined by histologic examination (n⫽ 50), cytologic examination (n⫽ 10), or clinical follow-up (n ⫽ 20).
*Two tumors were completely excised by excision bi- opsy before fluorine-18 fluorodeoxyglucose positron emis- sion tomography.
Krabbe et al. PET Observer Agreement in Head and Neck SCC.
J Oral Maxillofac Surg 2010.
18F-FDG PET STUDY
All 80 patients underwent whole body 18F-FDG PET. The scans were performed on 1 of the 2 cameras available: an ECAT 951 or an ECAT HR⫹ whole body camera (Siemens CTI, Knoxville, TN). The ECAT HR⫹ device acquires 63 planes over 15.5 cm. The mea- sured resolution of the system is 6-mm full width at half maximum, transaxially in the center of the field of view. The ECAT 951 acquires 31 planes over a 10.9-cm field, also with a resolution of 6-mm full width at half maximum. The patients fasted for at least 4 hours before being administered FDG. In all cases, FDG was injected intravenously 90 minutes before the onset of scanning.
The 80 18F-FDG PET scans were reassessed by 4 independent observers: 2 experienced nuclear medi- cine physicians and 2 residents in nuclear medicine.
The most experienced nuclear medicine physician (nuclear medicine physician I) had 15 years of expe- rience evaluating 18F-FDG PET scans for head and neck cancer, the other nuclear medicine physician (nuclear medicine physician II) had 5 years of expe- rience. Resident I was about to finish the 4-year resi- dency, and resident II was approximately halfway through it.
The scans were presented without the patient or medical information other than the diagnosis of SCC of the oral cavity or oropharynx. All observers as- sessed all scans in random order twice, with a 3-week interval, resulting in a first and second data set for each observer. For the second assessment, the results of the first assessment were not shown. The18F-FDG PET scan interpretation was by visual assessment. The primary tumor and the presence of cervical metasta- ses and distant malignancy were assessed. The observ- ers were asked to identify and localize an eventual abnormally increased18F-FDG uptake using a standard scoring form. The abnormally increased18F-FDG up- take was graded on a 5-point scale: definitely benign, probably benign, equivocal, probably malignant, and definitely malignant. For data analysis, the results of the observers given in the 5-point scale were dichot- omized. The results for definitely benign and probably benign were considered negative for malignancy, and the results for equivocal, probably malignant, and definitely malignant were considered positive for ma- lignancy.
Inter- and Intraobserver Agreement
The inter- and intraobserver agreement was calcu- lated by comparing the dichotomized results of the 4 observers for detecting the primary tumor, cervical
metastases per neck side, and distant malignancy. The inter- and intraobserver agreement was calculated us- ing the absolute agreement and Cohen’s kappa ().
The absolute agreement is the ratio of the findings in which agreement exists with the total findings. Co- hen’s is the ratio between chance-corrected ob- served agreement and chance-corrected perfect agree- ment. Interpretations of the values were as follows:
⬍0.21 indicated poor; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, good; and⬎0.80 almost per- fect agreement.20Statistical analyses were performed using the Statistical Package for Social Sciences, for Windows, statistical package, version 12.1 (SPSS, Chi- cago, IL). The first data set of dichotomized findings was used to calculate the interobserver agreement between the 2 nuclear medicine physicians and be- tween the 2 residents. The first and second data set of dichotomized findings of each observer were used to calculate the intraobserver agreement of all observers.
To analyze whether the experience of the observ- ers influenced the agreement, the inter- and intraob- server agreement were compared using 95% confi- dence interval analysis.21 If no overlap in the 95%
confidence intervals was found, the differences in agreement were considered significant.
Influence of Tumor Localization and Tumor Size Determining the location of the primary tumor and any cervical metastases by the observers was studied to analyze the influence of tumor localization on the interobserver agreement between the nuclear medi- cine physicians. The influence of tumor size was also analyzed. The tumor size was determined from the pathologic findings when available or the radiologic results (CT, MRI, or US) when not available. Also, the effect of tumor size on the sensitivity of the nuclear medicine physicians’ findings was analyzed.
Sensitivity and Specificity
To examine whether the results in our study were valid, the sensitivity and specificity were calculated.
The first data set of dichotomized findings of the 4 observers was used to calculate the sensitivity and specificity of the 18F-FDG PET scan interpretations using the obtained histologic specimens, cytologic findings, or results from follow-up. To analyze the influence of the experience of the observers on the diagnostic properties, the sensitivity and specificity of the observers were compared using 95% confidence interval analysis.21If no overlap in the 95% intervals was found, the differences in sensitivity and specific- ity were considered significant.
INTER- AND INTRAOBSERVER AGREEMENT
The results of the inter- and intraobserver agree- ment of the 4 observers for detecting the primary
tumor, cervical metastases per neck side, and distant malignancy are summarized inTable 2. The interob- server agreement between the nuclear medicine phy- sicians was greater than the interobserver agreement between the residents, but no significant differences were found other than for detecting cervical metasta- ses (Table 2). The intraobserver agreement of the nuclear medicine physicians was, in general, greater than the intraobserver agreement of the residents for detecting malignancy; however, only 1 significant dif- ference was found (Table 2). Moreover, for all 4 observers, the intraobserver agreement was generally greater than the interobserver agreement. Two differ- ent PET cameras were used in the present study.
However, no difference in performance between the 2 cameras was found.
INFLUENCE OF TUMOR LOCALIZATION AND TUMOR SIZE
The interobserver agreement between the 2 nu- clear medicine physicians decreased when more pre- cise localization was required (Table 3). The and absolute agreement of the location of the primary tumor decreased when attempting to localize the pri- mary tumor in the oral cavity or oropharynx. When attempting to localize the cervical metastases in the separate nodal levels in the neck, the decreased for all levels. Only in level I did the interobserver agree- ment show a high (0.76). The absolute interob- server agreement remained high (ⱖ86%) when the level of localization was taken into account. For level V, no could be calculated, because the metastases were only found in this level by 1 observer.
The influence of tumor size on interobserver agree- ment is presented in Table 4. In 2 patients, the pri- mary tumor had already been completely removed before 18F-FDG PET scanning. In 7 of 78 scans, the nuclear medicine physicians did not agree on the presence of the primary tumor. In 6 of these 7 scans,
the primary tumor was classified as stage T1 with a maximal diameter of 20 mm and an invasion depth of 5 mm. In one scan, the primary tumor was classified as stage T2 with a maximal diameter of 31 mm and an invasion depth of 4 mm.
No influence from the size of the cervical metasta- ses on interobserver agreement was found. Only in 1 of the 10 neck sides in which disagreement existed was a cervical metastasis present. This lymph node was 25 mm.
The sensitivities of the nuclear medicine physicians for the primary tumor and cervical metastases catego- rized by tumor size are also listed in Table 4. The sensitivity increased with increased tumor size.
SENSITIVITY AND SPECIFICITY
The sensitivity and specificity of the interpretations of the 4 observers are summarized in Table 5. The least experienced resident (resident II) had the lowest
Table 2. INTER- AND INTRAOBSERVER AGREEMENT OF FDG-PET INTERPRETATIONS
Interobserver Agreement Intraobserver Agreement NMPI and
NMPII RI and RII NMPI NMPII RI RII
p0 p0 p0 p0 p0 p0
Primary tumor 0.91 0.58 0.76 0.29 0.90 0.54 0.94 0.58 0.91 0.19 0.85 0.64 Cervical metastases 0.94 0.83* 0.86 0.54* 0.94 0.83 0.94 0.86 0.93 0.81 0.93 0.71 Distant metastases/second
primary tumor 0.85 0.53 0.78 0.26 0.95† 0.84 0.88 0.66 0.79† 0.52 0.94 0.42 Abbreviations: NMPI, nuclear medicine physician I; NMPII, nuclear medicine physician II; RI, resident I; RII, resident II; p0, absolute agreement;, Cohen’s kappa.
*Significant difference between values of NMPs and residents.
†Significant difference between p0of NMPI and RI.
Krabbe et al. PET Observer Agreement in Head and Neck SCC. J Oral Maxillofac Surg 2010.
Table 3. INTEROBSERVER AGREEMENT BETWEEN NUCLEAR MEDICINE PHYSICIANS FOR
LOCALIZATION OF PRIMARY TUMOR AND CERVICAL METASTASES
Primary tumor 0.91* 0.58*
Oral cavity 0.76 0.44
Oropharynx 0.75 0.21
Cervical metastases 0.94* 0.83*
Level I 0.98 0.76
Level II 0.89 0.51
Level III 0.86 ⫺0.06
Level IV 0.95 0.53
Level V 0.94 —
Abbreviations as inTable 2.
*For easy comparison, p0 and values from Table 2 included.
Krabbe et al. PET Observer Agreement in Head and Neck SCC.
J Oral Maxillofac Surg 2010.
sensitivity for all 3 sites compared with the other 3 observers. Because of not adequately detecting malig- nancy, which involved fewer false-positive results, resident II scored better for specificity for cervical metastases and distant metastases/second primary tu- mor compared with the other observers.
The present study has demonstrated high inter- and intraobserver agreement in SCC in the oral cavity or oropharynx with18F-FDG PET. The inter- and intraob- server agreement of the residents were, in general, less than the agreement between the nuclear medi- cine physicians, although their agreement was still fair to good (Table 2). Differences in experience be- tween the 2 residents and the 2 nuclear medicine physicians did not result in the superiority of one compared to the other for intraobserver agreement, indicating that observer experience plays only a lim- ited role in the reproducibility of interpreting18F-FDG PET scans. Also, for sensitivity and specificity, ob- server experience seemed to play a limited role. No superiority in the sensitivity and specificity between the 2 nuclear medicine physicians was found, despite their 10-year difference in experience. The sensitivity and specificity of the more experienced resident reached almost the level of the nuclear medicine physicians.
The present study was cross-sectional; therefore, we were not able to show a real learning curve in evaluating the18F-FDG PET scans. However, because the least experienced resident had lower sensitivity in interpreting the18F-FDG PET scans, our results sug- gest a short learning phase exists, after which evalu- ating18F-FDG PET scans reaches an acceptable level.
Other imaging techniques such as US-guided fine nee- dle aspiration cytology, MRI, and CT are generally believed to be more experience and observer depen- dent.22-24 That the interpretation of18F-FDG PET im-
ages in the complex anatomic head and neck region is not very experience and observer dependent is an asset to the technique.
Although the present study found high interob- server agreement for the 2 nuclear medicine physi- cians for detecting malignancies, the values clearly decreased if more precise localization of the malig- nancies was required (Table 3). Only neck level I demonstrated high observer agreement, which can be explained by the easy recognition of this level on the scans. It was not surprising that the observer agree- ment decreased with more precise localization be- cause of the lack of anatomic detail on18F-FDG PET scans. This finding supports the additional value of combining PET with CT (PET/CT) for proper tumor localization, as shown by Syed et al.2518F-FDG PET/CT was superior to18F-FDG PET alone when comparing the interobserver agreement for precise anatomic local- ization of head and neck tumors.25
Tumor size influenced the interobserver agreement and sensitivity of the nuclear medicine physicians (Table 4). The agreements increased with tumor size, with the exception of metastases smaller than 1 cm.
These small cervical metastases showed high interob- server agreement despite the very low sensitivity. The high interobserver agreement resulted from the non- detection of the small metastases by both observers.
Missing metastases 5 mm or smaller was not surpris- ing against the background of the limited resolution of the PET camera.
One of the advantages of18F-FDG PET for the initial staging of head and neck cancer is the possibility of evaluating the whole body for malignancy. All distant metastases/second primary tumors were detected by both nuclear medicine physicians, except for one small superficial esophagus carcinoma, resulting in complete agreement for all second primary tumors and distant metastases. Disagreement, mostly for sus- pected malignancy in the lung or mediastinum, was present in 12 scans, all without proven second pri-
Table 4. ABSOLUTE INTEROBSERVER AGREEMENT AND SENSITIVITY OF NUCLEAR MEDICINE PHYSICIANS STRATIFIED BY TUMOR SIZE
Primary tumor stage T1 T2 T3-T4
Agreement NMPI-NMPII 60% (9/15) 95% (18/19) 100% (44/44)
Sensitivity NMPI 60% (8/15) 89% (17/19) 100% (44/44)
Sensitivity NMPII 67% (10/15) 95% (18/19) 100% (44/44)
Cervical metastasis size (mm) ⬍5 5 to⬍10 ⱖ10
Agreement NMPI-NMPII 100% (14/14) 100% (7/7) 96% (29/30)
Sensitivity NMPI 14% (2/14) 28% (2/7) 87% (27/30)
Sensitivity NMPII 14% (2/14) 28% (2/7) 93% (28/30)
Abbreviations as inTable 2.
Krabbe et al. PET Observer Agreement in Head and Neck SCC. J Oral Maxillofac Surg 2010.
mary tumors or distant metastases, highlighting the known false-positive risk of18F-FDG PET.26
The present study had some limitations. The values should be interpreted with caution because use of the does have a number of drawbacks.27,28 Most notably for our study, the drawback was the influence of the distribution of malignancy. The values tend toward lower values when the distribu- tion is asymmetric. In the present study, the presence of malignancy in the head and body was very asym- metrically distributed: 98% and 10%, respectively.
Thus, despite the high absolute agreement for detect- ing primary tumor and distant metastases, comparable to the agreement for detecting cervical metastases, the values of the primary tumor and distant metas- tases were lower than those for cervical metastases.
The histologic findings of the surgical specimens were used to determine the tumor size. However, for some patients with malignancy, surgical specimens were not obtained. For these patients, the tumor size was determined by CT, MRI, or US performed at diagnosis of the malignancy. The measured diameter was used as the malignancy size. Thus, it is possible that the measurements for these malignancies were somewhat overestimated.
It could be argued that an analysis of PET data is superfluous in the PET/CT era. However, PET/CT is a combination of 2 imaging techniques, each with its own characteristics. To understand the added value of the combination, the value of each of the components should be known. The results of our study have re- vealed that the interpretation of PET data is relatively observer experience independent; however,18F-FDG PET is lacking for locating a tumor. As such, the present study provides a strong argument for the use of PET/CT in the evaluation of SCC of the head and neck.
In conclusion, the 18F-FDG PET images of SCC of the oral cavity or oropharynx showed good inter- and intraobserver agreement for detecting malignancy.
Observer experience played a limited role in observer agreement. Even in difficult areas as the head and neck, the images can be interpreted reliably for oral and oropharyngeal cancer. Observer agreement de- creased, however, when more precise anatomic tu- mor localization was required. Observer agreement and sensitivity increased with tumor size. Small le- sions were missed by all observers, independent of experience, indicating that the role of18F-FDG PET in detecting small cervical metastases is limited.
1. Baatenburg de Jong RJ, Knegt P, Verwoerd CD: Reduction of the number of neck treatments in patients with head and neck cancer. Cancer 71:2312, 1993
NMPI(%)NMPII(%)RI(%)RII(%) SensitivitySpecificitySensitivitySpecificitySensitivitySpecificitySensitivitySpecificity Primarytumor89%(69/78)100%(2/2)92%(72/78)100%(2/2)92%(72/78)50%(1/2)71%*(55/78)50%(1/2) Cervicalmetastases61%(31/51)97%(106/109)63%(32/51)91%†(99/109)61%(31/51)94%(103/109)47%(24/51)100%†(109/109) Distantmetastases/second primarytumor88%(7/8)88%(63/72)88%(7/8)86%(62/72)75%(6/8)78%‡(56/72)63%(5/8)99%‡(71/72) AbbreviationsasinTable2. *SignificantdifferenceinsensitivitybetweenRIIandotherobservers. †SignificantdifferenceinspecificitybetweenRIIandNMPII. ‡SignificantdifferenceinspecificitybetweenRIandRII. Krabbeetal.PETObserverAgreementinHeadandNeckSCC.JOralMaxillofacSurg2010.
2. Bootz F, Lenz M, Skalej M, et al: Computed tomography (CT) and magnetic resonance imaging (MRI) in T-stage evaluation of oral and oropharyngeal carcinomas. Clin Otolaryngol Allied Sci 17:421, 1992
3. Takes RP, Knegt P, Manni JJ, et al: Regional metastasis in head and neck squamous cell carcinoma: Revised value of US with US-guided FNAB. Radiology 198:819, 1996
4. Close LG, Merkel M, Vuitch MF, et al: Computed tomographic evaluation of regional lymph node involvement in cancer of the oral cavity and oropharynx. Head Neck 11:309, 1989 5. Stern WB, Silver CE, Zeifer BA, et al: Computed tomography of
the clinically negative neck. Head Neck 12:109, 1990 6. van den Brekel MW, Castelijns JA, Croll GA, et al: Magnetic
resonance imaging vs palpation of cervical lymph node metas- tasis. Arch Otolaryngol Head Neck Surg 117:663, 1991 7. Rege S, Maass A, Chaiken L, et al: Use of positron emission
tomography with fluorodeoxyglucose in patients with ex- tracranial head and neck cancers. Cancer 73:3047, 1994 8. Goerres GW, Schmid DT, Gratz KW, et al: Impact of whole
body positron emission tomography on initial staging and ther- apy in patients with squamous cell carcinoma of the oral cavity.
Oral Oncol 39:547, 2003
9. Schmid DT, Stoeckli SJ, Bandhauer F, et al: Impact of positron emission tomography on the initial staging and therapy in locoregional advanced squamous cell carcinoma of the head and neck. Laryngoscope 113:888, 2003
10. Adams S, Baum RP, Stuckensen T, et al: Prospective compari- son of 18F-FDG PET with conventional imaging modalities (CT, MRI, US) in lymph node staging of head and neck cancer. Eur J Nucl Med 25:1255, 1998
11. Jabour BA, Choi Y, Hoh CK, et al: Extracranial head and neck:
PET imaging with 2-[F-18]fluoro-2-deoxy-D-glucose and MR im- aging correlation. Radiology 186:27, 1993
12. Di Martino E, Nowak B, Hassan HA, et al: Diagnosis and staging of head and neck cancer: A comparison of modern imaging modalities (positron emission tomography, computed tomog- raphy, color-coded duplex sonography) with panendoscopic and histopathologic findings. Arch Otolaryngol Head Neck Surg 126:1457, 2000
13. Kresnik E, Mikosch P, Gallowitsch HJ, et al: Evaluation of head and neck cancer with 18F-FDG PET: A comparison with con- ventional methods. Eur J Nucl Med 28:816, 2001
14. Kitagawa Y, Nishizawa S, Sano K, et al: Prospective comparison of 18F-FDG PET with conventional imaging modalities (MRI,
CT, and 67Ga scintigraphy) in assessment of combined intraarterial chemotherapy and radiotherapy for head and neck carcinoma. J Nucl Med 44:198, 2003
15. Dammann F, Horger M, Mueller-Berg M, et al: Rational diagno- sis of squamous cell carcinoma of the head and neck region:
Comparative evaluation of CT, MRI, and 18FDG PET. AJR Am J Roentgenol 184:1326, 2005
16. Ng SH, Yen TC, Liao CT, et al: 18F-FDG PET and CT/MRI in oral cavity squamous cell carcinoma: A prospective study of 124 patients with histologic correlation. J Nucl Med 46:1136, 2005 17. Schwartz DL, Rajendran J, Yueh B, et al: Staging of head and neck squamous cell cancer with extended-field FDG-PET. Arch Otolaryngol Head Neck Surg 129:1173, 2003
18. McGuirt WF, Greven K, Williams D III, et al: PET scanning in head and neck oncology: A review. Head Neck 20:208, 1998 19. Quon A, Fischbein NJ, McDougall IR, et al: Clinical role of
18F-FDG PET/CT in the management of squamous cell carci- noma of the head and neck and thyroid carcinoma. J Nucl Med 48:58S, 2007 (Suppl 1)
20. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 33:159, 1977
21. Altman DG, Machin D, Bryant TN, et al: Statistics with Confi- dence: Confidence Intervals and Statistic Guidelines. Bristol, UK, BMJ Books, 2005
22. Hoorweg JJ, Kruijt RH, Heijboer RJ, et al: Reliability of inter- pretation of CT examination of the larynx in patients with glottic laryngeal carcinoma. Otolaryngol Head Neck Surg 135:
23. Keberle M, Kenn W, Muller H, et al: Interobserver variability in CT of oro- and hypopharyngeal carcinomas. Rofo 173:583, 2001
24. Wu M, Burstein DE, Yuan S, et al: A comparative study of 200 fine needle aspiration biopsies performed by clinicians and cytopathologists. Laryngoscope 116:1212, 2006
25. Syed R, Bomanji JB, Nagabhushan N, et al: Impact of combined (18)F-FDG PET/CT in head and neck tumours. Br J Cancer 92:1046, 2005
26. Strauss LG: Fluorine-18 deoxyglucose and false-positive results:
A major problem in the diagnostics of oncological patients. Eur J Nucl Med 23:1409, 1996
27. Cicchetti DV, Feinstein AR: High agreement but low kappa. II.
Resolving the paradoxes. J Clin Epidemiol 43:551, 1990 28. Feinstein AR, Cicchetti DV: High agreement but low kappa. I.
The problems of two paradoxes. J Clin Epidemiol 43:543, 1990