F-FDGPETinSquamousCellCarcinomaoftheOralCavityandOropharynx:AStudyonInter-andIntraobserverAgreement 18

(1)

18 F-FDG PET in Squamous Cell Carcinoma of the Oral Cavity and Oropharynx: A Study on Inter- and

Intraobserver Agreement

Christiaan A. Krabbe, MD, DDS,* Jan Pruim, MD, PhD,†

Asbjørn M. Scholtens, MD,‡ Jan L.N. Roodenburg, DDS, PhD,§

Adrienne H. Brouwers, MD, PhD,储 T.T. Ha Phan, MD, PhD,¶

Ali Agool, MD,# and Pieter U. Dijkstra, PhD**

Purpose: Good observer agreement is mandatory for an effective imaging technique. However, little is known about the observer agreement of fluorine-18 fluorodeoxyglucose (¹⁸F-FDG) positron emission tomography (PET) in head and neck squamous cell carcinoma. The aim of the present study was to evaluate the inter- and intraobserver agreement of interpretations of¹⁸F-FDG PET in head and neck SCC and to assess the influence of observer experience, tumor localizing, and tumor size on the agreement.

Patients and Methods: ¹⁸F-FDG PET scans of 80 patients with oral and oropharyngeal SCC were reassessed twice by 2 experienced nuclear medicine physicians and 2 residents in nuclear medicine. The absolute agreement and Cohen’s kappa were calculated by comparing the results of the 4 observers for the primary tumor, cervical metastases, and distant metastases/second primary tumor. To analyze the sensitivity and specificity, the results were compared with the findings from the histologic specimens or the follow-up data.

Results: The interobserver agreement of the nuclear medicine physicians revealed an absolute agreement and kappa of 0.91 and 0.58 for detecting the primary tumor, 0.94 and 0.83 for detecting cervical metastases, and 0.85 and 0.53 for detecting distant metastases/second primary tumors, respectively. The intraobserver agreement was greater overall than the interobserver agreement. Compared with the nuclear medicine physicians, the residents scored lower in interobserver agreement. The interobserver agreement decreased when localizing the malignancy more precisely. The agreement and sensitivity increased with tumor size. However, for small metastases, a high observer agreement was found owing to the nondetection of these malignancies.

Conclusions: Good inter- and intraobserver agreement in SCC in the oral cavity or oropharynx with

18F-FDG PET was found. Observer experience had limited influence on observer agreement. However, the agreement level decreased when a more precise anatomic tumor localization was required.

©2010 American Association of Oral and Maxillofacial Surgeons J Oral Maxillofac Surg 68:21-27, 2010

*Resident, Department of Oral and Maxillofacial Surgery, Univer- sity Medical Center Groningen, Groningen, The Netherlands.

†Assistant Professor, Department of Nuclear Medicine and Mo- lecular Imaging, University Medical Center Groningen and Univer- sity of Groningen, Groningen, The Netherlands.

‡Research Fellow, Department of Nuclear Medicine and Molec- ular Imaging, University Medical Center Groningen and University of Groningen, Groningen, The Netherlands.

§Professor, Department of Oral and Maxillofacial Surgery, Uni- versity Medical Center Groningen and University of Groningen, Groningen, The Netherlands.

储Assistant Professor, Department of Nuclear Medicine and Mo- lecular Imaging, University Medical Center Groningen and Univer- sity of Groningen, Groningen, The Netherlands.

¶Resident, Department of Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, Groningen, The Netherlands.

#Resident, Department of Nuclear Medicine and Molecular Imaging, University Medical Center Groningen, Groningen, The Netherlands.

**Associate Professor, Center for Rehabilitation and Depart- ment of Oral and Maxillofacial Surgery, University Medical Cen- ter Groningen and University of Groningen, Groningen, The Netherlands.

Address correspondence and reprint requests to Dr Krabbe:

Department of Oral and Maxillofacial Surgery, University Medical Center Groningen, PO Box 30.001, Groningen 9700 RB, The Neth- erlands; e-mail:c.a.krabbe@kchir.umcg.nl

doi:10.1016/j.joms.2009.07.021

21

(2)

The management of oral and oropharyngeal squamous cell carcinoma (SCC) varies according to the tumor size, infiltration of surrounding tissues, and the absence or presence of metastases. Staging the tumor correctly is essential to ensure that the patient is treated optimally with the least possible amount of treatment-related morbidity. Usually, staging of oral and oropharyngeal SCC relies on clinical examination, computed tomography (CT), magnetic resonance imaging (MRI), and/or ultrasonography (US), with or without guided fine needle aspiration cytology.^1-6

Additionally, fluorine-18 fluorodeoxyglucose (¹⁸F- FDG) positron emission tomography (PET) has been shown to be an effective imaging technique in the diagnostic workup of oral and oropharyngeal SCC, especially in detecting locoregional and distant metastases.^7-9 Recently, the support for ¹⁸F-FDG PET as a primary imaging technique of oral and oropharyngeal SCC has grown considerably. The sensitivity and specificity of¹⁸F-FDG PET were both shown to be comparable or superior to conventional imaging.^10-16The advantage of PET lies in its ability to assess both locoregional and distant metastases in a single func- tional imaging modality.¹⁷The greatest drawback of

18F-FDG PET is its lack of anatomic detail and its relatively low resolution.¹⁸ This drawback has been eliminated by the development of the combined PET/CT imaging technique, a technique that has be- come more or less the standard.¹⁹

To be effective as a primary imaging technique, not only is good sensitivity and specificity required, but also consistency in the interpretations between the same observer at different times and between different observers is mandatory. This quality is independent of whether PET is used alone or combined with CT. In addition, it is important to know whether a certain level of expertise in evaluating¹⁸F-FDG PET images for oral and oropharyngeal SCC is required to interpret the findings effectively and consistently. Surprisingly little is known about the inter- and intraobserver agreement of¹⁸F-FDG PET images in head and neck SCC. Technological develop- ments seem to develop more quickly than the evaluation of observer properties and their influence on the interpretations of¹⁸F-FDG PET images. Therefore, the aim of the present study was to evaluate the inter- and intraobserver agreement of the interpretations of¹⁸F-FDG PET images in oral and oropharyngeal SCC and to assess the influence of observer experience, tumor localization, and tumor size on the agreement and sensitivity and specificity.

Patients and Methods

PATIENTS

The¹⁸F-FDG PET scans of 80 patients (31 women and 49 men with a mean age of 61.3 years, SD 11.3)

with newly diagnosed SCC of the oral cavity and/or oropharynx who had undergone ¹⁸F-FDG PET from 1999 to 2004 were retrieved. In all patients,¹⁸F-FDG PET scans were acquired for tumor staging before treatment. The diagnosis of SCC was confirmed histo- logically before ¹⁸F-FDG PET scanning. Of the 80 tumors, 62 were located in the oral cavity and 18 in the oropharynx. The T stage was determined from the histologic findings. The N stage was also determined from the histologic findings when available (n⫽ 50).

If no neck dissection had been performed, the N stage was determined from the cytologic finding (n⫽ 10) or, if also not available, the results of the diagnostic examinations (CT, MRI, and/or US) and clinical follow-up of at least 1.5 years (n⫽ 20). In 39 patients, cervical metastases (stage N⫹) were present in 51 neck sides; no cervical metastases were found (stage N0) in the other 41 patients. In 23 patients (28 neck sides), the cervical metastases were diagnosed by histologic examination, in 5 patients (5 neck sides), the metastases were diagnosed by cytologic examination, and in 11 patients (18 neck sides) by the clinical findings. The TN classification of the tumors is listed in Table 1. In 8 patients, a malignancy outside the head/neck region was diagnosed: 4 cases of lung carcinoma, 1 thyroid tumor, 1 skeletal and 1 infracla- vicular metastasis, and 1 esophagus carcinoma.

Of the 80 patients, 56 were treated with primary surgery, of whom 38 received supplementary radiotherapy. A total of 68 neck dissections in 50 patients were performed, of which 39 were supraomohyoid- eal, 28 were modified, and 1 was radical. A total of 19 patients were treated with primary radiotherapy, 7 of whom also received chemotherapy. On the neck sides without clinical evidence of cervical metastases but without cytologic or histologic findings obtained, no radiotherapy was applied. Finally, 5 patients received no therapy other than palliation.

Table 1. POST-TREATMENT TN CLASSIFICATION OF PRIMARY TUMORS

Stage N0 N1 N2 Total

T1 16 1 0 17*

T2 11 4 4 19

T3 5 3 2 10

T4 9 10 15 34

Total 41 18 21 80

Presence or absence of nodular involvement determined by histologic examination (n⫽ 50), cytologic examination (n⫽ 10), or clinical follow-up (n ⫽ 20).

*Two tumors were completely excised by excision bi- opsy before fluorine-18 fluorodeoxyglucose positron emission tomography.

Krabbe et al. PET Observer Agreement in Head and Neck SCC.

J Oral Maxillofac Surg 2010.

(3)

18F-FDG PET STUDY

All 80 patients underwent whole body ¹⁸F-FDG PET. The scans were performed on 1 of the 2 cameras available: an ECAT 951 or an ECAT HR⫹ whole body camera (Siemens CTI, Knoxville, TN). The ECAT HR⫹ device acquires 63 planes over 15.5 cm. The measured resolution of the system is 6-mm full width at half maximum, transaxially in the center of the field of view. The ECAT 951 acquires 31 planes over a 10.9-cm field, also with a resolution of 6-mm full width at half maximum. The patients fasted for at least 4 hours before being administered FDG. In all cases, FDG was injected intravenously 90 minutes before the onset of scanning.

STUDY DESIGN

The 80 ¹⁸F-FDG PET scans were reassessed by 4 independent observers: 2 experienced nuclear medicine physicians and 2 residents in nuclear medicine.

The most experienced nuclear medicine physician (nuclear medicine physician I) had 15 years of experience evaluating ¹⁸F-FDG PET scans for head and neck cancer, the other nuclear medicine physician (nuclear medicine physician II) had 5 years of experience. Resident I was about to finish the 4-year resi- dency, and resident II was approximately halfway through it.

The scans were presented without the patient or medical information other than the diagnosis of SCC of the oral cavity or oropharynx. All observers assessed all scans in random order twice, with a 3-week interval, resulting in a first and second data set for each observer. For the second assessment, the results of the first assessment were not shown. The¹⁸F-FDG PET scan interpretation was by visual assessment. The primary tumor and the presence of cervical metastases and distant malignancy were assessed. The observers were asked to identify and localize an eventual abnormally increased¹⁸F-FDG uptake using a standard scoring form. The abnormally increased¹⁸F-FDG uptake was graded on a 5-point scale: definitely benign, probably benign, equivocal, probably malignant, and definitely malignant. For data analysis, the results of the observers given in the 5-point scale were dichotomized. The results for definitely benign and probably benign were considered negative for malignancy, and the results for equivocal, probably malignant, and definitely malignant were considered positive for malignancy.

STATISTICAL ANALYSIS

Inter- and Intraobserver Agreement

The inter- and intraobserver agreement was calculated by comparing the dichotomized results of the 4 observers for detecting the primary tumor, cervical

metastases per neck side, and distant malignancy. The inter- and intraobserver agreement was calculated using the absolute agreement and Cohen’s kappa (␬).

The absolute agreement is the ratio of the findings in which agreement exists with the total findings. Co- hen’s ␬ is the ratio between chance-corrected ob- served agreement and chance-corrected perfect agreement. Interpretations of the␬ values were as follows:

⬍0.21 indicated poor; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, good; and⬎0.80 almost perfect agreement.²⁰Statistical analyses were performed using the Statistical Package for Social Sciences, for Windows, statistical package, version 12.1 (SPSS, Chi- cago, IL). The first data set of dichotomized findings was used to calculate the interobserver agreement between the 2 nuclear medicine physicians and between the 2 residents. The first and second data set of dichotomized findings of each observer were used to calculate the intraobserver agreement of all observers.

To analyze whether the experience of the observers influenced the agreement, the inter- and intraobserver agreement were compared using 95% confidence interval analysis.²¹ If no overlap in the 95%

confidence intervals was found, the differences in agreement were considered significant.

Influence of Tumor Localization and Tumor Size Determining the location of the primary tumor and any cervical metastases by the observers was studied to analyze the influence of tumor localization on the interobserver agreement between the nuclear medicine physicians. The influence of tumor size was also analyzed. The tumor size was determined from the pathologic findings when available or the radiologic results (CT, MRI, or US) when not available. Also, the effect of tumor size on the sensitivity of the nuclear medicine physicians’ findings was analyzed.

Sensitivity and Specificity

To examine whether the results in our study were valid, the sensitivity and specificity were calculated.

The first data set of dichotomized findings of the 4 observers was used to calculate the sensitivity and specificity of the ¹⁸F-FDG PET scan interpretations using the obtained histologic specimens, cytologic findings, or results from follow-up. To analyze the influence of the experience of the observers on the diagnostic properties, the sensitivity and specificity of the observers were compared using 95% confidence interval analysis.²¹If no overlap in the 95% intervals was found, the differences in sensitivity and specificity were considered significant.

Results

INTER- AND INTRAOBSERVER AGREEMENT

The results of the inter- and intraobserver agreement of the 4 observers for detecting the primary

(4)

tumor, cervical metastases per neck side, and distant malignancy are summarized inTable 2. The interobserver agreement between the nuclear medicine physicians was greater than the interobserver agreement between the residents, but no significant differences were found other than for detecting cervical metastases (Table 2). The intraobserver agreement of the nuclear medicine physicians was, in general, greater than the intraobserver agreement of the residents for detecting malignancy; however, only 1 significant difference was found (Table 2). Moreover, for all 4 observers, the intraobserver agreement was generally greater than the interobserver agreement. Two different PET cameras were used in the present study.

However, no difference in performance between the 2 cameras was found.

INFLUENCE OF TUMOR LOCALIZATION AND TUMOR SIZE

The interobserver agreement between the 2 nuclear medicine physicians decreased when more precise localization was required (Table 3). The ␬ and absolute agreement of the location of the primary tumor decreased when attempting to localize the primary tumor in the oral cavity or oropharynx. When attempting to localize the cervical metastases in the separate nodal levels in the neck, the␬ decreased for all levels. Only in level I did the interobserver agreement show a high ␬ (0.76). The absolute interobserver agreement remained high (ⱖ86%) when the level of localization was taken into account. For level V, no␬ could be calculated, because the metastases were only found in this level by 1 observer.

The influence of tumor size on interobserver agreement is presented in Table 4. In 2 patients, the primary tumor had already been completely removed before ¹⁸F-FDG PET scanning. In 7 of 78 scans, the nuclear medicine physicians did not agree on the presence of the primary tumor. In 6 of these 7 scans,

the primary tumor was classified as stage T1 with a maximal diameter of 20 mm and an invasion depth of 5 mm. In one scan, the primary tumor was classified as stage T2 with a maximal diameter of 31 mm and an invasion depth of 4 mm.

No influence from the size of the cervical metastases on interobserver agreement was found. Only in 1 of the 10 neck sides in which disagreement existed was a cervical metastasis present. This lymph node was 25 mm.

The sensitivities of the nuclear medicine physicians for the primary tumor and cervical metastases catego- rized by tumor size are also listed in Table 4. The sensitivity increased with increased tumor size.

SENSITIVITY AND SPECIFICITY

The sensitivity and specificity of the interpretations of the 4 observers are summarized in Table 5. The least experienced resident (resident II) had the lowest

Table 2. INTER- AND INTRAOBSERVER AGREEMENT OF FDG-PET INTERPRETATIONS

Variable

Interobserver Agreement Intraobserver Agreement NMPI and

NMPII RI and RII NMPI NMPII RI RII

p₀ ␬ p₀ ␬ p₀ ␬ p₀ ␬ p₀ ␬ p₀ ␬

Primary tumor 0.91 0.58 0.76 0.29 0.90 0.54 0.94 0.58 0.91 0.19 0.85 0.64 Cervical metastases 0.94 0.83* 0.86 0.54* 0.94 0.83 0.94 0.86 0.93 0.81 0.93 0.71 Distant metastases/second

primary tumor 0.85 0.53 0.78 0.26 0.95† 0.84 0.88 0.66 0.79† 0.52 0.94 0.42 Abbreviations: NMPI, nuclear medicine physician I; NMPII, nuclear medicine physician II; RI, resident I; RII, resident II; p0, absolute agreement;␬, Cohen’s kappa.

*Significant difference between␬ values of NMPs and residents.

†Significant difference between p0of NMPI and RI.

Krabbe et al. PET Observer Agreement in Head and Neck SCC. J Oral Maxillofac Surg 2010.

Table 3. INTEROBSERVER AGREEMENT BETWEEN NUCLEAR MEDICINE PHYSICIANS FOR

LOCALIZATION OF PRIMARY TUMOR AND CERVICAL METASTASES

Variable p₀ ␬

Primary tumor 0.91* 0.58*

Oral cavity 0.76 0.44

Oropharynx 0.75 0.21

Cervical metastases 0.94* 0.83*

Level I 0.98 0.76

Level II 0.89 0.51

Level III 0.86 ⫺0.06

Level IV 0.95 0.53

Level V 0.94 —

Abbreviations as inTable 2.

*For easy comparison, p₀ and ␬ values from Table 2 included.

Krabbe et al. PET Observer Agreement in Head and Neck SCC.

J Oral Maxillofac Surg 2010.

(5)

sensitivity for all 3 sites compared with the other 3 observers. Because of not adequately detecting malignancy, which involved fewer false-positive results, resident II scored better for specificity for cervical metastases and distant metastases/second primary tumor compared with the other observers.

Discussion

The present study has demonstrated high inter- and intraobserver agreement in SCC in the oral cavity or oropharynx with¹⁸F-FDG PET. The inter- and intraobserver agreement of the residents were, in general, less than the agreement between the nuclear medicine physicians, although their agreement was still fair to good (Table 2). Differences in experience between the 2 residents and the 2 nuclear medicine physicians did not result in the superiority of one compared to the other for intraobserver agreement, indicating that observer experience plays only a limited role in the reproducibility of interpreting¹⁸F-FDG PET scans. Also, for sensitivity and specificity, observer experience seemed to play a limited role. No superiority in the sensitivity and specificity between the 2 nuclear medicine physicians was found, despite their 10-year difference in experience. The sensitivity and specificity of the more experienced resident reached almost the level of the nuclear medicine physicians.

The present study was cross-sectional; therefore, we were not able to show a real learning curve in evaluating the¹⁸F-FDG PET scans. However, because the least experienced resident had lower sensitivity in interpreting the¹⁸F-FDG PET scans, our results sug- gest a short learning phase exists, after which evaluating¹⁸F-FDG PET scans reaches an acceptable level.

Other imaging techniques such as US-guided fine needle aspiration cytology, MRI, and CT are generally believed to be more experience and observer dependent.^22-24 That the interpretation of¹⁸F-FDG PET im-

ages in the complex anatomic head and neck region is not very experience and observer dependent is an asset to the technique.

Although the present study found high interobserver agreement for the 2 nuclear medicine physicians for detecting malignancies, the␬ values clearly decreased if more precise localization of the malignancies was required (Table 3). Only neck level I demonstrated high observer agreement, which can be explained by the easy recognition of this level on the scans. It was not surprising that the observer agreement decreased with more precise localization because of the lack of anatomic detail on¹⁸F-FDG PET scans. This finding supports the additional value of combining PET with CT (PET/CT) for proper tumor localization, as shown by Syed et al.²⁵¹⁸F-FDG PET/CT was superior to¹⁸F-FDG PET alone when comparing the interobserver agreement for precise anatomic localization of head and neck tumors.²⁵

Tumor size influenced the interobserver agreement and sensitivity of the nuclear medicine physicians (Table 4). The agreements increased with tumor size, with the exception of metastases smaller than 1 cm.

These small cervical metastases showed high interobserver agreement despite the very low sensitivity. The high interobserver agreement resulted from the nondetection of the small metastases by both observers.

Missing metastases 5 mm or smaller was not surprising against the background of the limited resolution of the PET camera.

One of the advantages of¹⁸F-FDG PET for the initial staging of head and neck cancer is the possibility of evaluating the whole body for malignancy. All distant metastases/second primary tumors were detected by both nuclear medicine physicians, except for one small superficial esophagus carcinoma, resulting in complete agreement for all second primary tumors and distant metastases. Disagreement, mostly for sus- pected malignancy in the lung or mediastinum, was present in 12 scans, all without proven second pri-

Table 4. ABSOLUTE INTEROBSERVER AGREEMENT AND SENSITIVITY OF NUCLEAR MEDICINE PHYSICIANS STRATIFIED BY TUMOR SIZE

Variable

Primary tumor stage T1 T2 T3-T4

Agreement NMPI-NMPII 60% (9/15) 95% (18/19) 100% (44/44)

Sensitivity NMPI 60% (8/15) 89% (17/19) 100% (44/44)

Sensitivity NMPII 67% (10/15) 95% (18/19) 100% (44/44)

Cervical metastasis size (mm) ⬍5 5 to⬍10 ⱖ10

Agreement NMPI-NMPII 100% (14/14) 100% (7/7) 96% (29/30)

Sensitivity NMPI 14% (2/14) 28% (2/7) 87% (27/30)

Sensitivity NMPII 14% (2/14) 28% (2/7) 93% (28/30)

Abbreviations as inTable 2.

Krabbe et al. PET Observer Agreement in Head and Neck SCC. J Oral Maxillofac Surg 2010.

(6)

mary tumors or distant metastases, highlighting the known false-positive risk of¹⁸F-FDG PET.²⁶

The present study had some limitations. The ␬ values should be interpreted with caution because use of the␬ does have a number of drawbacks.^27,28 Most notably for our study, the drawback was the influence of the distribution of malignancy. The ␬ values tend toward lower values when the distribution is asymmetric. In the present study, the presence of malignancy in the head and body was very asym- metrically distributed: 98% and 10%, respectively.

Thus, despite the high absolute agreement for detecting primary tumor and distant metastases, comparable to the agreement for detecting cervical metastases, the␬ values of the primary tumor and distant metastases were lower than those for cervical metastases.

The histologic findings of the surgical specimens were used to determine the tumor size. However, for some patients with malignancy, surgical specimens were not obtained. For these patients, the tumor size was determined by CT, MRI, or US performed at diagnosis of the malignancy. The measured diameter was used as the malignancy size. Thus, it is possible that the measurements for these malignancies were somewhat overestimated.

It could be argued that an analysis of PET data is superfluous in the PET/CT era. However, PET/CT is a combination of 2 imaging techniques, each with its own characteristics. To understand the added value of the combination, the value of each of the components should be known. The results of our study have revealed that the interpretation of PET data is relatively observer experience independent; however,¹⁸F-FDG PET is lacking for locating a tumor. As such, the present study provides a strong argument for the use of PET/CT in the evaluation of SCC of the head and neck.

In conclusion, the ¹⁸F-FDG PET images of SCC of the oral cavity or oropharynx showed good inter- and intraobserver agreement for detecting malignancy.

Observer experience played a limited role in observer agreement. Even in difficult areas as the head and neck, the images can be interpreted reliably for oral and oropharyngeal cancer. Observer agreement decreased, however, when more precise anatomic tumor localization was required. Observer agreement and sensitivity increased with tumor size. Small le- sions were missed by all observers, independent of experience, indicating that the role of¹⁸F-FDG PET in detecting small cervical metastases is limited.

References

1. Baatenburg de Jong RJ, Knegt P, Verwoerd CD: Reduction of the number of neck treatments in patients with head and neck cancer. Cancer 71:2312, 1993

Table5.SENSITIVITYANDSPECIFICITYOFFDG-PETINTERPRETATIONSFOREACHOBSERVER Variable

NMPI(%)NMPII(%)RI(%)RII(%) SensitivitySpecificitySensitivitySpecificitySensitivitySpecificitySensitivitySpecificity Primarytumor89%(69/78)100%(2/2)92%(72/78)100%(2/2)92%(72/78)50%(1/2)71%*(55/78)50%(1/2) Cervicalmetastases61%(31/51)97%(106/109)63%(32/51)91%†(99/109)61%(31/51)94%(103/109)47%(24/51)100%†(109/109) Distantmetastases/second primarytumor88%(7/8)88%(63/72)88%(7/8)86%(62/72)75%(6/8)78%‡(56/72)63%(5/8)99%‡(71/72) AbbreviationsasinTable2. *SignificantdifferenceinsensitivitybetweenRIIandotherobservers. †SignificantdifferenceinspecificitybetweenRIIandNMPII. ‡SignificantdifferenceinspecificitybetweenRIandRII. Krabbeetal.PETObserverAgreementinHeadandNeckSCC.JOralMaxillofacSurg2010.

(7)

2. Bootz F, Lenz M, Skalej M, et al: Computed tomography (CT) and magnetic resonance imaging (MRI) in T-stage evaluation of oral and oropharyngeal carcinomas. Clin Otolaryngol Allied Sci 17:421, 1992

3. Takes RP, Knegt P, Manni JJ, et al: Regional metastasis in head and neck squamous cell carcinoma: Revised value of US with US-guided FNAB. Radiology 198:819, 1996

4. Close LG, Merkel M, Vuitch MF, et al: Computed tomographic evaluation of regional lymph node involvement in cancer of the oral cavity and oropharynx. Head Neck 11:309, 1989 5. Stern WB, Silver CE, Zeifer BA, et al: Computed tomography of

the clinically negative neck. Head Neck 12:109, 1990 6. van den Brekel MW, Castelijns JA, Croll GA, et al: Magnetic

resonance imaging vs palpation of cervical lymph node metastasis. Arch Otolaryngol Head Neck Surg 117:663, 1991 7. Rege S, Maass A, Chaiken L, et al: Use of positron emission

tomography with fluorodeoxyglucose in patients with extracranial head and neck cancers. Cancer 73:3047, 1994 8. Goerres GW, Schmid DT, Gratz KW, et al: Impact of whole

body positron emission tomography on initial staging and therapy in patients with squamous cell carcinoma of the oral cavity.

Oral Oncol 39:547, 2003

9. Schmid DT, Stoeckli SJ, Bandhauer F, et al: Impact of positron emission tomography on the initial staging and therapy in locoregional advanced squamous cell carcinoma of the head and neck. Laryngoscope 113:888, 2003

10. Adams S, Baum RP, Stuckensen T, et al: Prospective comparison of 18F-FDG PET with conventional imaging modalities (CT, MRI, US) in lymph node staging of head and neck cancer. Eur J Nucl Med 25:1255, 1998

11. Jabour BA, Choi Y, Hoh CK, et al: Extracranial head and neck:

PET imaging with 2-[F-18]fluoro-2-deoxy-D-glucose and MR imaging correlation. Radiology 186:27, 1993

12. Di Martino E, Nowak B, Hassan HA, et al: Diagnosis and staging of head and neck cancer: A comparison of modern imaging modalities (positron emission tomography, computed tomography, color-coded duplex sonography) with panendoscopic and histopathologic findings. Arch Otolaryngol Head Neck Surg 126:1457, 2000

13. Kresnik E, Mikosch P, Gallowitsch HJ, et al: Evaluation of head and neck cancer with 18F-FDG PET: A comparison with conventional methods. Eur J Nucl Med 28:816, 2001

14. Kitagawa Y, Nishizawa S, Sano K, et al: Prospective comparison of 18F-FDG PET with conventional imaging modalities (MRI,

CT, and 67Ga scintigraphy) in assessment of combined intraarterial chemotherapy and radiotherapy for head and neck carcinoma. J Nucl Med 44:198, 2003

15. Dammann F, Horger M, Mueller-Berg M, et al: Rational diagnosis of squamous cell carcinoma of the head and neck region:

Comparative evaluation of CT, MRI, and 18FDG PET. AJR Am J Roentgenol 184:1326, 2005

16. Ng SH, Yen TC, Liao CT, et al: 18F-FDG PET and CT/MRI in oral cavity squamous cell carcinoma: A prospective study of 124 patients with histologic correlation. J Nucl Med 46:1136, 2005 17. Schwartz DL, Rajendran J, Yueh B, et al: Staging of head and neck squamous cell cancer with extended-field FDG-PET. Arch Otolaryngol Head Neck Surg 129:1173, 2003

18. McGuirt WF, Greven K, Williams D III, et al: PET scanning in head and neck oncology: A review. Head Neck 20:208, 1998 19. Quon A, Fischbein NJ, McDougall IR, et al: Clinical role of

18F-FDG PET/CT in the management of squamous cell carcinoma of the head and neck and thyroid carcinoma. J Nucl Med 48:58S, 2007 (Suppl 1)

20. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 33:159, 1977

21. Altman DG, Machin D, Bryant TN, et al: Statistics with Confi- dence: Confidence Intervals and Statistic Guidelines. Bristol, UK, BMJ Books, 2005

22. Hoorweg JJ, Kruijt RH, Heijboer RJ, et al: Reliability of interpretation of CT examination of the larynx in patients with glottic laryngeal carcinoma. Otolaryngol Head Neck Surg 135:

129, 2006

23. Keberle M, Kenn W, Muller H, et al: Interobserver variability in CT of oro- and hypopharyngeal carcinomas. Rofo 173:583, 2001

24. Wu M, Burstein DE, Yuan S, et al: A comparative study of 200 fine needle aspiration biopsies performed by clinicians and cytopathologists. Laryngoscope 116:1212, 2006

25. Syed R, Bomanji JB, Nagabhushan N, et al: Impact of combined (18)F-FDG PET/CT in head and neck tumours. Br J Cancer 92:1046, 2005

26. Strauss LG: Fluorine-18 deoxyglucose and false-positive results:

A major problem in the diagnostics of oncological patients. Eur J Nucl Med 23:1409, 1996

27. Cicchetti DV, Feinstein AR: High agreement but low kappa. II.

Resolving the paradoxes. J Clin Epidemiol 43:551, 1990 28. Feinstein AR, Cicchetti DV: High agreement but low kappa. I.

The problems of two paradoxes. J Clin Epidemiol 43:543, 1990