| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Correspondence |
Toronto, Canada
To the Editor:
Naguib et al., further elucidated the relationship of airway measures with the risk of difficult intubation.1 Their clinical model, however, may not be applicable for use in clinical practice.
First, "all known" clinical risk criteria were not included in the analysis. For example, atlanto-occipital extension was not included as a predictor measure -only head and neck movement was included as part of the Wilson risk sum score, which is unreliable. Other important measures may have altered their model, such as that described by Bellhouse et al.2 All potentially important variables should be assessed to ensure that the prediction model is reliable and valid.
Second, their models were developed on only 25 patients who were difficult to intubate. Up to ten clinical measures were included in their multivariable analysis: for every predictor variable, there were only 2.5 patients with difficult intubation. However, to ensure that multivariable models do not suffer from over-fitting, where they perform well only on the sample from which they were derived, the outcome: variable ratio should be greater than 5:1.3
Therefore, the model is most likely over-fitted and may not be applicable to other populations. There are at least two methods of validation to determine if the model suffers from over-fitting: internal validation such as bootstrapping, and external validation on a new group of patients.4,5 The authors did not validate their model. Their statement that the model's high ROC area implies reproducibility is false: in this setting, ROC area simply measures the accuracy of the model when it is reapplied to the sample population; it has no relation to reproducibility or external validity.6
References
1
Naguib M, Malabarey T, AlSatli RA, Damegh SA, Samarkandi AH. Predictive models for difficult laryngoscopy and intubation. A clinical, radiologic and three-dimensional computer imaging study. Can J Anesth 1999; 46: 748–59.
2 Bellhouse CP, Dore C. Criteria of estimating likelihood of difficulty of endotracheal intubation with the Macintosh laryngoscope. Anaesth Intensive Care 1988; 16: 329–37.[Medline]
3 Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing error. Stat Med 1996; 15: 361–87.[Medline]
4 Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules. Applications and methodological standards. N Engl J Med 1985; 313: 793–9.[Abstract]
5 Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA 1997; 277: 488–94.[Abstract]
6 Feinstein AR. Multiple logistic regression. In: Feinstein AR (Ed.). Multivariable Analysis: An Introduction. New Haven: Yale University Press, 1996: 297–330.
Iowa City, Iowa, Usa
We appreciate Dr. Karkouti's interest in our study.1 Dr Karkouti did not provide any evidence to support his contention that "our clinical model may not be applicable for use in clinical practice."
We suspect that the head and neck movement (as part of the Wilson risk sum score) and the radiological measurements (with the head erect and mouth closed, and with the head fully extended on the neck and the mouth fully) represent atlanto-occipital extension. We addressed in our paper the reliability of the Wilson risk sum score. We included all of the radiological measurements described by Bellhouse and Doré.2 Therefore, we do not follow Dr. Karkouti's statement regarding Bellhouse's study.
Of importance, our model was validated with bootstrapping. Our model was developed on data from 65 patients and not only from the 25 patients who were difficult to intubate. Dr. Karkouti's comments on the ROC curve are incorrect. For detailed discussion on the area under ROC curve, see Hanley and McNeil.3 We are currently conducting a clinical study on a different population of patients to evaluate whether the scale applies.
References
1 Naguib M, Malabarey T, AlSatli RA, Al Damegh S, Samarkandi AH. Predictive models for difficult laryngoscopy and intubation. A clinical, radiologic and three-dimensional computer imaging study. Can J Anesth 1999; 46: 748–59.
2 Bellhouse CP, Doré C. Criteria for estimating likelihood of difficulty of endotracheal intubation with the Macintosh laryngoscope. Anaesth Intensive Care 1988; 61: 329–37.
3
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29–36.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
