Impact of exclusion criteria on case mix, outcome, and length of stay for the severity of disease scoring methods in common use in critical care.

Objective: To identify the exclusion criteria for the major severity of disease scoring methods in critical care and to investigate the impact of the exclusion criteria on the case mix, outcomes and length of stay for admissions to intensive care units (ICUs) in England, Wales and Northern Ireland.

Design: Cohort study-analysis of prospectively collected data.

Setting: 127 adult, general (mixed medical/surgical) ICUs in England, Wales and Northern Ireland.

Patients: 120,503 admissions between 1995 and 2001.

Interventions: None.

Measurements And Main Results: Thirteen different exclusion criteria were identified from the original methodological/validation papers and data collection manuals for APACHE II, APACHE III, SAPS II and MPM II. Application of the original exclusion criteria for the four, major severity of disease scoring methods resulted in exclusion of between 11.5% and 14.6% of admissions. Hospital mortality for the overall cohort was 29.0% but ranged from 4.7% to 50.1% among those groups excluded. After application of the exclusion criteria for each scoring method, there was little difference in overall hospital mortality or median ICU and hospital length of stay for the included admissions when compared with the original cohort. At the level of individual ICUs, there were differences in hospital mortality before and after exclusions-minimum -3.1% to maximum 9.5% (APACHE II), minimum -2.8% to maximum 9.4% (APACHE III), minimum -3.1% to maximum 16.1% (SAPS II) and minimum -3.1% to maximum 16.5% (MPM II). The mean difference across individual ICUs was -0.5 % (95% CI -0.7% to -0.2%) for APACHE II, -0.2% (95% CI -0.2% to 0.1%) for APACHE III, 2.0% (95% CI 1.7% to 2.4%) for SAPS II and 2.1% (95% CI 1.7% to 2.5%) for MPM II. SAPS II and MPM II showed systematic variation. A survey of the literature found wide variation in the exclusion criteria reported in subsequent, published research using a single severity of disease scoring method (APACHE II).

Conclusions: Exclusion criteria used in critical care research are often ill-defined and poorly reported. More attention to the choice of exclusion criteria and their effect on the reported results is essential. We hope this study will raise the need for both better reporting of exclusion criteria applied in studies and promote the need for a common set of explicit exclusion criteria for these methods.