Evaluating the effectiveness of four contextual features in classifying annotated clinical conditions in emergency department reports.

Objective: Determine how four contextual features (Validity, Certainty, Directionality, and Temporality) contribute to classification of respiratory syndrome-related clinical conditions as acute, chronic, or absent from manual annotations in Emergency Department Reports. Based on the results, we will direct our research towards automatic identification of the contextual features found to be discriminating.

Methods: A physician annotated all instances of 56 clinical conditions in 120 ED reports and encoded four contextual features for every annotation. We classified clinical conditions using the contextual features and measured agreement to reference standard classifications made by the physician using a weighted kappa (Kw).

Results: Kw was 0.518 when not using any of the features and 0.953 when using all of the features.

Conclusion: Validity, Directionality, and Temporality all improved accuracy. Negation(Directionality) was the most important feature for improving accuracy. Using Certainty made the classification worse.