Using chief complaints for syndromic surveillance: a review of chief complaint based classifiers in North America.

A major goal of Natural Language Processing in the public health informatics domain is the automatic extraction and encoding of data stored in free text patient records. This extracted data can then be utilized by computerized systems to perform syndromic surveillance. In particular, the chief complaint--a short string that describes a patient's symptoms--has come to be a vital resource for syndromic surveillance in the North American context due to its near ubiquity.

Developing syndrome definitions based on consensus and current use.

Objective: Standardized surveillance syndromes do not exist but would facilitate sharing data among surveillance systems and comparing the accuracy of existing systems. The objective of this study was to create reference syndrome definitions from a consensus of investigators who currently have or are building syndromic surveillance systems.

Design: Clinical condition-syndrome pairs were catalogued for 10 surveillance systems across the United States and the representatives of these systems were brought together for a workshop to discuss consensus syndrome definitions.

ConText: an algorithm for determining negation, experiencer, and temporal status from clinical reports.

In this paper we describe an algorithm called ConText for determining whether clinical conditions mentioned in clinical reports are negated, hypothetical, historical, or experienced by someone other than the patient. The algorithm infers the status of a condition with regard to these properties from simple lexical clues occurring in the context of the condition. The discussion and evaluation of the algorithm presented in this paper address the questions of whether a simple surface-based approach which has been shown to work well for negation can be successfully transferred to other contextual properties of clinical conditions, and to what extent this approach is portable among different clinical report types.

Estimating the joint disease outbreak-detection time when an automated biosurveillance system is augmenting traditional clinical case finding.

The goals of automated biosurveillance systems are to detect disease outbreaks early, while exhibiting few false positives. Evaluation measures currently exist to estimate the expected detection time of biosurveillance systems. Researchers also have developed models that estimate clinician detection of cases of outbreak diseases, which is a process known as clinical case finding.

Evaluation of preprocessing techniques for chief complaint classification.

Objective: To determine whether preprocessing chief complaints before automatically classifying them into syndromic categories improves classification performance.

Methods: We preprocessed chief complaints using two preprocessors (CCP and EMT-P) and evaluated whether classification performance increased for a probabilistic classifier (CoCo) or for a keyword-based classifier (modification of the NYC Department of Health and Mental Hygiene chief complaint coder (KC)).

Results: CCP exhibited high accuracy (85%) in preprocessing chief complaints but only slightly improved CoCo's classification performance for a few syndromes.

Evaluation of training with an annotation schema for manual annotation of clinical conditions from emergency department reports.

Objective: Determine whether agreement among annotators improves after being trained to use an annotation schema that specifies: what types of clinical conditions to annotate, the linguistic form of the annotations, and which modifiers to include.

Methods: Three physicians and 3 lay people individually annotated all clinical conditions in 23 emergency department reports. For annotations made using a Baseline Schema and annotations made after training on a detailed annotation schema, we compared: (1) variability of annotation length and number and (2) annotator agreement, using the F-measure.

Evaluating the effectiveness of four contextual features in classifying annotated clinical conditions in emergency department reports.

Objective: Determine how four contextual features (Validity, Certainty, Directionality, and Temporality) contribute to classification of respiratory syndrome-related clinical conditions as acute, chronic, or absent from manual annotations in Emergency Department Reports. Based on the results, we will direct our research towards automatic identification of the contextual features found to be discriminating.

Methods: A physician annotated all instances of 56 clinical conditions in 120 ED reports and encoded four contextual features for every annotation.

Classification of emergency department chief complaints into 7 syndromes: a retrospective analysis of 527,228 patients.

Study Objective: Electronic surveillance systems often monitor triage chief complaints in hopes of detecting an outbreak earlier than can be accomplished with traditional reporting methods. We measured the accuracy of a Bayesian chief complaint classifier called CoCo that assigns patients 1 of 7 syndromic categories (respiratory, botulinic, gastrointestinal, neurologic, rash, constitutional, or hemorrhagic) based on free-text triage chief complaints.

Methods: We compared CoCo's classifications with criterion syndromic classification based on International Classification of Diseases, Ninth Revision (ICD-9) discharge diagnoses.

Inductive creation of an annotation schema for manually indexing clinical conditions from emergency department reports.

Evaluating automated indexing applications requires comparing automatically indexed terms against manual reference standard annotations. However, there are no standard guidelines for determining which words from a textual document to include in manual annotations, and the vague task can result in substantial variation among manual indexers. We applied grounded theory to emergency department reports to create an annotation schema representing syntactic and semantic variables that could be annotated when indexing clinical conditions.

Generating a reliable reference standard set for syndromic case classification.

Objective: To generate and measure the reliability for a reference standard set with representative cases from seven broad syndromic case definitions and several narrower syndromic definitions used for biosurveillance.

Design: From 527,228 eligible patients between 1990 and 2003, we generated a set of patients potentially positive for seven syndromes by classifying all eligible patients according to their ICD-9 primary discharge diagnoses. We selected a representative subset of the cases for chart review by physicians, who read emergency department reports and assigned values to 14 variables related to the seven syndromes.

Classifying free-text triage chief complaints into syndromic categories with natural language processing.

Objective: Develop and evaluate a natural language processing application for classifying chief complaints into syndromic categories for syndromic surveillance.

Introduction: Much of the input data for artificial intelligence applications in the medical field are free-text patient medical records, including dictated medical reports and triage chief complaints. To be useful for automated systems, the free-text must be translated into encoded form.

Identifying respiratory findings in emergency department reports for biosurveillance using MetaMap.

Clinical conditions described in patients' dictated reports are necessary for automated detection of patients with respiratory illnesses such as inhalational anthrax and pneumonia. We applied MetaMap to emergency department reports to extract a set of 71 clinical conditions relevant to detection of a lower respiratory outbreak. We indexed UMLS terms in emergency department reports with MetaMap, filtered the indexed output with a specialized lexicon of UMLS terms for the domain, and mapped the clinical conditions of interest to concepts in the lexicon.

Fever detection from free-text clinical records for biosurveillance.

Automatic detection of cases of febrile illness may have potential for early detection of outbreaks of infectious disease either by identification of anomalous numbers of febrile illness or in concert with other information in diagnosing specific syndromes, such as febrile respiratory syndrome. At most institutions, febrile information is contained only in free-text clinical records. We compared the sensitivity and specificity of three fever detection algorithms for detecting fever from free-text.

Representative threats for research in public health surveillance.

A large number of biological agents can cause natural or bioterroristic disease outbreaks and each can present in a bewildering number of ways (e.g., a few cases versus many cases, confined to a building versus widely disseminated).