A framework for evaluating usability of clinical monitoring technology.

Technology design is a complex task, and acceptability is enhanced when usability is central to its design. Evaluating usability is a challenge for purchasers and developers of technology. We have developed a framework for testing the usability of clinical monitoring technology through literature review and experience designing clinical monitors.

Group differences in physician responses to handheld presentation of clinical evidence: a verbal protocol analysis.

Background: To identify individual differences in physicians' needs for the presentation of evidence resources and preferences for mobile devices.

Methods: Within-groups analysis of responses to semi-structured interviews. Interviews consisted of using prototypes in response to task-based scenarios.

Evidence in the palm of your hand: development of an outcomes-focused knowledge translation intervention.

Aim: The aim of the project was to develop an electronic information gathering and dissemination system to support both nursing-sensitive outcomes data collection and evidence-based decision-making at the point-of-patient care.

Background: With the current explosion of health-related knowledge, it is a challenge for nurses to regularly access information that is most current. The Internet provides timely access to health information, however, nurses do not readily use the Internet to access practice information because of being task-driven and coping with heavy workloads.

Using simulation methods to analyze and predict changes in workflow and potential problems in the use of a bar-coding medication order entry system.

We discuss a novel methodological approach to the analysis of a medication order entry system prior to system release. The approach involved use of realistic scenarios (where physicians and nurses interacted with a system and dummy patient) where the sessions were video recorded in their entirety. The data were analyzed using a qualitative coding scheme for identifying usability problems and changes in workflow.

The Virtual Usability Laboratory: its application in evaluation and analysis of use of Web-based health information resources and simulations.

The evaluation of the effectiveness, usage and usability of Web-based systems has become a major issue. In this presentation we describe our experiences in developing and testing a system known as the VULab ("Virtual Usability Laboratory"). The objective of our work was to develop a system to support the collection, integration and analysis of a range of usage data.

Use of simulation in the study of clinician workflow.

This paper describes a methodological framework for conducting evaluations of clinical workflow and system impact based on simulated user interactions. The approach involves collection of a rich set of data consisting of audio and video recordings of interactions of healthcare workers with health care information systems and their associated devices. Methodological considerations and issues in conducting such studies are discussed.

Predicting changes in workflow resulting from healthcare information systems: ensuring the safety of healthcare.

This paper describes an approach to studying medical error and workflow that can be applied to help ensure the safety of new healthcare information systems. The approach focuses on identification of usability problems resulting from implementation of new information technology, as well as identification of problems related to changes in workflow. The paper illustrates how the approach can be applied in the simulation-based analyses of emerging healthcare information systems.

Low-cost rapid usability engineering: designing and customizing usable healthcare information systems.

It is essential that the healthcare systems we develop are usable, meet user information needs and are safe. To ensure system usability, a variety of methods have emerged from the area of usability engineering and have been adapted to healthcare. The authors have been applying methods of usability engineering, working with hospitals and companies to develop more usable healthcare information systems for over 15 years.

Identifying and preventing technology-induced error using simulations: application of usability engineering techniques.

In this paper, we describe a framework for the analysis of technology-induced errors, extending approaches from the emerging area of usability engineering. The approach involves collection of a rich set of data consisting of audio and video recordings of interactions of healthcare workers with health information systems under simulated conditions. The application of the approach is discussed, along with methodological considerations and issues in conducting such studies.

A framework for capturing the interactions between laypersons' understanding of disease, information gathering behaviors, and actions taken during an epidemic.

This paper provides a description of a methodological framework designed to capture the inter-relationships between the lay publics' understanding of health-related processes, information gathering behaviors, and actions taken during an outbreak. We developed and refined our methods during a study involving eight participants living in severe acute respiratory syndrome (SARS)-affected areas (Hong Kong, Taiwan, and Toronto). The framework is an adaptation of narrative analysis, a qualitative method that is used to investigate a phenomenon through interpretation of the stories people tell about their experiences.

Technology induced error and usability: the relationship between usability problems and prescription errors when using a handheld application.

This paper describes an innovative approach to the evaluation of a handheld prescription writing application. Participants (10 physicians) were asked to perform a series of tasks involving entering prescriptions into the application from a medication list. The study procedure involved the collection of data consisting of transcripts of the subjects who were asked to "think aloud" while interacting with the prescription writing program to enter medications.

The relationship of usability to medical error: an evaluation of errors associated with usability problems in the use of a handheld application for prescribing medications.

This paper describes an innovative approach to the evaluation of a handheld prescription writing application. Participants (10 physicians) were asked to perform a series of tasks involving entering prescriptions into the application from a medication list. The study procedure involved the collection of data consisting of transcripts of the subjects who were asked to "think aloud" while interacting with the prescription writing program to enter medications.

Cognitive and usability engineering methods for the evaluation of clinical information systems.

Increasingly healthcare policy and decision makers are demanding evidence to justify investments in health information systems. This demand requires an adequate evaluation of these systems. A wide variety of approaches and methodologies have been applied in assessing the impact of information systems in health care, ranging from controlled clinical trials to use of questionnaires and interviews with users.

From prototype to production system: lessons learned from the evolution of the SignOut System at Mount Sinai Medical Center.

Approaches to the development of information systems in large health care institutions range from prototyping to conventional development of large scale production systems. This paper discusses the development of the SignOut System at Mount Sinai Medical Center, which was designed in 1997 to capture vital resident information. Local need quickly outstripped proposed delays for building a production system and a prototype system quickly became a production system.

The patient clinical information system (PatCIS): technical solutions for and experience with giving patients access to their electronic medical records.

As health records evolve into electronic form, increasing demand is being made to provide patients with access to them. We sought to study the character and impact of such access to determine how patients use such records, what cognitive effects it has on them, and how it affects their relationship with their health care providers. We created the Patient Clinical Information System (PatCIS) to interface with the clinical data repository at New York Presbyterian Hospital (NYPH) to allow patients to add to and review their medical data.

Usability evaluation of an experimental text summarization system and three search engines: implications for the reengineering of health care interfaces.

This paper describes the comparative evaluation of an experimental automated text summarization system, Centrifuser and three conventional search engines - Google, Yahoo and About.com. Centrifuser provides information to patients and families relevant to their questions about specific health conditions.

Patients' and physicians' understanding of health and biomedical concepts: relationship to the design of EMR systems.

Objective: The aim of this paper is to examine knowledge organization and reasoning strategies involved in physician-patient communication and to consider how these are affected by the use of computer tools, in particular, electronic medical record (EMR) systems.

Design: In the first part of the paper, we summarize results from a study in which patients were interviewed before their interactions with physicians and where physician-patient interactions were recorded and analyzed to evaluate patients' and physicians' understanding of the patient problem. We give a detailed presentation of one of such interaction, with characterizations of physician and patient models.

Evaluation in the design of health information systems: application of approaches emerging from usability engineering.

This paper examines the role of evaluation in the design of health care information systems. A framework is presented for considering evaluation in the context of software development processes, in particular, the systems development life cycle (SDLC). Variations on standard design methodologies are then discussed, including methods based on rapid development and continual evaluation of prototype systems.