Elicitation and aggregation of multimodal estimates improve wisdom of crowd effects on ordering tasks.

We present a wisdom of crowds study where participants are asked to order a small set of images based on the number of dots they contain and then to guess the respective number of dots in each image. We test two input elicitation interfaces-one elicits the two modalities of estimates jointly and the other independently. We show that the latter interface yields higher quality estimates, even though the multimodal estimates tend to be more self-contradictory.

A review of mathematical models of human trust in automation.

Understanding how people trust autonomous systems is crucial to achieving better performance and safety in human-autonomy teaming. Trust in automation is a rich and complex process that has given rise to numerous measures and approaches aimed at comprehending and examining it. Although researchers have been developing models for understanding the dynamics of trust in automation for several decades, these models are primarily conceptual and often involve components that are difficult to measure.

Measuring trust: a text analysis approach to compare, contrast, and select trust questionnaires.

Introduction: Trust has emerged as a prevalent construct to describe relationships between people and between people and technology in myriad domains. Across disciplines, researchers have relied on many different questionnaires to measure trust. The degree to which these questionnaires differ has not been systematically explored.

Evaluating the Impact of Uncertainty Visualization on Model Reliance.

Machine learning models have gained traction as decision support tools for tasks that require processing copious amounts of data. However, to achieve the primary benefits of automating this part of decision-making, people must be able to trust the machine learning model's outputs. In order to enhance people's trust and promote appropriate reliance on the model, visualization techniques such as interactive model steering, performance analysis, model comparison, and uncertainty visualization have been proposed.

Trusting Automation: Designing for Responsivity and Resilience.

Objective: This paper reviews recent articles related to human trust in automation to guide research and design for increasingly capable automation in complex work environments.

Background: Two recent trends-the development of increasingly capable automation and the flattening of organizational hierarchies-suggest a reframing of trust in automation is needed.

Method: Many publications related to human trust and human-automation interaction were integrated in this narrative literature review.

Bridging psychology and engineering to make technology work for people.

Engineering grand challenges and big ideas not only demand innovative engineering solutions, but also typically involve and affect human thought, behavior, and quality of life. To solve these types of complex problems, multidisciplinary teams must bring together experts in engineering and psychological science, yet fusing these distinct areas can be difficult. This article describes how Human Systems Engineering (HSE) researchers have confronted such challenges at the interface of humans and technological systems.

Cooperation in Human-Agent Systems to Support Resilience: A Microworld Experiment.

Objective: This study uses a dyadic approach to understand human-agent cooperation and system resilience.

Background: Increasingly capable technology fundamentally changes human-machine relationships. Rather than reliance on or compliance with more or less reliable automation, we investigate interaction strategies with more or less cooperative agents.

Quantitative ethnographic study of physician workflow and interactions with electronic health record systems.

This study explores the relationship between primary care physicians' interactions with health information technology and primary care workflow. Clinical encounters were recorded with high-resolution video cameras to capture physicians' workflow and interaction with two objects of interest, the electronic health record (EHR) system, and their patient. To analyze the data, a coding scheme was developed based on a validated list of primary care tasks to define the presence or absence of a task, the time spent on each task, and the sequence of tasks.

Identifying design considerations for a shared decision aid for use at the point of outpatient clinical care: An ethnographic study at an inner city clinic.

Background And Objective: Computerized decision aids could facilitate shared decision-making at the point of outpatient clinical care. The objective of this study was to investigate whether a computerized shared decision aid would be feasible to implement in an inner-city clinic by evaluating the current practices in shared decision-making, clinicians' use of computers, patient and clinicians' attitudes and beliefs toward computerized decision aids, and the influence of time on shared decision-making.

Methods: Qualitative data analysis of observations and semi-structured interviews with patients and clinicians at an inner-city outpatient clinic.

Organizational and technological correlates of nurses' trust in a smart intravenous pump.

The aim of this study was to understand technology and system characteristics that contribute to nurses' ratings of trust in a smart intravenous pump. Nurses' trust in new technologies can influence how technologies are used. Trust in technology is defined as a person's belief that a technology will not fail them.