Artificial intelligence-generated feedback on social signals in patient-provider communication: technical performance, feedback usability, and impact.

Objectives: Implicit bias perpetuates health care inequities and manifests in patient-provider interactions, particularly nonverbal social cues like dominance. We investigated the use of artificial intelligence (AI) for automated communication assessment and feedback during primary care visits to raise clinician awareness of bias in patient interactions.

Materials And Methods: (1) Assessed the technical performance of our AI models by building a machine-learning pipeline that automatically detects social signals in patient-provider interactions from 145 primary care visits.

Screen or No Screen? Lessons Learnt from a Real-World Deployment Study of Using Voice Assistants With and Without Touchscreen for Older Adults.

While voice user interfaces offer increased accessibility due to hands-free and eyes-free interactions, older adults often have challenges such as constructing structured requests and perceiving how such devices operate. Voice-first user interfaces have the potential to address these challenges by enabling multimodal interactions. Standalone voice + touchscreen Voice Assistants (VAs), such as Echo Show, are specific types of devices that adopt such interfaces and are gaining popularity.

Holo-Stroke: Assessing for Immersive Stroke Care Through Stroke Hologram Teleportation.

Augmented reality enables the wearer to see both their physical environment and virtual objects. Holograms could allow 3D video of providers to be transmitted to distant sites, allowing patients to interact with virtual providers as if they are in the same physical space. Our aim was to determine if Tele-Stroke augmented with Holo-Stroke, compared with Tele-Stroke alone, could improve satisfaction and perception of immersion for the patient.

Designing Communication Feedback Systems To Reduce Healthcare Providers' Implicit Biases In Patient Encounters.

Healthcare providers' implicit bias, based on patients' physical characteristics and perceived identities, negatively impacts healthcare access, care quality, and outcomes. Feedback tools are needed to help providers identify and learn from their biases. To incorporate providers' perspectives on the most effective ways to present such feedback, we conducted semi-structured design critique sessions with 24 primary care providers.

ConverSense: An Automated Approach to Assess Patient-Provider Interactions using Social Signals.

Patient-provider communication influences patient health outcomes, and analyzing such communication could help providers identify opportunities for improvement, leading to better care. Interpersonal communication can be assessed through "social-signals" expressed in non-verbal, vocal behaviors like interruptions, turn-taking, and pitch. To automate this assessment, we introduce a machine-learning pipeline that ingests audio-streams of conversations and tracks the magnitude of four social-signals: dominance, interactivity, engagement, and warmth.

Imagining Improved Interactions: Patients' Designs To Address Implicit Bias.

Implicit biases may negatively influence healthcare providers' behaviors toward patients from historically marginalized communities, impacting providers' communication style, clinical decision-making, and delivery of quality care. Existing interventions to mitigate negative experiences of implicit biases are primarily designed to increase recognition and management of stereotypes and prejudices through provider-facing tools and resources. However, there is a gap in understanding and designing interventions from patient perspectives.

Ascertaining Whether an Intelligent Voice Assistant Can Meet Older Adults' Health-Related Needs in the Context of a Geriatrics 5Ms Framework.

Unlabelled: The Geriatrics 5Ms: Medications, Mind, Mobility, what Matters most and Multicomplexity is a framework to address the complex needs of older adults. Intelligent Voice Assistants (IVAs) are increasingly popular and have potential to support health-related needs of older adults. We utilized previously collected qualitative data on older adults' views of how an IVA may address their health-related needs and ascertained their fit into the Geriatrics 5Ms framework.

How do Older Adults Set Up Voice Assistants? Lessons Learned from a Deployment Experience for Older Adults to Set Up Standalone Voice Assistants.

While standalone Voice Assistants (VAs) are promising to support older adults' daily routine and wellbeing management, onboarding and setting up these devices can be challenging. Although some older adults choose to seek assistance from technicians and adult children, easy set up processes that facilitate independent use are still critical, especially for those who do not have access to external resources. We aim to understand the older adults' experience while setting up commercially available voice-only and voice-first screen-based VAs.

Towards Visualization of Time-Series Ecological Momentary Assessment (EMA) Data on Standalone Voice-First Virtual Assistants.

Population aging is an increasingly important consideration for health care in the 21th century, and continuing to have access and interact with digital health information is a key challenge for aging populations. Voice-based Intelligent Virtual Assistants (IVAs) are promising to improve the Quality of Life (QoL) of older adults, and coupled with Ecological Momentary Assessments (EMA) they can be effective to collect important health information from older adults, especially when it comes to repeated time-based events. However, this same EMA data is hard to access for the older adult: although the newest IVAs are equipped with a display, the effectiveness of visualizing time-series based EMA data on standalone IVAs has not been explored.

Mixed reality surgical mentoring of combat casualty care related procedures in a perfused cadaver model: Initial results of a randomized feasibility study.

Background: Most telemedicine modalities have limited ability to enhance procedural and operative care. We developed a novel system to provide synchronous bidirectional expert mixed reality-enabled virtual procedural mentoring. In this feasibility study, we evaluated mixed reality mentoring of combat casualty care related procedures in a re-perfused cadaver model.

Maybe they had a bad day: how LGBTQ and BIPOC patients react to bias in healthcare and struggle to speak out.

Objective: People who experience marginalization, including Black, Indigenous, People of Color (BIPOC) and Lesbian, Gay, Bisexual, Transgender, Queer, Plus (ie, all other marginalized genders and sexual orientations) people (LGBTQ+) experience discrimination during healthcare interactions, which negatively impacts patient-provider communication and care. Yet, scarce research examines the lived experience of unfair treatment among patients from marginalized groups to guide patient-centered tools that improve healthcare equity.

Materials And Methods: We interviewed 25 BIPOC and/or LGBTQ+ people about their experiences of unfair treatment and discrimination when visiting healthcare providers.

Battling Bias in Primary Care Encounters: Informatics Designs to Support Clinicians.

Although clinical training in implicit bias is essential for healthcare equity, major gaps remain both for effective educational strategies and for tools to help identify implicit bias. To understand the perspectives of clinicians on the design of these needed strategies and tools, we conducted 21 semi-structured interviews with primary care clinicians about their perspectives and design recommendations for tools to improve patient-centered communication and to help mitigate implicit bias. Participants generated three types of solutions to improve communication and raise awareness of implicit bias: digital nudges, guided reflection, and data-driven feedback.

Broken down by bias: Healthcare biases experienced by BIPOC and LGBTQ+ patients.

Bias toward historically marginalized patients affects patient-provider interactions and can lead to lower quality of care and poor health outcomes for patients who are Black, Indigenous, People of Color (BIPOC) and Lesbian, Gay, Bisexual, Transgender and Gender Diverse (LGBTQ+). We gathered experiences with biased healthcare interactions and suggested solutions from 25 BIPOC and LGBTQ+ people. Through qualitative thematic analysis of interviews, we identified ten themes.

Social Mobile Approaches to Reducing Weight (SMART) 2.0: protocol of a randomized controlled trial among young adults in university settings.

Background: Excess weight gain in young adulthood is associated with future weight gain and increased risk of chronic disease. Although multimodal, technology-based weight-loss interventions have the potential to promote weight loss among young adults, many interventions have limited personalization, and few have been deployed and evaluated for longer than a year. We aim to assess the effects of a highly personalized, 2-year intervention that uses popular mobile and social technologies to promote weight loss among young adults.

Understanding Barriers and Design Opportunities to Improve Healthcare and QOL for Older Adults through Voice Assistants.

Voice-based Intelligent Virtual Assistants (IVAs) promise to improve healthcare management and Quality of Life (QOL) by introducing the paradigm of hands-free and eye-free interactions. However, there has been little understanding regarding the challenges for designing such systems for older adults, especially when it comes to healthcare related tasks. To tackle this, we consider the processes of care delivery and QOL enhancements for older adults as a collaborative task between patients and providers.

Toward a Unified Metadata Schema for Ecological Momentary Assessment with Voice-First Virtual Assistants.

Ecological momentary assessment (EMA) is used to evaluate subjects' behaviors and moods in their natural environments, yet collecting real-time and self-report data with EMA is challenging due to user burden. Integrating voice into EMA data collection platforms through today's intelligent virtual assistants (IVAs) is promising due to hands-free and eye-free nature. However, efficiently managing conversations and EMAs is non-trivial and time consuming due to the ambiguity of the voice input.

Grindr Users Take More Risks, but Are More Open to Human Immunodeficiency Virus (HIV) Pre-exposure Prophylaxis: Could This Dating App Provide a Platform for HIV Prevention Outreach?

Background: Technology has changed the way that men who have sex with men (MSM) seek sex. More than 60% of MSM in the United States use the internet and/or smartphone-based geospatial networking apps to find sex partners. We correlated use of the most popular app (Grindr) with sexual risk and prevention behavior among MSM.

SMART 2.0: A Multimodal Weight Loss Intervention for Young Adults.

A significant number of young Americans are vulnerable to excess weight gain, especially during the college years. While technology-based weight loss interventions have the potential to be very engaging, short-term approaches showed limited success. In our work we aim to better understand the impact of long-term, multimodal, technology-based weight loss interventions, and study their potential for greater effect among college students.

Using Participatory Design to Inform the Connected and Open Research Ethics (CORE) Commons.

Mobile health (mHealth) research involving pervasive sensors, mobile apps and other novel data collection tools and methods present new ethical, legal, and social challenges specific to informed consent, data management and bystander rights. To address these challenges, a participatory design approach was deployed whereby stakeholders contributed to the development of a web-based commons to support the mHealth research community including researchers and ethics board members. The CORE (Connected and Open Research Ethics) platform now features a community forum, a resource library and a network of nearly 600 global members.

Embedded Merge & Split: Visual Adjustment of Data Grouping.

Data grouping is among the most frequently used operations in data visualization. It is the process through which relevant information is gathered, simplified, and expressed in summary form. Many popular visualization tools support automatic grouping of data (e.

NIH support of mobile, imaging, pervasive sensing, social media and location tracking (MISST) research: laying the foundation to examine research ethics in the digital age.

Mobile Imaging, pervasive Sensing, Social media and location Tracking (MISST) tools used in research are raising new ethical challenges for scientists and the Institutional Review Boards (IRBs) charged with protecting human participants. Yet, little guidance exists to inform the ethical design and the IRB's regulatory review of MISST research. MISST tools/methods produce personal health data that is voluminous and granular and, which may not be subject to policies like the Health Information Portability and Accessibility Act (HIPAA).

Keystrokes, Mouse Clicks, and Gazing at the Computer: How Physician Interaction with the EHR Affects Patient Participation.

Background: Evidence is mixed regarding how physicians' use of the electronic health record (EHR) affects communication in medical encounters.

Objective: To investigate whether the different ways physicians interact with the computer (mouse clicks, key strokes, and gaze) vary in their effects on patient participation in the consultation, physicians' efforts to facilitate patient involvement, and silence.

Design: Cross-sectional, observational study of video and event recordings of primary care and specialty consultations.

Ethical and regulatory challenges of research using pervasive sensing and other emerging technologies: IRB perspectives.

Vast quantities of personal health information and private identifiable information are being created through mobile apps, wearable sensors, and social networks. While new strategies and tools for obtaining health data have expanded researchers' abilities to design and test personalized and adaptive health interventions, the deployment of pervasive sensing and computational techniques to gather research data is raising ethical challenges for Institutional Review Boards (IRBs) charged with protecting research participants. To explore experiences with, and perceptions about, technology-enabled research, and identify solutions for promoting responsible conduct of this research we conducted focus groups with human research protection program and IRB affiliates.

Physician activity during outpatient visits and subjective workload.

We describe methods for capturing and analyzing EHR use and clinical workflow of physicians during outpatient encounters and relating activity to physicians' self-reported workload. We collected temporally-resolved activity data including audio, video, EHR activity, and eye-gaze along with post-visit assessments of workload. These data are then analyzed through a combination of manual content analysis and computational techniques to temporally align streams, providing a range of process measures of EHR usage, clinical workflow, and physician-patient communication.

Face-to-Face and Online Networks: College Students' Experiences in a Weight-Loss Trial.

This study aimed to understand how college students participating in a 2-year randomized controlled trial (Project SMART: Social and Mobile Approach to Reduce Weight; N = 404) engaged their social networks and used social and mobile technologies to try and lose weight. Participants in the present study (n = 20 treatment, n = 18 control) were approached after a measurement visit and administered semi-structured interviews. Interviews were analyzed using principles from grounded theory.