Large Language Models in an App: Conducting a Qualitative Synthetic Data Analysis of How Snapchat's "My AI" Responds to Questions About Sexual Consent, Sexual Refusals, Sexual Assault, and Sexting.

Comprehensive sexual health education for young people often remains largely inaccessible, leaving gaps in knowledge about sexual consent, refusals, sexual assault, and sexting. Snapchat's My AI, tailored to its users, offers accessible and potentially stigma-free responses to sexual health queries, showing promise in addressing these gaps. This study aimed to evaluate My AI's effectiveness in handling questions related to sexual communication, assault, and sexting, and its response consistency across users. Collaborating with young adults and leveraging prior literature, our team formulated questions on sexual consent, refusals, sexual assault, and sexting submitted to My AI by 15 researchers. Each team member opened a chat with My AI in Snapchat and asked it a set of questions in the same order. We then conducted a qualitative content analysis to assess for consistency across users. My AI responses emphasized the importance of clear, honest consent and refusal communication through physical contact or direct conversation. With potential experiences of sexual assault or sexting, it advised consultation with a trusted caregiver. While My AI provided similar responses across researchers, variations were noted. My AI's responses mirrored common sexual health education messages and the responses used language reflective of sympathy toward potential assault victims or those who were asked for sexual images. My AI's potential to enhance accessibility to sexual health information could empower young people in making informed sexual health decisions. However, variability in responses may limit impact and comprehension, underscoring the importance of educators complementing these tools.