Establishing Human Observer Criterion in Evaluating Artificial Social Intelligence Agents in a Search and Rescue Task.

Artificial social intelligence (ASI) agents have great potential to aid the success of individuals, human-human teams, and human-artificial intelligence teams. To develop helpful ASI agents, we created an urban search and rescue task environment in Minecraft to evaluate ASI agents' ability to infer participants' knowledge training conditions and predict participants' next victim type to be rescued. We evaluated ASI agents' capabilities in three ways: (a) comparison to ground truth-the actual knowledge training condition and participant actions; (b) comparison among different ASI agents; and (c) comparison to a human observer criterion, whose accuracy served as a reference point.

From Teams to Teamness: Future Directions in the Science of Team Cognition.

Objective: We review the current state-of-the-art in team cognition research, but more importantly describe the limitations of existing theories, laboratory paradigms, and measures considering the increasing complexities of modern teams and the study of team cognition.

Background: Research on, and applications of, team cognition has led to theories, data, and measures over the last several decades.

Method: This article is based on research questions generated in a spring 2022 seminar on team cognition at Arizona State University led by the first author.