Background: Situational judgments tests have been increasingly used to help training programs for the health professions incorporate professionalism attributes into their admissions process. While such tests have strong psychometric properties for testing professional attributes and are feasible to implement in high-volume, high-stakes selection, little is known about constructed-response situational judgment tests and their validity.
Methods: We will conduct a systematic review of primary published or unpublished studies reporting on the association between scores on constructed-response situational judgment tests and scores on other tests that measure personal, interpersonal, or professional attributes in training programs for the health professions. In addition to searching electronic databases, we will contact academics and researchers and undertake backward and forward searching. Two reviewers will independently screen the papers and decide on their inclusion, first based on the titles and abstracts of all citations, and then according to the full texts. Data extraction will be done independently by two reviewers using a data extraction form to chart study details and key findings. Studies will be assessed for the risk of bias and quality by two reviewers using the "Quality In Prognosis Studies" tool. To synthesize evidence, we will test the statistical heterogeneity and conduct a psychometric meta-analysis using a random-effects model. If adequate data are available, we will explore whether the meta-analytic correlation varies across different subgroups (e.g., race, gender).
Discussion: The findings of this study will inform best practices for admission and selection of applicants for training programs for the health professions and encourage further research on constructed-response situational judgment tests, in particular their validity.
Trial Registration: The protocol for this systematic review has been registered in PROSPERO [CRD42022314561]. https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022314561.
Download full-text PDF |
Source |
---|---|
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC9879421 | PMC |
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0280493 | PLOS |
BMC Med Educ
November 2024
Medical Education Department, Educational Development Center, Shahid Sadoughi University of Medical Sciences, Yazd, Iran.
Background: The study aims to assess the situational judgment capability of students in various professions, including medicine, surgical nursing, anesthesia nursing, and emergency medical technology, using a validated and adapted Situational Judgment Test (SJT).
Methods: The cross-sectional study was conducted at Qom University of Medical Sciences in 2023-2024. The study consisted of two steps: (1) adaptation and validity assessment of the SJT in various health professions, and (2) evaluation of students' situational judgment capability using the adapted SJT.
Br J Math Stat Psychol
February 2024
University of Georgia, Athens, Georgia, USA.
Textual data are increasingly common in test data as many assessments include constructed response (CR) items as indicators of participants' understanding. The development of techniques based on natural language processing has made it possible for researchers to rapidly analyse large sets of textual data. One family of statistical techniques for this purpose are probabilistic topic models.
View Article and Find Full Text PDFPLoS One
August 2023
Department of Research, Acuity Insights, Toronto, Ontario, Canada.
The study explored the impacts of patterns of language use (PLU) and socio-economic status (SES) on Casper, a constructed-response situational judgment test (SJT). 10,266 applicants from two U.S.
View Article and Find Full Text PDFPLoS One
January 2023
Acuity Insights, Toronto, Canada.
Background: Situational judgments tests have been increasingly used to help training programs for the health professions incorporate professionalism attributes into their admissions process. While such tests have strong psychometric properties for testing professional attributes and are feasible to implement in high-volume, high-stakes selection, little is known about constructed-response situational judgment tests and their validity.
Methods: We will conduct a systematic review of primary published or unpublished studies reporting on the association between scores on constructed-response situational judgment tests and scores on other tests that measure personal, interpersonal, or professional attributes in training programs for the health professions.
Adv Health Sci Educ Theory Pract
October 2021
Department of Emergency Medicine, Inselspital University Hospital, University of Berne, 3010, Freiburgstrasse, Berne, Switzerland.
The use of response formats in assessments of medical knowledge and clinical reasoning continues to be the focus of both research and debate. In this article, we report on an experimental study in which we address the question of how much list-type selected response formats and short-essay type constructed response formats are related to differences in how test takers approach clinical reasoning tasks. The design of this study was informed by a framework developed within cognitive psychology which stresses the importance of the interplay between two components of reasoning-self-monitoring and response inhibition-while solving a task or case.
View Article and Find Full Text PDFEnter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!