Too Anecdotal to Be True? Mechanical Turk Is Not All Bots and Bad Data: Response to Webb and Tangney (2022).

Perspect Psychol Sci

Department of Management and Entrepreneurship, School of Business, Virginia Commonwealth University.

Published: November 2024

AI Article Synopsis

  • Some researchers said that data from Amazon's MTurk was not very good, claiming it was only 2.6% valid.
  • We think they made some mistakes in how they designed their study and collected information, which made their results not so strong.
  • In our response, we share tips on how to get better and more reliable data from online surveys.

Article Abstract

In response to Webb and Tangney (2022) we call into question the conclusion that data collected on Amazon's Mechanical Turk (MTurk) was "at best-only 2.6% valid" (p. 1). We suggest that Webb and Tangney made certain choices during the study-design and data-collection process that adversely affected the quality of the data collected. As a result, the anecdotal experience of these authors provides weak evidence that MTurk provides low-quality data as implied. In our commentary we highlight best practice recommendations and make suggestions for more effectively collecting and screening online panel data.

Download full-text PDF

Source
http://dx.doi.org/10.1177/17456916241234328DOI Listing

Publication Analysis

Top Keywords

webb tangney
12
mechanical turk
8
response webb
8
tangney 2022
8
data collected
8
data
5
anecdotal true?
4
true? mechanical
4
turk bots
4
bots bad
4

Similar Publications

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!