Social and Behavioral Sciences Commons^™

The Reliability Of Crowdsourcing: Latent Trait Modeling With Mechanical Turk, Matt Baucum, Steven Rouse Dr., Cindy Miller-Perrin, Elizabeth Mancuso Dr.

Seaver College Research And Scholarly Achievement Symposium

Mechanical Turk, an online crowdsourcing platform, has recently received increased attention in the social sciences as studies continue to suggest its viability as a source for reliable experimental data. Given the ease with which large samples can be quickly and inexpensively gathered, it is worth examining whether Mechanical Turk can provide accurate experimental data for methodologies requiring such large samples. One such methodology is Item Response Theory, a psychometric paradigm that defines test items by a mathematical relationship between a respondent’s ability and the probability of item endorsement. To test whether Mechanical Turk can serve as a reliable source of …