Physical Sciences and Mathematics Commons^™

Evaluating Human Versus Machine Learning Performance In Classifying Research Abstracts, Yeow Chong Goh, Xin Qing Cai, Walter Theseira, Giovanni Ko, Khiam Aik Khor

Research Collection School Of Economics

We study whether humans or machine learning (ML) classification models are better at classifying scientific research abstracts according to a fixed set of discipline groups. We recruit both undergraduate and postgraduate assistants for this task in separate stages, and compare their performance against the support vectors machine ML algorithm at classifying European Research Council Starting Grant project abstracts to their actual evaluation panels, which are organised by discipline groups. On average, ML is more accurate than human classifiers, across a variety of training and test datasets, and across evaluation panels. ML classifiers trained on different training sets are also more …