Engineering Commons^™

Harnessing Confidence For Report Aggregation In Crowdsourcing Environments, Hadeel Alhosaini, Xianzhi Wang, Lina Yao, Zhong Yang, Farookh Hussain, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

Crowdsourcing is an effective means of accomplishing human intelligence tasks by leveraging the collective wisdom of crowds. Given reports of various accuracy degrees from workers, it is important to make wise use of these reports to derive accurate task results. Intuitively, a task result derived from a sufficient number of reports bears lower uncertainty, and higher uncertainty otherwise. Existing report aggregation research, however, has largely neglected the above uncertainty issue. In this regard, we propose a novel report aggregation framework that defines and incorporates a new confidence measure to quantify the uncertainty associated with tasks and workers, thereby enhancing result …