Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 4 of 4
Full-Text Articles in Computer Sciences
Your Cursor Reveals: On Analyzing Workers’ Browsing Behavior And Annotation Quality In Crowdsourcing Tasks, Pei-Chi Lo, Ee-Peng Lim
Your Cursor Reveals: On Analyzing Workers’ Browsing Behavior And Annotation Quality In Crowdsourcing Tasks, Pei-Chi Lo, Ee-Peng Lim
Research Collection School Of Computing and Information Systems
In this work, we investigate the connection between browsing behavior and task quality of crowdsourcing workers performing annotation tasks that require information judgements. Such information judgements are often required to derive ground truth answers to information retrieval queries. We explore the use of workers’ browsing behavior to directly determine their annotation result quality. We hypothesize user attention to be the main factor contributing to a worker’s annotation quality. To predict annotation quality at the task level, we model two aspects of task-specific user attention, also known as general and semantic user attentions . Both aspects of user attention can be …
Testsgd: Interpretable Testing Of Neural Networks Against Subtle Group Discrimination, Mengdi Zhang, Jun Sun, Jingyi Wang, Bing Sun
Testsgd: Interpretable Testing Of Neural Networks Against Subtle Group Discrimination, Mengdi Zhang, Jun Sun, Jingyi Wang, Bing Sun
Research Collection School Of Computing and Information Systems
Discrimination has been shown in many machine learning applications, which calls for sufficient fairness testing before their deployment in ethic-relevant domains. One widely concerning type of discrimination, testing against group discrimination, mostly hidden, is much less studied, compared with identifying individual discrimination. In this work, we propose TestSGD, an interpretable testing approach which systematically identifies and measures hidden (which we call ‘subtle’) group discrimination of a neural network characterized by conditions over combinations of the sensitive attributes. Specifically, given a neural network, TestSGD first automatically generates an interpretable rule set which categorizes the input space into two groups. Alongside, TestSGD …
Generalization Through Diversity: Improving Unsupervised Environment Design, Wenjun Li, Pradeep Varakantham, Dexun Li
Generalization Through Diversity: Improving Unsupervised Environment Design, Wenjun Li, Pradeep Varakantham, Dexun Li
Research Collection School Of Computing and Information Systems
Agent decision making using Reinforcement Learning (RL) heavily relies on either a model or simulator of the environment (e.g., moving in an 8x8 maze with three rooms, playing Chess on an 8x8 board). Due to this dependence, small changes in the environment (e.g., positions of obstacles in the maze, size of the board) can severely affect the effectiveness of the policy learned by the agent. To that end, existing work has proposed training RL agents on an adaptive curriculum of environments (generated automatically) to improve performance on out-of-distribution (OOD) test scenarios. Specifically, existing research has employed the potential for the …
Champions For Social Good: How Can We Discover Social Sentiment And Attitude-Driven Patterns In Prosocial Communication?, Raghava Rao Mukkamala, Robert J. Kauffman, Helle Zinner Henriksen
Champions For Social Good: How Can We Discover Social Sentiment And Attitude-Driven Patterns In Prosocial Communication?, Raghava Rao Mukkamala, Robert J. Kauffman, Helle Zinner Henriksen
Research Collection School Of Computing and Information Systems
The UN High Commissioner on Refugees (UNHCR) is pursuing a social media strategy to inform people about displaced populations and refugee emergencies. It is actively engaging public figures to increase awareness through its prosocial communications and improve social informedness and support for policy changes in its services. We studied the Twitter communications of UNHCR social media champions and investigated their role as high-profile influencers. In this study, we offer a design science research and data analytics framework and propositions based on the social informedness theory we propose in this paper to assess communication about UNHCR’s mission. Two variables—refugee-emergency and champion …