Digital Commons Network^™

Towards Enriching Responses With Crowd-Sourced Knowledge For Task-Oriented Dialogue, Yingxu He, Lizi Liao, Zheng Zhang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Task-oriented dialogue agents are built to assist users in completing various tasks. Generating appropriate responses for satisfactory task completion is the ultimate goal. Hence, as a convenient and straightforward way, metrics such as success rate, inform rate etc., have been widely leveraged to evaluate the generated responses. However, beyond task completion, there are several other factors that largely affect user satisfaction, which remain under-explored. In this work, we focus on analyzing different agent behavior patterns that lead to higher user satisfaction scores. Based on the findings, we design a neural response generation model EnRG. It naturally combines the power of …

A Comparative Study On Feature Extraction And Classification/Clustering Of Fake News And Conspiracy Theories From Twitter Data, Deb Shana

Theses and Dissertations

Fake news and conspiracy theories have become largely abundant in the expanding world of social media. They predominantly affect the beliefs and thoughts of the public, resulting in chaos. They have always existed throughout the last few decades. They have been linked to prejudice, revolutions and genocide across history. They have also been known to have propelled people to reject mainstream medicines to an extent where some diseases are recurring in some parts of the world. They impose a serious impact since they are capable of spreading very fast Thus, it is very important to find suitable ways to detect …

Disentangling Hate In Online Memes, Ka Wei, Roy Lee, Rui Cao, Ziqing Fan, Jing Jiang, Wen Haw Chong

Research Collection School Of Computing and Information Systems

Hateful and offensive content detection has been extensively explored in a single modality such as text. However, such toxic information could also be communicated via multimodal content such as online memes. Therefore, detecting multimodal hateful content has recently garnered much attention in academic and industry research communities. This paper aims to contribute to this emerging research topic by proposing DisMultiHate, which is a novel framework that performed the classification of multimodal hateful content. Specifically, DisMultiHate is designed to disentangle target entities in multimodal memes to improve the hateful content classification and explainability. We conduct extensive experiments on two publicly available …

Conquer: Contextual Query-Aware Ranking For Video Corpus Moment Retrieval, Zhijian Hou, Chong-Wah Ngo, W. K. Chan

Research Collection School Of Computing and Information Systems

This paper tackles a recently proposed Video Corpus Moment Retrieval task. This task is essential because advanced video retrieval applications should enable users to retrieve a precise moment from a large video corpus. We propose a novel CONtextual QUery-awarE Ranking (CONQUER) model for effective moment localization and ranking. CONQUER explores query context for multi-modal fusion and representation learning in two different steps. The first step derives fusion weights for the adaptive combination of multi-modal video content. The second step performs bi-directional attention to tightly couple video and query as a single joint representation for moment localization. As query context is …

Token Shift Transformer For Video Classification, Zhang Hao, Yanbin. Hao, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Transformer achieves remarkable successes in understanding 1 and 2-dimensional signals (e.g., NLP and Image Content Understanding). As a potential alternative to convolutional neural networks, it shares merits of strong interpretability, high discriminative power on hyper-scale data, and flexibility in processing varying length inputs. However, its encoders naturally contain computational intensive operations such as pair-wise self-attention, incurring heavy computational burden when being applied on the complex 3-dimensional video signals. This paper presents Token Shift Module (i.e., TokShift), a novel, zero-parameter, zero-FLOPs operator, for modeling temporal relations within each transformer encoder. Specifically, the TokShift barely temporally shifts partial [Class] token features back-and-forth …

Animal Communication And Sustainability Project, Reiko Kataoka, Tridha Chatterjee

Assignment Prompts

This assignment is for the Ling 123 course. This is a GE course under Area R, and it is a large course (about 10 sections, max 25 students in each section) taught by multiple instructors.

The assignment is designed as a term project. With one or two preparatory class meetings, this two-part project (three-part including the learning) could be completed in 4-5 weeks, culminating in group presentations. The assignment would be best given toward the end of the semester as a capstone project after students have gained in-depth knowledge on mechanisms and adaptive functions of animal communication systems and the …

Uncovering Patterns In Reviewers' Feedback To Scene Description Authors, Rosiana Natalie, Jolene Kar Inn Loh, Huei Suen Tan, Joshua Shi-Hao Tseng, Hernisa Kacorri, Kotaro Hara

Research Collection School Of Computing and Information Systems

Audio descriptions (ADs) can increase access to videos for blind people. Researchers have explored different mechanisms for generating ADs, with some of the most recent studies involving paid novices; to improve the quality of their ADs, novices receive feedback from reviewers. However, reviewer feedback is not instantaneous. To explore the potential for real-time feedback through automation, in this paper, we analyze 1,120 comments that 40 sighted novices received from a sighted or a blind reviewer. We find that feedback patterns tend to fall under four themes: (i) Quality; commenting on different AD quality variables, (ii) Speech Act; the utterance or …

Holoboard: A Large-Format Immersive Teaching Board Based On Pseudo Holographics, Jiangtao Gong, Teng Han, Siling Guo, Jiannan Li, Siyu Zha, Liuxin Zhang, Feng Tian, Qianying Wang, Yong Rui

Research Collection School Of Computing and Information Systems

In this paper, we present HoloBoard, an interactive large-format pseduo-holographic display system for lecture based classes. With its unique properties of immersive visual display and transparent screen, we designed and implemented a rich set of novel interaction techniques like immersive presentation, role-play, and lecturing behind the scene that are potentially valuable for lecturing in class. We conducted a controlled experimental study to compare a HoloBoard class with a normal class through measuring students’ learning outcomes and three dimensions of engagement (i.e., behavioral, emotional, and cognitive engagement). We used pre-/post- knowledge tests and multimodal learning analytics to measure students’ learning outcomes …

Aixfood'21: 3rd Workshop On Aixfood, Ricardo Guerrero, Michael Spranger, Shuqiang Jiang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Food and cooking analysis present exciting research and application challenges for modern AI systems, particularly in the context of multimodal data such as images or video. A meal that appears in a food image is a product of a complex progression of cooking stages, often described in the accompanying textual recipe form. In the cooking process, individual ingredients change their physical properties, become combined with other food components, all to produce a final, yet highly variable, appearance of the meal. Recognizing food items or meals on a plate from images or videos, their physical properties such as the amount, nutritional …

Integrated Discourse Analysis & Learning Skills Framework For Class Conversations, Devyn Wei Hung Tan, Gottipati Swapna, Kyong Jin Shim, Shankararaman, Venky

Research Collection School Of Computing and Information Systems

Constructive interactions through discussion forums allow students to open their horizons and thought processes to acquire more knowledge and develop skills. Thus, discussion forums play an important role in supporting learning. Additionally, the discussion forum provides the content for creating a knowledge repository. It contains discussion threads related to key course topics that are debated by the students. One approach to understanding the student learning experience is through the analysis of the discussion threads. This research proposes the application of discourse analysis and collaborative learning frameworks to discussion forums to gain further insights into the student’s learning in a classroom. …

Data Analysis Of The “2021 Covid, Equity And Social Justice Showcase”, Cristo Leon, James Lipuma

STEM Month

During the “2021 STEM for All Video Showcase” (NSF, 2021) funded by the National Science Foundation, 287 short videos showcasing federally funded projects aimed at improving STEM and CS education were presented.

The videos highlight strategies to engage students during COVID-19 and address educational inequities.

Users’ Sentiment Analysis Toward National Digital Library Of India: A Quantitative Approach For Understanding User Perception, Ritu Sharma, Sarita Gulati, Amanpreet Kaur, Rupak Chakravarty

Library Philosophy and Practice (e-journal)

Sentiment analysis is also known as opinion mining. Sentiment analysis is contextual mining of text which identifies and extracts subjective information in textual data. It is extremely used by business, educational organizations, and social media monitoring to gain the general outlook of the wide public regarding their product and policy. The current study looks for gaining insights into user reviews on the National Digital Library of India (NDLI) mobile app (android and iOS). For this purpose, sentiment analysis will be used. It yields an average of 3.64/5 ratings based on 11,861 reviews. The dataset includes a total of 4560 user …

Cognition-Enhanced Machine Learning For Better Predictions With Limited Data, Florian Sense, Ryan Wood, Michael G. Collins, Joshua Fiechter, Aihua W. Wood, Michael Krusmark, Tiffany Jastrzembski, Christopher W. Myers

Faculty Publications

The fields of machine learning (ML) and cognitive science have developed complementary approaches to computationally modeling human behavior. ML's primary concern is maximizing prediction accuracy; cognitive science's primary concern is explaining the underlying mechanisms. Cross-talk between these disciplines is limited, likely because the tasks and goals usually differ. The domain of e-learning and knowledge acquisition constitutes a fruitful intersection for the two fields’ methodologies to be integrated because accurately tracking learning and forgetting over time and predicting future performance based on learning histories are central to developing effective, personalized learning tools. Here, we show how a state-of-the-art ML model can …

Generative Adversarial Networks For Classic Cryptanalysis, Deanne Charan

Master's Projects

The necessity of protecting critical information has been understood for millennia. Although classic ciphers have inherent weaknesses in comparison to modern ciphers, many classic ciphers are extremely challenging to break in practice. Machine learning techniques, such as hidden Markov models (HMM), have recently been applied with success to various classic cryptanalysis problems. In this research, we consider the effectiveness of the deep learning technique CipherGAN---which is based on the well- established generative adversarial network (GAN) architecture---for classic cipher cryptanalysis. We experiment extensively with CipherGAN on a number of classic ciphers, and we compare our results to those obtained using HMMs.

Expansion Of The Department Of Mathematics At Princeton University And The Founding Of The School Of Mathematics At The Institute For Advanced Study: 1900-1950, Andrew Beman-Cavallaro

Great Plains Graduate Review

From 1900 to 1950 Princeton, New Jersey, hosted two of the most prestigious institutions and provided a location for the expansive mathematical investigations taking place just before, during, and immediately after the Second World War. Princeton University and the Institute for Advanced Study housed, fed, and provided workspaces for an array of Mathematicians uncovering new research methodologies (resulting in the defeat of both the Nazi Party and the Empire of Nippon), the foundation for modern experimental Mathematics, and expansions of Theoretical and Applied Physics.

The View From Ventress - 2020-21 Academic Year, University Of Mississippi, College Of Liberal Arts

Liberal Arts Newsletters

News from the College of Liberal Arts

Becoming A Place Of Unrest : Environmental Crisis And Ecophenomenological Praxis, Robert Booth

Ohio University Press Open Access Books

The key to mitigating the environmental crisis isn’t just based on science; it depends upon a profound philosophical revision of how we think about and behave in relation to the world.

Our ongoing failure to interrupt the environmental crisis in a meaningful way stems, in part, from how we perceive the environment—what Robert Booth calls the "more-than-human world.” Anthropocentric presumptions of this world, inherited from natural science, have led us to better scientific knowledge about environmental problems and more science-based—yet inadequate—practical “solutions.” That’s not enough, Booth argues. Rather, he asserts that we must critically and self-reflexively revise how we perceive …

In-Game Social Interactions To Facilitate Esl Students' Morphological Awareness, Language And Literacy Skills, Yolanda A. Rankin, Sana Tibi, Casey Kennington, Na-Eun Han

Computer Science Faculty Publications and Presentations

Video games that require players to utilize a target or second language to complete tasks have emerged as alternative pedagogical tools for Second Language Acquisition (SLA). With the exception of vocabulary acquisition, much of the prior research in game-based SLA fails to gauge students' literacy skills, specifically their morphological awareness or understanding of the smallest meaningful linguistic units (e.g., prefixes, suffixes, and roots). Given this shortcoming, we utilize a two-player online game to facilitate social interactions between Native English Speakers (NES) and English as a Second Language (ESL) students as a mechanism to generate ESL students' written output in the …

Towards Cnl-Based Verbalization Of Computational Contracts, Inari Listenmaa, Maryam Hanafiah, Regina Cheong, Andreas Kallberg

Centre for Computational Law (2022-2025)

We present a CNL, which is a component of L4, a domain-specific programming language for drafting laws and contracts. Along with formal verification, L4’s core functionalities include natural language generation. We present the NLG pipeline and an interactive process for ambiguity resolution.

Does Bert Understand Idioms? A Probing-Based Empirical Study Of Bert Encodings Of Idioms, Minghuan Tan, Jing Jiang

Research Collection School Of Computing and Information Systems

Understanding idioms is important in NLP. In this paper, we study to what extent pre-trained BERT model can encode the meaning of a potentially idiomatic expression (PIE) in a certain context. We make use of a few existing datasets and perform two probing tasks: PIE usage classification and idiom paraphrase identification. Our experiment results suggest that BERT indeed can separate the literal and idiomatic usages of a PIE with high accuracy. It is also able to encode the idiomatic meaning of a PIE to some extent.

Enhancing Project Based Learning With Unsupervised Learning Of Project Reflections, Hua Leong Fwa

Research Collection School Of Computing and Information Systems

Natural Language Processing (NLP) is an area of research and application that uses computers to analyze human text. It has seen wide adoption within several industries but few studies have investigated it for use in evaluating the effectiveness of educational interventions and pedagogies. Pedagogies such as Project based learning (PBL) centers on learners solving an authentic problem or challenge which leads to knowledge creation and higher engagement. PBL also lends itself well in plugging the gap between what is taught in classrooms and applying the knowledge gained to the real working environment. In this study, we seek to investigate how …

Exploiting Group Structures To Infer Social Interactions From Videos, Maksim Bolonkin

Dartmouth College Ph.D Dissertations

In this thesis, we consider the task of inferring the social interactions between humans by analyzing multi-modal data. Specifically, we attempt to solve some of the problems in interaction analysis, such as long-term deception detection, political deception detection, and impression prediction. In this work, we emphasize the importance of using knowledge about the group structure of the analyzed interactions. Previous works on the matter mostly neglected this aspect and analyzed a single subject at a time. Using the new Resistance dataset, collected by our collaborators, we approach the problem of long-term deception detection by designing a class of histogram-based features …

Exploring The Personality Of Virtual Tutors In Conversational Foreign Language Practice, Johanna Dobbriner, Cathy Ennis, Robert J. Ross

Conference papers

Fluid interaction between virtual agents and humans requires the understanding of many issues of conversational pragmatics. One such issue is the interaction between communication strategy and personality. As a step towards developing models of personality driven pragmatics policies, in this paper, we present our initial experiment to explore differences in user interaction with two contrasting avatar personalities. Each user saw a single personality in a video-call setting and gave feedback on the interaction. Our expectations, that a more extroverted outgoing positive personality would be a more successful tutor, were only partially confirmed. While this personality did induce longer conversations in …

Biasheal: On-The-Fly Black-Box Healing Of Bias In Sentiment Analysis Systems, Zhou Yang, Harshit Jain, Jieke Shi, Muhammad Hilmi Asyrofi, David Lo

Research Collection School Of Computing and Information Systems

Although Sentiment Analysis (SA) is widely applied in many domains, existing research has revealed that the unfairness in SA systems can be harmful to the welfare of less privileged people. Several works propose pre-processing and in-processing methods to eliminate bias in SA systems, but little attention is paid to utilizing post-processing methods to heal bias. Postprocessing methods are particularly important for systems that use third-party SA services. Systems that use such services have no access to the SA engine or its training data and thus cannot apply pre-processing nor in-processing methods. Therefore, this paper proposes a black-box post-processing method to …

Learning And Evaluating Chinese Idiom Embeddings, Minghuan Tan, Jing Jiang

Research Collection School Of Computing and Information Systems

We study the task of learning and evaluating Chinese idiom embeddings. We first construct a new evaluation dataset that contains idiom synonyms and antonyms. Observing that existing Chinese word embedding methods may not be suitable for learning idiom embeddings, we further present a BERT-based method that directly learns embedding vectors for individual idioms. We empirically compare representative existing methods and our method. We find that our method substantially outperforms existing methods on the evaluation dataset we have constructed.

Injecting Descriptive Meta-Information Into Pre-Trained Language Models With Hypernetworks, Wenying Duan, Xiaoxi He, Zimu Zhou, Hong Rao, Lothar Thiele

Research Collection School Of Computing and Information Systems

Pre-trained language models have been widely adopted as backbones in various natural language processing tasks. However, existing pre-trained language models ignore the descriptive meta-information in the text such as the distinction between the title and the mainbody, leading to over-weighted attention to insignificant text. In this paper, we propose a hypernetwork-based architecture to model the descriptive meta-information and integrate it into pre-trained language models. Evaluations on three natural language processing tasks show that our method notably improves the performance of pre-trained language models and achieves the state-of-the-art results on keyphrase extraction.

Detecting Stance On Covid-19 Vaccine In A Polarized Media, Rodica Ceslov

Dissertations, Theses, and Capstone Projects

The growing polarization in the United States has been widely reported. Media coverage plays an important role in shaping public opinion and influences public debates on complex and unfamiliar topics. There are some benefits to individuals and society from political polarization and conflict between opposing viewpoints. However, recent research has primarily highlighted the negative consequences of polarization which reached an all-time high. One such topic is the Covid-19 vaccine which was developed in record time, and the public learned about its safety and possible risks through the media coverage.

In this capstone, we examine U.S. news media coverage on the …

Mollusc Shells From Neolithic Contexts In The Lake Eyasi Basin, Northern Tanzania, Musa S. Mwitondi, Albert S. Mjandwa, Pastory M. Bushozi

Tanzania Journal of Science

The study of the Neolithic period in the Lake Eyasi Basin was dominated by attempts to formulate the area’s chronology, mobility, settlement patterns, subsistence, and cultural capabilities of Neolithic people as attested by domesticated animals, pottery, and lithic artefacts. Occasionally, studies on molluscs were mentioned, but rarely described in detail. Neolithic sites across the Lake Eyasi Basin have yielded remains of both terrestrial (gastropods) and freshwater mollusc shells (mussels). The abundance of mollusc shell remains in the archaeological records of the Lake Eyasi Basin have played a great role in chronological settings, mobility and community integrations, studies of settlement patterns, …

Partially Speaker-Dependent Automatic Speech Recognition Using Deep Neural Networks, Christopher J. Li, Michelle Spigner

Journal of the South Carolina Academy of Science

No abstract provided.

Reducing The Manual Annotation Effort For Handwriting Recognition Using Active Transfer Learning, Eric Burdett

Theses and Dissertations

Handwriting recognition systems have achieved remarkable performance over the past several years with the advent of deep neural networks. For high-quality recognition, these models require large amounts of labeled training data, which can be difficult to obtain. Various methods to reduce this effort have been proposed in the realms of active and transfer learning, but not in combination. We propose a framework for fitting new handwriting recognition models that joins active and transfer learning into a unified framework. Empirical results show the superiority of our method compared to traditional active learning, transfer learning, or standard supervised training schemes.