Digital Commons Network^™

Hack24f: Llms And World Knowledge Problems, Chengtaon Lin, Joseph Lobo, Huuthanhvy Nguyen, Brian Santos, Ting Xu

Paul English Applied Artificial Intelligence (AI) Institute Publications

While AI “hallucinations” have been widely publicized by the media as shortcomings of Large Language Models (LLMs), there is much less awareness about shortcomings that derive from the potential lack of understanding of the world, from a potential lack of commonsense knowledge. We want to raise awareness among our student body about the potential fallibility of LLMs when they the problems that they need to solve require a basic understanding of the world. We hope that this awareness will prompt everyone to question more carefully the outputs generated by LLMs.

Beyond Binary: Revealing Variations In Islamophobic Content With Hierarchical Multi-Class Classification, Esraa Aldreabi, Khawlah M. Harahsheh, Mukul Dev Chhangani, Chung-Hao Chen, Jeremy Blackburn

Electrical & Computer Engineering Faculty Publications

In the digital age, the rise of Islamophobia-marked by an irrational fear or discrimination against Islam and Muslims-has emerged as a pressing issue, especially on social media platforms. In this paper we employs a multi-class classification system, moving beyond traditional binary models. We categorize Islamophobic content into three main classes and various subclasses, covering a range from subtle biases to explicit incitement. Comparative analysis of data from Reddit and Twitter illuminates the distinct prevalence and types of Islamophobic content specific to each platform. This paper deepens our understanding of digital Islamophobia and provides insights for crafting targeted online counter strategies. …

Toward Inclusivity: Rethinking Islamophobic Content Classification In The Digital Age, Esraa Aldreabi, Mukul Dev Chhangani, Khawlah M. Harahsheh, Justin M. Lee, Chung-Hao Chen

Electrical & Computer Engineering Faculty Publications

In this paper, we implement a comprehensive three-class system to categorize social media discussions about Islam and Muslims, enhancing the typical binary approach. These classes are: I) General Discourse About Islam and Muslims, II) Criticism of Islamic Teachings and Figures, and III) Comments Against Muslims. These categories are designed to balance the nuances of free speech while protecting diverse groups like Muslims, ex-Muslims, LGBTQ+ communities, and atheists. By utilizing machine learning and employing transformer-based models, we analyze the distribution and characteristics of these classes in social media content. Our findings reveal distinct patterns of user engagement with topics related to …

Shedding Light On Software Engineering-Specific Metaphors And Idioms, Mia Mohammad Imran, Preetha Chatterjee, Kostadin Damevski

Computer Science Faculty Research & Creative Works

Use of figurative language, such as metaphors and idioms, is common in our daily-life communications, and it can also be found in Software Engineering (SE) channels, such as comments on GitHub. Automatically interpreting figurative language is a challenging task, even with modern Large Language Models (LLMs), as it often involves subtle nuances. This is particularly true in the SE domain, where figurative language is frequently used to convey technical concepts, often bearing developer affect (e.g., 'spaghetti code). Surprisingly, there is a lack of studies on how figurative language in SE communications impacts the performance of automatic tools that focus on …

Eye-Gaze Guided Multi-Modal Alignment For Medical Representation Learning, Chong Ma, Hanqi Jiang, Wenting Chen, Yiwei Li, Zihao Wu, Xiaowei Yu, Zhengliang Liu, Lei Guo, Dajiang Zhu, Tuo Zhang, Dinggang Shen, Tianming Liu, Xiang Li

Computer Science Faculty Research & Creative Works

In the medical multi-modal frameworks, the alignment of cross-modality features presents a significant challenge. However, existing works have learned features that are implicitly aligned from the data, without considering the explicit relationships in the medical context. This data-reliance may lead to low generalization of the learned alignment relationships. In this work, we propose the Eye-gaze Guided Multi-modal Alignment (EGMA) framework to harness eye-gaze data for better alignment of medical visual and textual features. We explore the natural auxiliary role of radiologists' eye-gaze data in aligning medical images and text and introduce a novel approach by using eye-gaze data, collected synchronously …

Low-Resource Automatic Speech Recognition Domain Adaptation – A Case-Study In Aviation Maintenance, Nadine Amin, Tracy L. Yother, Julia Rayz

Journal of Aviation/Aerospace Education & Research

With timeliness and efficiency being critical in the aviation maintenance industry, the need has been growing for smart technological solutions that optimize and streamline the different underlying tasks (Bergkvist & Sabbagh, 2021). One such task is the technical documentation of the performed maintenance operations (Chandola et al., 2022). Instead of manual documentation, voice tools that transcribe spoken logbook entries allow technicians to document their work right away in a hands-free and time efficient manner. However, an accurate automatic speech recognition (ASR) model requires large training corpora (Siyaev & Jo, 2021a), which are lacking in the domain of aviation maintenance. In …

Effect Of Specific Data Variations On Automated Speaker Recognition, Ethan David Meighen

Graduate Theses, Dissertations, and Problem Reports (ETD)

Speaker recognition is not a new biometric modality but there are still many obstacles in the way in order for it to become as used as fingerprint recognition, facial recognition, and iris recognition. Many real-world environmental conditions, hardware device variations, and human behavior present serious challenges to the use of opportunistic voice or speaker samples for identification purposes. Non-idealities, identified as nuisance factors, include environmental noise, input device quality, length of utterance, sample rate variation, and unscripted data are common nuisance factors that can impact speaker recognition match score performance. The impact of the nuisance factors listed above were evaluated …

A Comparison Of Opennmt Sequence Model For Indonesian Automatic Question Generation, Yuniar Indrihapsari, Handaru Jati, N. Nurkhamid, Ratna Wardani, Pradana Setialana, Muhamad Izzudin Mahali, Danang Wijaya, Dhista Dwi Nur Ardiansyah, Satya Adhiyaksa Ardy, Maria Bernadetha Charlotta Wonda Tiala, Andi Hakim Al-Khawarizmi, Widya Ardiyanto

Elinvo (Electronics, Informatics, and Vocational Education)

Evaluation of learners is a crucial aspect of the educational system. However, creating evaluation instruments is a process that demands teachers' time and energy. The researcher developed the Indonesia Automatic Question Generator in this study using an architecture modified from past studies. The primary goals of this project are (1) to construct an AQG tool utilizing the OpenNMT series and (2) to analyze and compare the model's performance. As a data source, this study employs the SQuAD 2.0 dataset and numerous sequence techniques, including BiGRU, BiLSTM, and Transformer. The researcher trained the models using OpenNMT-py and Google Collaboratory. This approach …

Intelligent Security System In A Campus Building Using Rfid To Improve Security For Elevator Users, D. Desmira, Mustofa Abi Hamid, Lilis Noviawati, Tb. Ai Munandar, M. Martias, Didik Aribowo

Elinvo (Electronics, Informatics, and Vocational Education)

This study aims to develop an intelligent system of elevator access with RFID (Radio Frequency Identification) using fuzzy logic as an intelligent system, a safety controller of elevator doors that will open when the power goes out. This research made an Arduino Uno-based elevator access system with an RFID card using fuzzy logic as an intelligence system and markers for each floor. Each floor has an RFID card that contains the information data of each floor. This research uses research and development (R&D) methods with a waterfall development model. The research procedure consists of several stages, namely the requirement analysis …

Development Of An Android-Based Cultural Heritage Map App, Muhammad Irfan Luthfi, Ratna Wardani, Nisa Dwi Septiyanti

Elinvo (Electronics, Informatics, and Vocational Education)

This study aims to (1) develop an Android-based cultural heritage map app that can provide the location of cultural heritages in the Special Region in Yogyakarta along with its descriptions, images, and videos; and (2) understand the quality of the app based on standard software quality testing ISO / IEC 25010 on functional suitability, performance efficiency, and compatibility. The development method utilized a waterfall software development model that consists of communication, planning, modeling, construction, and deployment. The results of this study are 1) an Android-based cultural heritage map app that can provide the location of cultural heritages in the Special …

Cross-Language Tourism News Retrieval System Using Google Translate Api On Sebi Search Engine, H. Husni, Arif Muntasa, Sigit Susanto Putro, Zulfi Osman

Elinvo (Electronics, Informatics, and Vocational Education)

Cross-Language Information Retrieval (CLIR) is responsible for retrieving information stored in a language different from the language of the query provided by the user. Some translation methods commonly used in CLIR are Dictionary, Parallel corpora, Comparable corpora, Machine translator, Ontology, and Transitive-based. The query must be translated to the target language, followed by preprocessing and calculating the similarity between the query and all documents in the corpus. The problem is the time and accuracy of query translation. Moreover, the queries are not written as complete sentences according to certain language rules. Stemming, for example, every language has its own method. …

Design And Development Of Industrial Practice Monitoring And Assessment Systems Using Tsukamoto Fuzzy Logic, Tri Yuli Pahtoni, Fatchul Arifin

Elinvo (Electronics, Informatics, and Vocational Education)

Vocational high schools are given flexibility for their students to carry out direct learning in the industry as part of the practical education activities of implementing student skills. The implementation of industrial practice requires a special way to find out and monitor each student's activities so that the achievements of the implementation of industrial practice can be carried out properly. The implementation of industrial work practice assessment has several assessment criteria. These criteria include attendance, neatness, attitude, skills, and knowledge. The problems found in the assessment system are still done manually so that the effectiveness is minimal. This study aims …

Outsourcing Voting To Ai: Can Chatgpt Advise Index Funds On Proxy Voting Decisions?, Chen Wang

Fordham Journal of Corporate & Financial Law

Released in November 2022, Chat Generative Pre-training Transformer (“ChatGPT”), has risen rapidly to prominence, and its versatile capabilities have already been shown in a variety of fields. Due to ChatGPT’s advanced features, such as extensive pre-training on diverse data, strong generalization ability, fine-tuning capabilities, and improved reasoning, the use of AI in the legal industry could experience a significant transformation. Since small passive funds with low-cost business models generally lack the financial resources to make informed proxy voting decisions that align with their shareholders’ interests, this Article considers the use of ChatGPT to assist small investment funds, particularly small passive …

A Critical Computing Curriculum Design Case: Exploring Tribal Sovereignty For Middle School Students, Kristin A. Searle, Aubrey Rogowski, Colby Tofel-Grehl, Mengying Jiang

Journal of Computer Science Integration

We report on our efforts to design an integrated computing curriculum for middle school students in Montana that is in line with the Kapor Center’s focus on culturally sustaining-revitalizing pedagogies. Montana provides a unique context for doing this work because a state constitutional mandate requires all K-12 students to learn about tribal histories and cultures through Indian Education For All (IEFA). IEFA centers around seven essential understandings about Indigenous peoples in Montana that are integrated across content areas. In addition, implementation of Montana’s CS standards began in the 2021–2022 school year. In the curricular design, we sought to bring together …

Guilty Machines: On Ab-Sens In The Age Of Ai, Dylan Lackey, Katherine Weinschenk

Critical Humanities

For Lacan, guilt arises in the sublimation of ab-sens (non-sense) into the symbolic comprehension of sen-absexe (sense without sex, sense in the deficiency of sexual relation), or in the maturation of language to sensibility through the effacement of sex. Though, as Slavoj Žižek himself points out in a recent article regarding ChatGPT, the split subject always misapprehends the true reason for guilt’s manifestation, such guilt at best provides a sort of evidence for the inclusion of the subject in the order of language, acting as a necessary, even enjoyable mark of the subject’s coherence (or, more importantly, the subject’s separation …

Culturally Responsive-Sustaining Computational Thinking: Enactment In Elementary Classrooms, Victoria Macann, Aman Yadav

Journal of Computer Science Integration

Technology has increasingly permeated many aspects of everyday life and this evolution raises the need for individuals to understand how the digital world works and what opportunities and risks it brings (Nouri, Zhang, Mannila & Norén, 2019). For this to be an experience for everyone, we need to rethink how we integrate computational thinking (CT) and provide teachers with tools to center their students’ identities, experiences, and cultures in the classroom. In this paper, we present two case studies of primary (elementary) teachers from a full primary (student ages 5–13) semi-rural school in the North Island of New Zealand that …

Probing And Enhancing The Reliance Of Transformer Models On Poetic Information, Almas Abdibayev

Dartmouth College Ph.D Dissertations

Transformer models have achieved remarkable success in the widest variety of domains, spanning not just a multitude of tasks within natural language processing, but also those in computer vision, speech, and reinforcement learning. The key to this success is largely attributed to the self-attention mechanism, particularly its ability to scale in performance as it grows in the number of parameters. Extensive effort has been underway to study the major linguistic properties learned by these models during the course of their pretraining. However, the role of certain finer linguistic phenomena present in language and their utilization by Transformers has not been …

Study Of Augmentations On Historical Manuscripts Using Trocr, Erez Meoded

Theses and Dissertations

Historical manuscripts are an essential source of original content. For many reasons, it is hard to recognize these manuscripts as text. This thesis used a state-of-the-art Handwritten Text Recognizer, TrOCR, to recognize a 16^th-century manuscript. TrOCR uses a vision transformer to encode the input images and a language transformer to decode them back to text. We showed that carefully preprocessed images and designed augmentations can improve the performance of TrOCR. We suggest an ensemble of augmented models to achieve an even better performance.

Ncmpy: A Modelling Software For Neutrosophic Cognitive Maps Based On Python Package, Ilanthenral Kandasamy, Divakar Arumugam, Aryan Rathore, Ateeth Arun, Manan Jain, Vasantha .W.B, Florentin Smarandache

Neutrosophic Systems with Applications

Cognitive maps are a vital tool that can be used for knowledge representation and reasoning. Fuzzy Cognitive Maps (FCMs) are popular soft computing techniques used to model large and complex systems, and they can aid in explainable artificial intelligence (AI). FCMs, however, cannot model the indeterminacy that arises in a system due to various uncertainties. Neutrosophic Cognitive Maps (NCMs), upgraded FCMs that could model indeterminacy, were introduced to address this issue. NCMs are a generalization of FCMs, a field of cognitive science firmly based on neural networks. NCMs have been used to solve a wide range of problems. NCMs were …

On B-Anti-Open Sets: A Formal Definition, Proofs, And Examples, Sudeep Dey, Priyanka Paul, Gautam Chandra Ray

Neutrosophic Systems with Applications

The concepts of open sets, closed sets, the interior of a set, and the exterior of a set are the most basic concepts in the study of topological spaces in any setting. When we turn our attention to the concept of anti-topological spaces, we encounter analogous fundamental concepts, such as the definition of anti-open sets, anti-closed sets, anti-interior, anti-exterior, etc. These concepts have already been introduced and studied by mathematicians worldwide. In this article, we introduce and study the concepts of b-anti-open set, b-anti-closed set, anti-b-interior, and anti-b-closure in the context of anti-topological spaces and investigate some of their basic …

Overcoming Foreign Language Anxiety In An Emotionally Intelligent Tutoring System, Daneih Ismail

College of Computing and Digital Media Dissertations

Learning a foreign language entails cognitive and emotional obstacles. It involves complicated mental processes that affect learning and emotions. Positive emotions such as motivation, encouragement, and satisfaction increase learning achievement, while negative emotions like anxiety, frustration, and confusion may reduce performance. Foreign Language Anxiety (FLA) is a specific type of anxiety accompanying learning a foreign language. It is considered a main impediment that hinders learning, reduces achievements, and diminishes interest in learning.

Detecting FLA is the first step toward reducing and eventually overcoming it. Previously, researchers have been detecting FLA using physical measurements and self-reports. Using physical measures is direct …

The Holistic Archival Personality Profiling Model (Happm): Comprehensive Data Integration For Personality Analysis, James Hutson, Pace Ellsworth

Faculty Scholarship

The traditional approach to biographical profiling, predominantly reliant on limited and fragmented datasets, has frequently resulted in superficial personality understandings. This is largely due to an overemphasis on official records and notable events, neglecting the rich tapestry of everyday experiences and personal interactions that significantly shape personalities. To address this shortcoming, this article introduces a multi-disciplinary methodology, The Holistic Archival Personality Profiling Model (HAPPM), which integrates a diverse array of archival materials, including personal correspondences, social media footprints, and family memorabilia. This approach involves digitizing various data forms, including handwritten documents, into machine-readable text, and then semantically classifying this data …

Multimodal Image Synthesis And Editing: The Generative Ai Era, Fangneng Zhan, Yingchen Yu, Rongliang Wu, Jiahui Zhang, Shijian Lu, Lingjie Liu, Adam Kortylewski, Christian Theobalt, Eric Xing

Machine Learning Faculty Publications

As information exists in various modalities in real world, effective interaction and fusion among multimodal information plays a key role for the creation and perception of multimodal data in computer vision and deep learning research. With superb power in modeling the interaction among multimodal information, multimodal image synthesis and editing has become a hot research topic in recent years. Instead of providing explicit guidance for network training, multimodal guidance offers intuitive and flexible means for image synthesis and editing. On the other hand, this field is also facing several challenges in alignment of multimodal features, synthesis of high-resolution images, faithful …

Reinforced Target-Driven Conversational Promotion, Huy Quang Dao, Lizi Liao, Dung D. Le, Yuxiang Nie

Research Collection School Of Computing and Information Systems

The ability to proactively engage with users towards pitching products is highly desired for conversational assistants. However, existing conversational recommendation methods overemphasize on acquiring user preferences while ignore the strategic planning for nudging users towards accepting a designated item. Hence, these methods fail to promote specified items with engaging responses. In this work, we propose a Reinforced Target-driven Conversational Promotion (RTCP) framework for conversational promotion. RTCP integrates short-term and long-term planning via a balanced gating mechanism. Inside which, the dialogue actions are predicted via a knowledge-integrated multi-head attention and guided via reinforcement learning rewards. RTCP then employs action-guided prefix tuning …

End-To-End Task-Oriented Dialogue: A Survey Of Tasks, Methods, And Future Directions, Libo Qin, Wenbo Pan, Qiguang Chen, Lizi Liao, Zhou Yu, Yue Zhang, Wanxiang Che, Min Li

Research Collection School Of Computing and Information Systems

End-to-end task-oriented dialogue (EToD) can directly generate responses in an end-to-end fashion without modular training, which attracts escalating popularity. The advancement of deep neural networks, especially the successful use of large pre-trained models, has further led to significant progress in EToD research in recent years. In this paper, we present a thorough review and provide a unified perspective to summarize existing approaches as well as recent trends to advance the development of EToD research. The contributions of this paper can be summarized: (1) First survey: to our knowledge, we take the first step to present a thorough survey of this …

Clusterprompt: Cluster Semantic Enhanced Prompt Learning For New Intent Discovery, Jinggui Liang, Lizi Liao

Research Collection School Of Computing and Information Systems

The discovery of new intent categories from user utterances is a crucial task in expanding agent skills. The key lies in how to efficiently solicit semantic evidence from utterances and properly transfer knowledge from existing intents to new intents. However, previous methods laid too much emphasis on relations among utterances or clusters for transfer learning, while paying less attention to the usage of semantics. As a result, these methods suffer from in-domain over-fitting and often generate meaningless new intent clusters due to data distortion. In this paper, we present a novel approach called Cluster Semantic Enhanced Prompt Learning (CsePL) for …

Mermaid: A Dataset And Framework For Multimodal Meme Semantic Understanding, Shaun Toh, Adriel Kuek, Wen Haw Chong, Roy Ka Wei Lee

Research Collection School Of Computing and Information Systems

Memes are widely used to convey cultural and societal issues and have a significant impact on public opinion. However, little work has been done on understanding and explaining the semantics expressed in multimodal memes. To fill this research gap, we introduce MERMAID, a dataset consisting of 3,633 memes annotated with their entities and relations, and propose a novel MERF pipeline that extracts entities and their relationships in memes. Our framework combines state-of-the-art techniques from natural language processing and computer vision to extract text and image features and infer relationships between entities in memes. We evaluate the proposed framework on a …

Estimating Propensity For Causality-Based Recommendation Without Exposure Data, Zhongzhou Liu, Yuan Fang, Min Wu

Research Collection School Of Computing and Information Systems

Causality-based recommendation systems focus on the causal effects of user-item interactions resulting from item exposure (i.e., which items are recommended or exposed to the user), as opposed to conventional correlation-based recommendation. They are gaining popularity due to their multi-sided benefits to users, sellers and platforms alike. However, existing causality-based recommendation methods require additional input in the form of exposure data and/or propensity scores (i.e., the probability of exposure) for training. Such data, crucial for modeling causality in recommendation, are often not available in real-world situations due to technical or privacy constraints. In this paper, we bridge the gap by proposing …

Discourses Across Periods Of Time, Amanda Raquel Moreno

Theses

This literature review explores the revolutionary effect of generative artificial intelligence (AI) and virtual reality (VR) on digital art history, specifically concentrating on their capacity to enable dialogical exchanges with historical figures and deepen the understanding of artworks. This study considers the current state of research, detecting key methodologies, areas of improvement, and possible challenges and ethical concerns. The example historical figure used in this analysis is the iconic Mexican artist Frida Kahlo. Kahlo’s refusal to correspond to a specific artistic style makes her an ideal subject for generative AI and VR-based investigation, offering fresh insights into her work. The …

Examining The Inter-Consistency Of Large Language Models: An In-Depth Analysis Via Debate, Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, Bing Qin

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have shown impressive capabilities in various applications, but they still face various inconsistency issues. Existing works primarily focus on the inconsistency issues within a single LLM, while we complementarily explore the inter-consistency among multiple LLMs for collaboration. To examine whether LLMs can collaborate effectively to achieve a consensus for a shared goal, we focus on commonsense reasoning, and introduce a formal debate framework (FORD) to conduct a three-stage debate among LLMs with real-world scenarios alignment: fair debate, mismatched debate, and roundtable debate. Through extensive experiments on various datasets, LLMs can effectively collaborate to reach a consensus …