Digital Commons Network^™

Exploring How Uncertain Labels From Non-Consensus Panels Affect Machine Learning, Amal Almansour

College of Computing and Digital Media Dissertations

A dataset becomes meaningful for analysis when it contains more representative features. Machine and deep learning models rely on annotated instances for training. The annotation process is usually done either by humans (experts or crowdsourcing) or by models. In many cases, the variability between humans (the inter-observer variability) in evaluation leads to uncertainty in the learning process. Due to the lack of reliable labels in large datasets, the inter-observer variability can be quantified with different methods to estimate the ground truth label (i.e., referenced standard label) for model learning.

In health care, with the rise of artificial intelligence in clinical …

Mention Detection In Turkish Coreference Resolution, Şeni̇z Demi̇r, Hani̇fi̇ İbrahi̇m Akdağ

Turkish Journal of Electrical Engineering and Computer Sciences

A crucial step in understanding natural language is detecting mentions that refer to real-world entities in a text and correctly identifying their boundaries. Mention detection is commonly considered a preprocessing step in coreference resolution which is shown to be helpful in several language processing applications such as machine translation and text summarization. Despite recent efforts on Turkish coreference resolution, no standalone neural solution to mention detection has been proposed yet. In this article, we present two models designed for detecting Turkish mentions by using feed-forward neural networks. Both models extract all spans up to a fixed length from input text …

Enhancing Dtc Control Of Im Using Fuzzy Logic And Three-Level Inverter: A Comparative Study, Siham Mencou, Majid Benyakhlef, Elbachir Tazi

Turkish Journal of Electrical Engineering and Computer Sciences

Direct torque control is the most appropriate strategy for induction motor drive systems, due to its considerable ability to reduce the impact of of machine parameter variations, while offering fast dynamic response and simplified control implementation. However, persistent problems associated with high torque ripple and variable switching frequencies prevent its widespread adoption. To overcome these limitations, several techniques have been developed, in particular the use of multi-level inverters and fuzzy logic algorithms. This article proposes an in-depth evaluation of these techniques in a MATALB/Simulink environment, under various operational conditions. The main objective is to provide a detailed performance analysis of …

Detecting Lgbtq+ Instances Of Cyberbullying, Arslan Bisharat, Manuel Madrigal, Mohammed Abuhamad, Deborah Hall, Yasin Silva

Computer Science: Faculty Publications and Other Works

Social media continues to have an impact on the trajectory of humanity. However, its introduction has also weaponized keyboards, allowing the abusive language normally reserved for in-person bullying to jump onto the screen, i.e., cyberbullying. Cyberbullying poses a significant threat to adolescents globally, affecting the mental health and well-being of many. A group that is particularly at risk is the LGBTQ+ community, as researchers have uncovered a strong correlation between identifying as LGBTQ+ and suffering from greater online harassment. Therefore, it is critical to develop machine learning models that can accurately discern cyberbullying incidents as they happen to LGBTQ+ members. …

Investigating The Intersection Of Ai And Cybercrime: Risks, Trends, And Countermeasures, Sanaika Shetty, Kyung-Shick Choi, Insun Park

International Journal of Cybersecurity Intelligence & Cybercrime

No abstract provided.

Adaptive Worlds: Generative Ai In Game Design And Future Of Gaming, And Interactive Media, Jay Ratican, James Hutson

Faculty Scholarship

Generative AI is revolutionizing the field of game design, introducing unprecedented adaptability and personalization in gameplay. The latest advancements in AI-driven engines enable real-time content creation, offering dynamic, player-driven experiences that diverge from traditional pre-programmed narratives. This shift marks a transition toward "choose your own adventure" formats, with an unlimited number of variations in levels, enemies, collectibles, and weaponry, tailored to each player's decisions. Google's GameNGen, for example, showcases AI's capacity to recreate classic games like DOOM, learning and generating gameplay in real time. These innovations are not restricted to gaming alone; they extend to edutainment, television, and film, where …

Rethinking Retrieval Augmented Fine-Tuning In An Evolving Llm Landscape, Nicholas Sager, Timothy Cabaza, Matthew Cusack, Ryan Bass, Joaquin Dominguez

SMU Data Science Review

This study explores the utilization of Retrieval Augmented Fine-Tuning (RAFT) to enhance the performance of Large Language Models (LLMs) in domain-specific Retrieval Augmented Generation (RAG) tasks. By integrating domain-specific information during the retrieval process, RAG aims to reduce hallucination and improve the accuracy of LLM outputs. We investigate the use of RAFT, an approach that enhances LLMs by incorporating domain-specific knowledge and effectively handling distractor documents. This paper validates previous work, which found that RAFT can considerably improve the performance of Llama2-7B in specific domains. We also expand upon previous work into new state-of-the-art open-source models and other datasets with …

Technoculture And Language Models In Archaeology: Reconstructing And Preserving Cultural Narratives Through Digital Humanities, James Hutson

Faculty Scholarship

Technoculture, which examines the intersection of culture and technology, has increasingly permeated archaeological practice, transforming both scholarly research and public engagement [1-3]. The introduction of digital tools such as virtual reality (VR), geographic information systems (GIS), and large language models (LLMs) has democratized access to archaeological knowledge, enabling communities to engage more actively with their cultural heritage [4-6]. This short article explores the mutual influence of technocultural studies and AI technologies on archaeology, with a focus on the preservation and reconstruction of cultural narratives through digital means.

The first aspect of this intersection lies in how technocultural tools are creating …

Putting Gpt-4o To The Sword: A Comprehensive Evaluation Of Language, Vision, Speech, And Multimodal Proficiency, Sakib Shahriar, Brady D. Lund, Nishith Reddy Mannuru, Muhammad Arbab Arshad, Kadhim Hayawi, Ravi Varma Kumar Bevara, Aashrith Mannuru, Laiba Batool

All Works

As large language models (LLMs) continue to advance, evaluating their comprehensive capabilities becomes significant for their application in various fields. This research study comprehensively evaluates the language, vision, speech, and multimodal capabilities of GPT-4o. The study employs standardized exam questions, reasoning tasks, and translation assessments to assess the model’s language capability. Additionally, GPT-4o’s vision and speech capabilities are tested through image classification and object-recognition tasks, as well as accent classification. The multimodal evaluation assesses the model’s performance in integrating visual and linguistic data. Our findings reveal that GPT-4o demonstrates high accuracy and efficiency across multiple domains in language and reasoning …

Artificial Intelligence In Orthopaedic Education: A Comparative Analysis Of Chatgpt And Bing Ai’S Orthopaedic In-Training Examination Performance, Clark Chen, Vivek Biololikar, Duncan Vannest, James Raphael, Gene Shaffer

Einstein Health Papers

Background: This study evaluated the performance of generative artificial intelligence (AI) models on the Orthopaedic In-Training Examination (OITE), an annual exam administered to U.S. orthopaedic residency programs. Methods: ChatGPT 3.5 and Bing AI GPT 4.0 were evaluated on standardised sets of multiple-choice questions drawn from the American Academy of Orthopaedic Surgeons OITE online question bank spanning 5 years (2018–2022). A total of 1165 questions were posed to each AI system. The performance of both systems was standardised using the latest versions of ChatGPT 3.5 and Bing AI GPT 4.0. Historical data of resident scores taken from the annual OITE technical …

Multi-Modal Alignment Via Hyperbolic Geometry, Suyu Liu

Dissertations and Theses Collection (Open Access)

Strong capabilities of generalization to unseen domains are vital for deep neural networks. While existing methods have shown promising results without source domain access, they mostly rely on models that are extensively pre-trained on source domains or overlook the intricate hierarchical structures inherent in visual and textual features. These limitations may have bad impacts on performances, especially on datasets with many classes. To overcome this, in this paper we propose a novel approach that projects the model onto hyperbolic geometry and employs geometric optimal transport to align cross-modal features in an unsupervised manner. Unlike Euclidean geometry, hyperbolic geometry is characterized …

Text-Driven Video Prediction, Xue Song, Jingjing Chen, Bin Zhu, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Current video generation models usually convert signals indicating appearance and motion received from inputs (e.g., image and text) or latent spaces (e.g., noise vectors) into consecutive frames, fulfilling a stochastic generation process for the uncertainty introduced by latent code sampling. However, this generation pattern lacks deterministic constraints for both appearance and motion, leading to uncontrollable and undesirable outcomes. To this end, we propose a new task called Text-driven Video Prediction (TVP). Taking the first frame and text caption as inputs, this task aims to synthesize the following frames. Specifically, appearance and motion components are provided by the image and caption …

Aligning Human And Computational Coherence Evaluations, Jia Peng Lim, Hady Wirawan Lauw

Research Collection School Of Computing and Information Systems

Automated coherence metrics constitute an efficient and popular way to evaluate topic models. Previous work presents a mixed picture of their presumed correlation with human judgment. This work proposes a novel sampling approach to mining topic representations at a large scale while seeking to mitigate bias from sampling, enabling the investigation of widely used automated coherence metrics via large corpora. Additionally, this article proposes a novel user study design, an amalgamation of different proxy tasks, to derive a finer insight into the human decision-making processes. This design subsumes the purpose of simple rating and outlier-detection user studies. Similar to the …

Probing Effects Of Contextual Bias On Number Magnitude Estimation, Xuehao Du, Ping Ji, Wei Qin, Lei Wang, Yunshi Lan

Research Collection School Of Computing and Information Systems

The semantic understanding of numbers requires association with context. However, powerful neural networks overfit spurious correlations between context and numbers in training corpus can lead to the occurrence of contextual bias, which may affect the network's accurate estimation of number magnitude when making inferences in real-world data. To investigate the resilience of current methodologies against contextual bias, we introduce a novel out-of- distribution (OOD) numerical question-answering (QA) dataset that features specific correlations between context and numbers in the training data, which are not present in the OOD test data. We evaluate the robustness of different numerical encoding and decoding methods …

Getting To The Point: Contrasting Directness And Warmth In Motivational Embodied Conversational Agents, Michael O'Mahony, Cathy Ennis, Robert Ross

Conference papers

Enhancing long-term engagement with conversational agents remains a significant challenge. Controlling the perceived warmth or directness of an agent’s personality through the style of its generated text could be used to increase user likeability. This paper reports an investigation of a Wizard-of-Oz (WoZ) mediated study of two variants of a motivational embodied conversational agent to measure user perception of and attitudes towards warmth in interaction style. Results show a significant effect of users preferring an agent with a "more direct" personality for this scenario, though this effect is in many ways nuanced.

Climate Change Prediction Model Using Mcdm Technique Based On Neutrosophic Soft Functions With Aggregate Operators, Kainat Muniba, Muhammad Naveed Jafar, Asma Riffat, Jawaria Mukhtar, Adeel Saleem

Neutrosophic Systems with Applications

The increasing impact of climate change necessitates innovative approaches in modeling and prediction to mitigate its adverse effects. This paper introduces a novel methodology integrating Neutrosophic Soft Functions (NSFs) into climate change prediction frameworks. NSFs, a hybrid of Neutrosophic Set Theory and Soft Set Theory, provide a flexible framework for handling uncertain and imprecise information inherent in climate data. This study explores the application of NSFs in capturing the complex interplay of various climatic variables, including temperature, precipitation, humidity, and atmospheric pressure, thereby enhancing the accuracy and reliability of climate change predictions. By incorporating NSFs into existing predictive models, such …

Genixer : Empowering Multimodal Large Language Models As A Powerful Data Generator, Henry Hengyuan Zhao, Pan Zhou, Mike Zheng Shou

Research Collection School Of Computing and Information Systems

Multimodal Large Language Models (MLLMs) demonstrate exceptional problem-solving capabilities, but few research studies aim to gauge the ability to generate visual instruction tuning data. This paper proposes to explore the potential of empowering MLLMs to generate data independently without relying on GPT-4. We introduce Genixer, a comprehensive data generation pipeline consisting of four key steps: (i) instruction data collection, (ii) instruction template design, (iii) empowering MLLMs, and (iv) data generation and filtering. Additionally, we outline two modes of data generation: task-agnostic and task-specific, enabling controllable output. We demonstrate that a synthetic VQA-like dataset trained with LLaVA1.5 enhances performance on 10 …

Enhancing Stance Classification On Social Media Using Quantified Moral Foundations, Hong Zhang, Quoc-Nam Nguyen, Prasanta Bhattacharya, Wei Gao, Liang Ze Wong, Brandon Siyuan Loh, Joseph J. P. Simons, Jisun An

Research Collection School Of Computing and Information Systems

This study enhances stance detection on social media by incorporating deeper psychological attributes, specifically individuals’ moral foundations. These theoretically-derived dimensions aim to provide an interpretable profile of an individual’s moral concerns which, in recent work, has been linked to behaviour in a range of domains including society, politics, health, and the environment. In this paper, we investigate how moral foundation dimensions can contribute to detecting an individual’s stance on a given target. Specifically, we incorporate moral foundation features extracted from text, along with semantic features, to classify stances at both message-and user-levels using traditional machine learning and Large Language Models …

Granular3d: Delving Into Multi-Granularity 3d Scene Graph Prediction, Kaixiang Huang, Jingru Yang, Jin Wang, Shengfeng He, Zhan Wang, Haiyan He, Qifeng Zhang, Guodong Lu

Research Collection School Of Computing and Information Systems

This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intricate 3D scenes, largely due to a focus on global scene processing and single-scale feature extraction. To overcome these limitations, we introduce Granular3D, a novel approach that shifts the focus towards multi-granularity analysis by predicting relation triplets from specific sub-scenes. One key is the Adaptive Instance Enveloping Method (AIEM), which establishes an approximate envelope structure around irregular instances, providing shape-adaptive local point cloud sampling, thereby …

Health Benefits And Adverse Effects Of Kratom: A Social Media Text-Mining Approach, Abdullah Wahbeh, Mohammad A. Al-Ramahi, Omar El-Gayar, Tareq Nasralah, Ahmed El Noshokaty

All Faculty Scholarship (Archived)

Background: Kratom is a substance that alters one’s mental state and is used for pain relief, mood enhancement, and opioid withdrawal, despite potential health risks. In this study, we aim to analyze the social media discourse about kratom to provide more insights about kratom’s benefits and adverse effects. Also, we aim to demonstrate how algorithmic machine learning approaches, qualitative methods, and data visualization techniques can complement each other to discern diverse reactions to kratom’s effects, thereby complementing traditional quantitative and qualitative methods. Methods: Social media data were analyzed using the latent Dirichlet allocation (LDA) algorithm, PyLDAVis, and t-distributed stochastic neighbor …

Scene Text Detection And Recognition Via Discriminative Representation, Liang Zhao

Theses and Dissertations

Scene texts refer to arbitrary text presented in an image captured by a camera in the real world. The tasks of scene text detection and recognition from complex images play a crucial role in computer vision, with potential applications in scene understanding, information retrieval, robotics, autonomous driving, etc. Despite the notable progress made by existing deep-learning methods, achieving accurate text detection and recognition remains challenging for robust real-world applications. The challenges in scene text detection and recognition stem from: 1) diverse text shapes, fonts, colors, styles, layouts, etc.; 2) countless combinations of characters with unfixed attributes for complete detection, coupled …

On Parallelization Of Graph Algorithms, Performance Modelling And Autonomous 3d Printable Object Synthesis, Shams-Ul-Haq Syed

Theses and Dissertations

The degree of hardware level parallelism offered by today’s GPU architecture makes it ideal for problem domains with massive inherent parallelism potential, fields such as computer vision, image processing, graph theory and graph computations. We have identified three problem areas for purpose of this research dissertation, under the umbrella of performance improvement by harnessing the power of GPUs for novel applications. The first area is concerned with k-vertex connectivity in graph theory, the second area deals performance evaluation using extended roofline models for GPU parallel applications and finally the third problem area is related to synthesis 3D printable objects from …

Enhancing Clinical Relevance Of Pretrained Language Models Through Integration Of External Knowledge: Case Study On Cardiovascular Diagnosis From Electronic Health Records, Qiuhao Lu, Andrew Wen, Thien Nguyen, Hongfang Liu

Faculty, Staff and Student Publications

Background: Despite their growing use in health care, pretrained language models (PLMs) often lack clinical relevance due to insufficient domain expertise and poor interpretability. A key strategy to overcome these challenges is integrating external knowledge into PLMs, enhancing their adaptability and clinical usefulness. Current biomedical knowledge graphs like UMLS (Unified Medical Language System), SNOMED CT (Systematized Medical Nomenclature for Medicine-Clinical Terminology), and HPO (Human Phenotype Ontology), while comprehensive, fail to effectively connect general biomedical knowledge with physician insights. There is an equally important need for a model that integrates diverse knowledge in a way that is both unified and compartmentalized. …

Ee-Lce: An Event Extraction Framework Based On Llm-Generated Cot Explanation, Yanhua Yu, Yuanlong Wang, Yunshan Ma, Jie Li, Kangkang Lu, Zhiyong Huang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Generative models have been widely used in event extraction. However, the interpretability of event extraction has not been fully investigated. In this paper, we propose an Event Extraction framework based on LLM-generated CoT Explanation EE-LCE, which can generate chain-of-thought-style (CoT-style) explanations for events. To this end, we provide each sample of event datasets with an explanation of the reasoning process using a large language model (LLM) GPT-3.5, and fine-tune the Flan-T5 lightweight language model (LM) supervised by the augmented dataset, enhancing both interpretability and performance of the event extraction. Moreover, we use a prefix tree (trie) to normalize the decoding …

Cybersecurity In Education, Rahima Shelim

Theses, Dissertations and Culminating Projects

Cybersecurity is becoming increasingly important as we rely more on digital devices and programs to conduct our daily lives, including the transfer and storage of personal information. According to research, one of the most critical stages in improving cybersecurity is to implement an effective security awareness program. In this work, we seek to understand the existing level of security knowledge among college students, industry professionals and create a module to help raise the awareness. Our module's primary elements are interaction and the display of alarming effects of reckless cyber behaviors among common Internet/technology users. This report presents a simple systematic …

Chain-Of-Exemplar: Enhancing Distractor Generation For Multimodal Educational Question Generation, Haohao Luo, Yang Deng, Ying Shen, See-Kiong Ng, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Multiple-choice questions (MCQs) are important in enhancing concept learning and student engagement for educational purposes. Despite the multimodal nature of educational content, current methods focus mainly on text-based inputs and often neglect the integration of visual information. In this work, we study the problem of multimodal educational question generation, which aims at generating subject-specific educational questions with plausible yet incorrect distractors based on multimodal educational content. To tackle this problem, we introduce a novel framework, named Chain-of-Exemplar (CoE), which utilizes multimodal large language models (MLLMs) with Chain-of-Thought reasoning to improve the generation of challenging distractors. Furthermore, CoE leverages three-stage contextualized …

Interpretable Tensor Fusion, Saurabh Varshneya, Antoine Ledent, Philipp Liznerski, Andriy Balinskyy, Purvanshi Mehta, Waleed Mustafa, Marius Kloft

Research Collection School Of Computing and Information Systems

Conventional machine learning methods are predominantly designed to predict outcomes based on a single data type. However, practical applications may encompass data of diverse types, such as text, images, and audio. We introduce interpretable tensor fusion (InTense), a multimodal learning method training a neural network to simultaneously learn multiple data representations and their interpretable fusion. InTense can separately capture both linear combinations and multiplicative interactions of the data types, thereby disentangling higher-order interactions from the individual effects of each modality. InTense provides interpretability out of the box by assigning relevance scores to modalities and their associations, respectively. The approach is …

Nonfactoid Question Answering As Query-Focused Summarization With Graph-Enhanced Multihop Inference, Yang Deng, Wenxuan Zhang, Weiwen Xu, Ying Shen, Wai Lam

Research Collection School Of Computing and Information Systems

Nonfactoid question answering (QA) is one of the most extensive yet challenging applications and research areas in natural language processing (NLP). Existing methods fall short of handling the long-distance and complex semantic relations between the question and the document sentences. In this work, we propose a novel query-focused summarization method, namely a graph-enhanced multihop query-focused summarizer (GMQS), to tackle the nonfactoid QA problem. Specifically, we leverage graph-enhanced reasoning techniques to elaborate the multihop inference process in nonfactoid QA. Three types of graphs with different semantic relations, namely semantic relevance, topical coherence, and coreference linking, are constructed for explicitly capturing the …

Style: Improving Domain Transferability Of Asking Clarification Questions In Large Language Model Powered Conversational Agents, Yue Chen, Chen Huang, Yang Deng, Wenqiang Lei, Dingnan Jin, Jia Liu, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Equipping a conversational search engine with strategies regarding when to ask clarification questions is becoming increasingly important across various domains. Attributing to the context understanding capability of LLMs and their access to domain-specific sources of knowledge, LLM-based clarification strategies feature rapid transfer to various domains in a posthoc manner. However, they still struggle to deliver promising performance on unseen domains, struggling to achieve effective domain transferability. We take the first step to investigate this issue and existing methods tend to produce one-size-fits-all strategies across diverse domains, limiting their search effectiveness. In response, we introduce a novel method, called STYLE, to …

On The Multi-Turn Instruction Following For Conversational Web Agents, Yang Deng, Xuan Zhang, Wenxuan Zhang, Yifei Yuan, See-Kiong Ng, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Web agents powered by Large Language Models (LLMs) have demonstrated remarkable abilities in planning and executing multi-step interactions within complex web-based environments, fulfilling a wide range of web navigation tasks. Despite these advancements, the potential for LLM-powered agents to effectively engage with sequential user instructions in real-world scenarios has not been fully explored. In this work, we introduce a new task of Conversational Web Navigation, which necessitates sophisticated interactions that span multiple turns with both the users and the environment, supported by a specially developed dataset named Multi-Turn Mind2Web (MT-Mind2Web). To tackle the limited context length of LLMs and the …