Digital Commons Network^™

Vibemus: Proactive Agentic System For Music Personalization, Zhiliang Guo, Teng Tu, Yunshan Ma, Xun Yang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) enable diverse forms of AI-assisted creation, yet they often struggle to bridge the preference-articulation gap: users may provide incomplete or vague intentions or lack the vocabulary to specify what they want, yielding outputs misaligned with true preferences. To address this gap and facilitate music creation in a vibe-centric environment, we introduce VibeMus, a proactive agentic system built on open-source components. The system engages in multi-turn dialogue to progressively determine the music’s emotion, genre, lyrics, and other aspects before generation. Simulated evaluations show that proactive clarification improves alignment with users’ intended nuances. Our approach is training-free, leveraging …

Scientific Multimodal Summarization : Integrating Knowledge Across Textual, Visual And Auditory Content, Zusheng Tan

Lingnan Theses (MPhil & PhD)

As scientific publications increasingly incorporate multimodal content, ranging from textual descriptions to figures, tables, presentation videos, and audio, there is a growing need for summarization systems that can effectively process and integrate information across these diverse modalities.

This work presents a comprehensive exploration of Scientific Multimodal Summarization, introducing a series of novel architectures and datasets aimed at advancing this emerging field. 1): We begin by introducing CMT-Sum, which integrates multimodal scientific source content (i.e., primarily paper text and figures) to generate high-quality textual summaries and identify representative graphical abstracts. We refer to this task as Scientific Multimodal Summarization with …

A Dynamic Hierarchical Attention Framework For Multimodal Malware Detection, Tamanna Nazmin

Graduate Theses and Dissertations

The increasing use of Android in the worldwide mobile ecosystem has come along with a significant increase in advanced malware, highlighting the critical necessity for efficient, scalable, and adaptable detection systems. Despite recent advancements in machine learning improving malware detection, the majority of current solutions are limited to one, two, or three data modalities, hence neglecting the comprehensive behavioral spectrum of contemporary multi-vector threats. This thesis presents the first comprehensive multimodal framework for Android malware detection, which combines textual, time-series (temporal), graph-based (structural), and visual information using an innovative hierarchical attention mechanism and Dynamic Fusion Controller(DFC). Our methodology consistently classifies …

Lighttransfer: Your Long-Context Llm Is Secretly A Hybrid Model With Effortless Adaptation, Xuan Zhang, Fengzhuo Zhang, Cunxiao Du, Chao Du, Tianyu Pang, Wei Gao, Min Lin

Research Collection School Of Computing and Information Systems

Scaling language models to handle longer contexts introduces substantial memory challenges due to the growing cost of key-value (KV) caches. Motivated by the efficiency gains of hybrid models and the broad availability of pretrained large transformer backbones, we explore transitioning transformer models into hybrid architectures for a more efficient generation. In this work, we propose LightTransfer, a lightweight method that transforms models such as LLaMA into hybrid variants. Our approach identifies lazy layers -- those focusing on recent or initial tokens -- and replaces their full attention with streaming attention. This transformation can be performed without any training for long-context …

Towards Multimodal Emotional Support Conversation Systems, Yuqi Chu, Lizi Liao, Zhiyuan Zhou, Chong-Wah Ngo, Richang Hong

Research Collection School Of Computing and Information Systems

The integration of conversational artificial intelligence (AI) into mental health care promises a new horizon for therapist-client interactions, aiming to closely emulate the depth and nuance of human conversations. Despite the potential, the current landscape of conversational AI is markedly limited by its reliance on single-modal data, constraining the systems’ ability to empathize and provide effective emotional support. This limitation stems from a paucity of resources that encapsulate the multimodal nature of human communication essential for therapeutic counseling. To address this gap, we introduce the Multimodal Emotional Support Conversation (MESC) dataset, a first-of-its-kind resource enriched with comprehensive annotations across text, …

Guiding Multiple Remote Users In Physical Tasks With Language-Driven Robotic Telepresence, Ruyi Li, Jingfei Guo, Xinyi Zhang, Xuji Zhang, Zeqing Li, Jiannan Li, Jiangtao Gong

Research Collection School Of Computing and Information Systems

Remote assistance through robotic telepresence could involve both control and memory challenges, particularly in one expert to multiple workers situation. In this work, we proposed a novelty language-driven interface to facilitate remote collaboration through telepresence robots. Through operations and maintenance expert interviews and a scenario simulation study, we identified key pain points in executing one-expert-multiple-workers remote guidance using the telepresence robot and proposed two design goals, which together consist of five sub-design goals with corresponding features. These features were integrated into a standard telepresence robot, resulting in the development of a Collaborative LLM-based Embodied Assistant Robot, named CLEAR Robot. A …

Developing Deep Learning Methods For Cognitive Impairment Detection Based On Non-Invasive Data, Muath Alsuhaibani

Electronic Theses and Dissertations

Cognitive impairment detection is on the rise to help reduce the burden of healthcare costs on institutions and individuals. Mild Cognitive Impairment (MCI) is an early stage of cognitive decline progressing to Alzheimer’s disease (AD) or AD-related Dementia (ADRD). Detecting the early stages of AD/ADRD is crucial for early interventions among older adults to mitigate cognitive decline over time. However, the current diagnostic methods are often costly and/or invasive, such as MRI and PET scans. Thus, the search for non-invasive and cost-effective screening tools for the early detection of cognitive impairment using speech, language, visual, and motor data is growing. …

Impact Of Retrieval Augmented Generation And Large Language Model Complexity On Undergraduate Exams Created And Taken By Ai Agents, Erick S. Tyndall, Colleen Gayheart, Alexandre Some, Joseph Genz, Torrey J. Wagner, Brent T. Langhals

Faculty Publications

The capabilities of large language models (LLMs) have advanced to the point where entire textbooks can be queried using retrieval-augmented generation (RAG), enabling AI to integrate external, up-to-date information into its responses. This study evaluates the ability of two OpenAI models, GPT-3.5 Turbo and GPT-4 Turbo, to create and answer exam questions based on an undergraduate textbook. 14 exams were created with four true-false, four multiple-choice, and two short-answer questions derived from an open-source Pacific Studies textbook. Model performance was evaluated with and without access to the source material using text-similarity metrics such as ROUGE-1, cosine similarity, and word embeddings. …

Modeling The Importance Of Life Exposure Factors On Memory Performance In Diverse Older Adults: A Machine Learning Approach, Evan Fletcher, Marianne Chanti-Ketterl, Emily Hokett, Yi Lor, Umesh Venkatesan, Ruijia Chen, Omonigho M. Bubu, Rachel Whitmer, Paola Gilsanz, Zvinka Z. Zlatar

Moss-Magee Rehabilitation Papers

INTRODUCTION: Many health life exposure factors (LEFs) influence cognitive decline and dementia incidence, but their relative importance to episodic memory (an early indicator of cognitive decline) among diverse older adults is unclear. We used machine learning to rank LEFs for memory performance in a large and diverse US cohort.

METHODS: Kaiser Healthy Aging and Diverse Life Experiences (KHANDLE) and Study of Healthy Aging in African Americans (STAR), participants underwent neuropsychological testing and answered questionnaires about multiple LEFs. XGBoost and Shapley Additive exPlanation values ranked the importance of factors influencing cross-sectional episodic memory in the full sample and by sex and …

Towards Scalable Schema Mapping Using Large Language Models, Christopher Buss, Mahdis Safari, Arash Termehchy, David Maier, Stefan Lee

Computer Science Faculty Publications and Presentations

The growing need to integrate information from many diverse sources poses significant scalability challenges for data integration systems. These systems often rely on manually written schema mappings, which are complex and costly to maintain. While recent advances suggest that large language models (LLMs) can assist in automating schema mapping, key challenges remain. We motivate future research in schema mapping generation by highlighting key challenges, presenting a competitive bidirectional schema matching pipeline, and exploring the limitations of current methods for generating more complex mappings.

Investigating Information Extraction And Language Models In Medical Domain Text Processing, Pouyan Nahed

UNLV Theses, Dissertations, Professional Papers, and Capstones

This dissertation demonstrates that carefully adapted language-model pipelines can transform unstructured clinical-trial and pharmacological prose into reliable, low-latency structured data. Four interconnected studies support this claim.Tri-AL platform. An open-source dashboard ingests all 440 k+ ClinicalTrials.gov records—including every historical revision—into a normalized schema and parses the 20 GB XML archive over 10x faster than a BeautifulSoup baseline, while exposing hooks for demographic analytics and supporting integration of user-defined modules. Clinical trial summarization. An encoder–decoder model is trained on 57k description–summary pairs to condense clinical trials into a few sentences. ROUGE evaluation shows a 20% improvement over the baseline, while graph-based evaluation …

Quantified Possibility Neutrosophic Soft Set Based Decision Support System For Enhanced Accuracy In Sustainable Supplier Selection Decision Analysis, Neha Andaleeb Khalid, Muhammad Saeed

Neutrosophic Systems with Applications

To deal with the concepts of vulnerability, ambiguity, and indeterminacy that are typical in complex decision analysis contexts, neutrosophic set-like structures are frequently used. Although neutrosophic sets are particularly good at handling indeterminate circumstances, indeterminacy plays a part in making the decision making process imprecise and ambiguous. This paper develops a more advanced technique to improve the accuracy of decision analysis problems: the Quantified Possibility Neutrosophic Soft Set Decision Support System (Q^t PNSSDSS). In order to reduce the element of indeterminacy present in conventional neutrosophic sets, the proposed DSS is based on the Quantified Neutrosophic Set, which employs a …

Some Properties Of Neutrosophic Cubic Hypersoft Sets, Lubna Nayab, Muhammad Gulistan, Fawad Hussain

Neutrosophic Systems with Applications

This study addresses the limitations of NCS in managing uncertainties associated with multiple attributes and their further bifurcation. To address this challenge, we propose a generalization of the neutrosophic cubic soft set, introducing the concept of "neutrosophic cubic hyper soft set." Within this framework, we define internal and external neutrosophic cubic hypersoft sets, along with operations such as P-intersection, P-union, P-restricted union, P-extended intersection, P-OR operator, P-AND operator, R-intersection, R-union, R-restricted union, R-extended intersection, R-OR operator, R-AND operator, complement, and relative complement of neutrosophic cubic hyper soft sets. The study explores and presents relevant results, demonstrating the enhanced capability of …

Digital Trading Platform Selection Under Neutrosophic Numbers And Multi-Criteria Decision-Making Methodology: An Illustrative Example, Eman Sayed, Karam M. Sallam, Ibrahim Alrashdi

Neutrosophic Systems with Applications

Uncertainty, vagueness, and incomplete information are pervasive in real-world decision-making scenarios, particularly in multi-criteria decision-making (MCDM) contexts involving expert judgments. To address these challenges, this study introduces a comprehensive decision-making framework that integrates Triangular Neutrosophic Numbers (TNNs) with the Weighted Aggregated Sum Product Assessment (WASPAS) method. The framework begins by capturing expert evaluations using TNNs, which effectively represent the degrees of truth, indeterminacy, and falsity inherent in subjective assessments. These evaluations are then transformed into crisp values using a novel score function that preserves the embedded uncertainty. The resulting decision matrices are aggregated into a unified structure to ensure consistency …

Topnet R1: A Multi-Stage Ai Framework For Topic Discovery In Scientific Abstracts, Md Elias Hossain

Theses and Dissertations

Scientific abstracts are rich sources of knowledge, yet extracting meaningful topics remains challenging due to limitations in existing topic modeling techniques. Traditional methods often struggle with interpretability, scalability, and contextual understanding. To overcome these issues, we introduce TopNet R1, a multi-stage ensemble framework that integrates traditional topic models with contextual embeddings from Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer 4 (GPT-4) large language models (LLMs). Top- Net R1 operates in three phases: (1) topic generation using Latent Dirichlet Allocation (LDA), Non-Negative Matrix Factorization (NMF), Latent Semantic Analysis (LSA), and Hierarchical Dirichlet Process (HDP); (2) LLM-assisted pattern recognition …

Unboxing The Black Box: Graph-Based Interpretability In Transformer Models, Ramak Nassiri

Theses and Dissertations

This thesis introduces a novel interpretability framework for transformer encoder models by hypothesizing that their internal embedding updates define a state transition system. We propose that transformers implicitly learn token-to-token influence dynamics, which can be analyzed using graph-theoretic methods. To validate this, we construct transition graphs from embedding changes and rank token importance using the PageRank algorithm. We develop two transformer models from scratch: a self-supervised model that incorporates serotype tokens to learn contextualized embeddings, and a supervised model initialized with these embeddings. Cosine similarity analysis of their token influence patterns reveals strong structural alignment, with values exceeding 93%. This …

Imputation Via Domain Adaptation: Rethinking Variable Subset Forecasting From Knowledge Transfer, Runchang Liang, Qi Hao, Yue Gao, Kunpeng Liu, Lu Jiang, Pengyang Wang, Minghao Yin

Computer Science Faculty Publications and Presentations

Multivariate time series forecasting in practical deployment faces a critical challenge termed Variable Subset Forecasting (VSF), where certain variables accessible during training are entirely missing during inference. This creates a stark discrepancy between the training (source domain with full variables) and inference (target domain with partial variables) environments, disrupting cross-variable dependencies and fragmenting global temporal patterns. Existing imputation methods, limited to transferring local knowledge (e.g., temporal neighbors or pairwise correlations), fail to capture essential global dynamics, leading to severe performance degradation under distribution shifts. To address these challenges, we redefine VSF as a cross-domain knowledge transfer problem and propose VIDA, …

Mpo: Multilingual Safety Alignment Via Reward Gap Optimization, Weixiang Zhao, Yulin Hu, Yang Deng, Tongtong Wu, Wenxuan Zhang, Jiahe Guo, An Zhang, Yanyan Zhao, Bing Qin, Tat-Seng Chua, Ting Liu

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have become increasingly central to AI applications worldwide, necessitating robust multilingual safety alignment to ensure secure deployment across diverse linguistic contexts. Existing preference learning methods for safety alignment, such as RLHF and DPO, are primarily monolingual and struggle with noisy multilingual data. To address these limitations, we introduce Multilingual reward gaP Optimization (MPO), a novel approach that leverages the well-aligned safety capabilities of the dominant language (e.g., English) to improve safety alignment across multiple languages. MPO directly minimizes the reward gap difference between the dominant language and target languages, effectively transferring safety capabilities while preserving the …

Think Both Ways: Teacher-Student Bidirectional Reasoning Enhances Mcq Generation And Distractor Quality, Yimiao Qiu, Yang Deng, Quanming Yao, Zhimeng Zhang, Zhiang Dong, Chang Yao, Jingyuan Chen

Research Collection School Of Computing and Information Systems

Generating high-quality Multiple Choice Questions (MCQs) remains challenging for educational tools due to the need for contextual relevance and plausible distractors. Existing methods still struggle with these dual requirements, leading to questions that lack depth and distractors that are either too obvious or irrelevant. In this paper, we propose BiFlow, a novel framework that integrates bidirectional reasoning perspectives: teacher reasoning generates contextually relevant questions and plausible distractors, while student reasoning evaluates question clarity and the misleading nature of the distractors. To further enhance reasoning, we introduce PathFinder, a mechanism that employs breadth-first search and Chainof-Thought (CoT) strategies to explore diverse …

How Developers Use Type-System Related Programming Language Features, Samuel W. Flint

Dissertations and Doctoral Documents, University of Nebraska-Lincoln, 2023–

Optional type annotations are a popular feature of programming languages that allow developers to omit explicit type information in code while, in some cases, retaining many of the benefits of static typing, such as in-code documentation, improved detection of type errors, or enforcement of code properties. However, how developers use and understand optional type annotations is not clear. The focus of this dissertation is to understand the use and comprehension of optional type annotations.

Optional type annotations are examined through four lenses: first, by examining the evolution of usage in a statically typed programming language (Kotlin, the default language for …

How Developers Use Type-System Related Programming Language Features, Samuel W. Flint

School of Computing: Dissertations, Theses, and Student Research

Optional type annotations are a popular feature of programming languages that allow developers to omit explicit type information in code while, in some cases, retaining many of the benefits of static typing, such as in-code documentation, improved detection of type errors, or enforcement of code properties. However, how developers use and understand optional type annotations is not clear. The focus of this dissertation is to understand the use and comprehension of optional type annotations.

Optional type annotations are examined through four lenses: first, by examining the evolution of usage in a statically typed programming language (Kotlin, the default language for …

Computational Fact-Checking With Limited Resources, Fengzhu Zeng

Dissertations and Theses Collection (Open Access)

The rapid dissemination of information through online platforms has sparked widespread concern about the propagation of misinformation. Manual fact-checking by pro- fessional fact-checkers is time-consuming and lacks scalability to address the vast volume of daily information. Consequently, computational fact-checking, driven by automated techniques in natural language processing (NLP), has garnered interest as
a potential solution. However, computational fact-checking faces critical challenges limited resources, particularly due to the issues of data scarcity and computing resource constraints. One key challenge is data scarcity, which arises from the constant generation of new information and emerging events on social media. This scarcity manifests in …

Dreamanime: Learning Style-Identity Textual Disentanglement For Anime And Beyond, Chenshu Xu, Yangyang Xu, Huaidong Zhang, Xuemiao Xu, Shengfeng He

Research Collection School Of Computing and Information Systems

Text-to-image generation models have significantly broadened the horizons of creative expression through the power of natural language. However, navigating these models to generate unique concepts, alter their appearance, or reimagine them in unfamiliar roles presents an intricate challenge. For instance, how can we exploit language-guided models to transpose an anime character into a different art style, or envision a beloved character in a radically different setting or role? This paper unveils a novel approach named DreamAnime, designed to provide this level of creative freedom. Using a minimal set of 2-3 images of a user-specified concept such as an anime character …

Focus: Evaluating Pre-Trained Vision-Language Models On Underspecification Reasoning, Kankan Zhou, Yibin Lai, Kyriakos Mouratidis, Jing Jiang

Research Collection School Of Computing and Information Systems

Humans possess a remarkable ability to interpret underspecified ambiguous statements by inferring their meanings from contexts such as visual inputs. This ability, however, may not be as developed in recent pre-trained visionlanguage models (VLMs). In this paper, we introduce a novel probing dataset called FOCUS to evaluate whether state-of-the-art VLMs have this ability. FOCUS consists of underspecified sentences paired with image contexts and carefully designed probing questions. Our experiments reveal that VLMs still fall short in handling underspecification even when visual inputs that can help resolve the ambiguities are available. To further support research in underspecification, FOCUS will be released …

Evowiki: Evaluating Llms On Evolving Knowledge, Wei Tang, Yixin Cao, Yang Deng, Jiahao Ying, Bo Wang, Yizhe Yang, Yuyue Zhao, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Yong Liao

Research Collection School Of Computing and Information Systems

Knowledge utilization is a critical aspect of LLMs, and understanding how they adapt to evolving knowledge is essential for their effective deployment. However, existing benchmarks are predominantly static, failing to capture the evolving nature of LLMs and knowledge, leading to inaccuracies and vulnerabilities such as contamination. In this paper, we introduce EvoWiki, an evolving dataset designed to reflect knowledge evolution by categorizing information into stable, evolved, and uncharted states. EvoWiki is fully auto-updated, enabling precise evaluation of continuously changing knowledge and newly released LLMs. Through experiments with Retrieval-Augmented Generation (RAG) and Continual Learning (CL), we evaluate how effectively LLMs adapt …

Xfinbench: Benchmarking Llms In Complex Financial Problem Solving And Reasoning, Zhihan Zhang, Yixin Cao, Lizi Liao

Research Collection School Of Computing and Information Systems

Solving financial problems demands complex reasoning, multimodal data processing, and a broad technical understanding, presenting unique challenges for current large language models (LLMs). We introduce **XFinBench**, a novel benchmark with 4,235 examples designed to evaluate LLM’s ability in solving comple**X**, knowledge-intensive **Fin**ancial problems across diverse graduate-level finance topics with multi-modal context. We identify five core capabilities of LLMs using XFinBench, i.e., _terminology understanding_, _temporal reasoning_, _future forecasting_, _scenario planning_, and _numerical modelling_. Upon XFinBench, we conduct extensive experiments on 18 leading models. The result shows that o1 is the best-performing text-only model with an overall accuracy of 67.3%, but still …

Taclr: A Scalable And Efficient Retrieval-Based Method For Industrial Product Attribute Value Identification, Yindu Su, Huike Zou, Lin Sun, Ting Zhang, Haiyang Yang, Chen Li Yu, David Lo, Qingheng Zhang, Shuguang Han, Jufeng Chen

Research Collection School Of Computing and Information Systems

Product Attribute Value Identification (PAVI) involves identifying attribute values from product profiles, a key task for improving product search, recommendation, and business analytics on e-commerce platforms. However, existing PAVI methods face critical challenges, such as inferring implicit values, handling outof-distribution (OOD) values, and producing normalized outputs. To address these limitations, we introduce Taxonomy-Aware Contrastive Learning Retrieval (TACLR), the first retrieval-based method for PAVI. TACLR formulates PAVI as an information retrieval task by encoding product profiles and candidate values into embeddings and retrieving values based on their similarity. It leverages contrastive training with taxonomy-aware hard negative sampling and employs adaptive inference …

Predicting Music Origin With Deep Learning, Fruzsina Ladanyi

Electronic Theses, Projects, and Dissertations

This project explores the usage of a late fusion deep learning architecture to predict the geographic origin of music. Mel-Frequency Cepstral Coefficients (MFCCs) and the language of the music sample are used as features. MFCCs were extracted from audio files to capture sound features. The language was identified using OpenAI’s Whisper model to provide additional context. A late fusion neural network architecture combining Long Short-Term Memory (LSTM) layers for sequential MFCC input and dense layers for non-sequential language features were employed to support both classification and regression tasks. The classification model achieved an accuracy of 33.03% across 56 countries or …

How To Do Things With Little Talking Tubes: Nonideal Speech Acts In The Digital Age, Anthony Holdier

Graduate Theses and Dissertations

In this work, I develop a view about what it means to share a social or conversational context with others, as well as what is normatively entailed by doing so. Working from a broadly Austinian perspective about the moral foundations of language use and how we use words to position ourselves within social space, as well as from a generally Stalnakerian social ontology (demarcating groups by dint of aligned or overlapping sets of commitments), I present three papers demonstrating how nonidealized, ordinary language philosophy can make sense of complex, real-world phenomena. Through analyses of heretical utterances, context collapse, and chatbot …

Faithfulrag: Fact-Level Conflict Modeling For Context-Faithful Retrieval-Augmented Generation, Qinggang Zhang, Zhishang Xiang, Yilin Xiao, Le Wang, Junhui Li, Xinrun Wang, Jinsong Su

Research Collection School Of Computing and Information Systems

Large language models (LLMs) augmented with retrieval systems have demonstrated significant potential in handling knowledge-intensive tasks. However, these models often struggle with unfaithfulness issues, generating outputs that either ignore the retrieved context or inconsistently blend it with the LLM’s parametric knowledge. This issue is particularly severe in cases of knowledge conflict, where the retrieved context conflicts with the model’s parametric knowledge. While existing faithful RAG approaches enforce strict context adherence through well-designed prompts or modified decoding strategies, our analysis reveals a critical limitation: they achieve faithfulness by forcibly suppressing the model’s parametric knowledge, which undermines the model’s internal knowledge structure …