Computer Sciences Commons^™

Scientific Multimodal Summarization : Integrating Knowledge Across Textual, Visual And Auditory Content, Zusheng Tan

Lingnan Theses (MPhil & PhD)

As scientific publications increasingly incorporate multimodal content, ranging from textual descriptions to figures, tables, presentation videos, and audio, there is a growing need for summarization systems that can effectively process and integrate information across these diverse modalities.

This work presents a comprehensive exploration of Scientific Multimodal Summarization, introducing a series of novel architectures and datasets aimed at advancing this emerging field. 1): We begin by introducing CMT-Sum, which integrates multimodal scientific source content (i.e., primarily paper text and figures) to generate high-quality textual summaries and identify representative graphical abstracts. We refer to this task as Scientific Multimodal Summarization with …

Machine Learning-Based Electric Vehicle Charging Demand Forecasting: A Systematized Literature Review, Maher Alaraj, Mohammed Radi, Elaf Alsisi, Munir Majdalawieh, Mohamed Darwish

All Works

The transport sector significantly contributes to global greenhouse gas emissions, making electromobility crucial in the race toward the United Nations Sustainable Development Goals. In recent years, the increasing competition among manufacturers, the development of cheaper batteries, the ongoing policy support, and people’s greater environmental awareness have consistently increased electric vehicles (EVs) adoption. Nevertheless, EVs charging needs—highly influenced by EV drivers’ behavior uncertainty—challenge their integration into the power grid on a massive scale, leading to potential issues, such as overloading and grid instability. Smart charging strategies can mitigate these adverse effects by using information and communication technologies to optimize EV charging …

A Study Of The Privacy Paradox Amongst Young Adults In The United Arab Emirates, Lena Yuryna Connolly, Michael Lang, Justin Giboney

All Works

The rapid digitalisation of society has significantly increased the collection and processing of personal data, raising concerns about individuals’ privacy. The privacy paradox, where individuals express privacy concerns yet continue to disclose personal information, has been widely studied in Western and Asian contexts, but remains underexplored in the Arab world. This study investigates privacy attitudes and behaviors in the United Arab Emirates (UAE), a region at the crossroads of traditional Islamic values and Western influences. Using survey data from 216 Emirati university students, we tested a model that incorporates five constructs: peer interaction and influence, desire for privacy, privacy concerns, …

Advancing U.S. Competitiveness In Agentic Gen Ai: A Strategic Framework For Interoperability And Governance, Satyadhar Joshi

Harrisburg University Other Works

Abstract : The rapid evolution of artificial intelligence has given rise to agentic AI systems—autonomous entities capable of perceiving their environment, making decisions, and executing actions with minimal human intervention. This work provides a systematic analysis of agentic AI frameworks, governance models, and implementation strategies. Drawing on a comprehensive review of the literature, we examine the current state of agentic AI technologies, highlight key challenges in governance, security, and ethical oversight, and compare architectural frameworks for responsible deployment. Our results, illustrated through detailed framework comparisons and governance analyses, demonstrate that while agentic AI holds transformative potential across multiple sectors, notable …

Bridging The Great Wall: China’S Evolving Cross-Border Data Flow Policies And Implications For Global Data Governance, Sheng Zhang, Henry S. Gao

Research Collection Yong Pung How School Of Law

Despite the rapid expansion of the digital economy, the global regulatory framework for data flows remains fragmented, with countries adopting divergent approaches shaped by their own regulatory priorities. As a key player in the Internet economy, China’s approach to cross-border data flows (CBDF) not only defines its domestic digital landscape but also influences emerging global norms. This paper takes a comprehensive view of the evolution of China’s CBDF regime, examining its development through both domestic and international lenses. Domestically, China’s regulation of CBDF has evolved from a security-first approach to one that seeks to balance security with economic development. This …

Optimal Abort Policy For Mission-Critical Systems Under Imperfect Condition Monitoring, Qiuzhuang Sun, Jiawen Hu, Zhi-Sheng Ye

Research Collection College of Integrative Studies

Although most on-demand mission-critical systems are engineered to be reliable to support critical tasks, occasional failures may still occur during missions. To increase system survivability, a common practice is to abort the mission before an imminent failure. We consider optimal mission abort for a system whose deterioration follows a general three-state (normal, defective, failed) semi-Markov chain. The failure is assumed self-revealed, whereas the healthy and defective states have to be inferred from imperfect condition-monitoring data. Because of the non-Markovian process dynamics, optimal mission abort for this partially observable system is an intractable stopping problem. For a tractable solution, we introduce …

Preference-Based Deep Reinforcement Learning For Historical Route Estimation, Boshen Pan, Yaoxin Wu, Zhiguang Cao, Yaqing Hou, Guangyu Zou, Qiang Zhang

Research Collection School Of Computing and Information Systems

Recent Deep Reinforcement Learning (DRL) techniques have advanced solutions to Vehicle Routing Problems (VRPs). However, many of these methods focus exclusively on optimizing distance-oriented objectives (i.e., minimizing route length), often overlooking the implicit drivers' preferences for routes. These preferences, which are crucial in practice, are challenging to model using traditional DRL approaches. To address this gap, we propose a preference-based DRL method characterized by its reward design and optimization objective, which is specialized to learn historical route preferences. Our experiments demonstrate that the method aligns generated solutions more closely with human preferences. Moreover, it exhibits strong generalization performance across a …

Exploring Parameter-Efficient Fine-Tuning Techniques For Code Generation With Large Language Models, Martin Weyssow, Xin Zhou, Kisub Kim, David Lo, Houari A. Sahraoui

Research Collection School Of Computing and Information Systems

Large language models (LLMs) demonstrate impressive capabilities to generate accurate code snippets given natural language intents in a zero-shot manner, i.e., without the need for specific fine-tuning. While prior studies have highlighted the advantages of fine-tuning LLMs, this process incurs high computational costs, making it impractical in resource-scarce environments, particularly for models with billions of parameters. To address these challenges, previous research explored in-context learning (ICL) and retrieval-augmented generation (RAG) as strategies to guide the LLM generative process with task-specific prompt examples. However, ICL and RAG introduce inconveniences, such as the need for designing contextually relevant prompts and the absence …

Shortcuts Everywhere And Nowhere: Exploring Multi-Trigger Backdoor Attacks, Yige Li, Jiabo He, Hanxun Huang, Jun Sun, Xingjun Ma, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Backdoor attacks have become a significant threat to the pre-training and deployment of deep neural networks (DNNs). Although numerous methods for detecting and mitigating backdoor attacks have been proposed, most rely on identifying and eliminating the “shortcut” created by the backdoor, which links a specific source class to a target class. However, these approaches can be easily circumvented by designing multiple backdoor triggers that create shortcuts everywhere and therefore nowhere specific. In this study, we explore the concept of Multi-Trigger Backdoor Attacks (MTBAs), where multiple adversaries leverage different types of triggers to poison the same dataset. By proposing and investigating …

Gti: Graph-Based Tree Index With Logarithm Updates For Nearest Neighbor Search In High-Dimensional Spaces, Ruoyao Ma, Yifan Zhu, Baihua Zheng, Lu Chen, Congcong Ge, Yunjun Gao

Research Collection School Of Computing and Information Systems

Nearest neighbor search (NNS) is fundamental for high-dimensional space retrieval and impacts various fields, such as pattern recognition, information retrieval, recommendation systems, and vector database management. Among existing NNS methods, graph-based methods often excel in query accuracy and efficiency. However, these methods face significant challenges, including high construction costs and difficulties with dynamic data updates. Recent efforts have focused on combining graph methods with hashing, quantization, and tree-based approaches to address these issues, but problems with large index sizes and update performance remain unresolved. In response, this paper proposes GTI, a novel, lightweight, and dynamic graph-based tree index for high-dimensional …

Deep Graph Anomaly Detection: A Survey And New Perspectives, Hezhe Qiao, Hanghang Tong, Nanyang Technological University, Irwin King, Charu Aggarwal, Guansong Pang

Research Collection School Of Computing and Information Systems

Graph anomaly detection (GAD), which aims to identify unusual graph instances (e.g., nodes, edges, subgraphs, or graphs), has attracted increasing attention in recent years due to its significance in a wide range of applications. Deep learning approaches, graph neural networks (GNNs) in particular, have been emerging as a promising paradigm for GAD, owing to its strong capability in capturing complex structure and/or node attributes in graph data. Considering the large number of methods proposed for GNN-based GAD, it is of paramount importance to summarize the methodologies and findings in the existing GAD studies, so that we can pinpoint effective model …

Learning Orientation Field For Osm-Guided Autonomous Navigation, Yuming Huang, Wei Gao, Zhiyuan Zhang, Maani Ghaffari, Dezhen Song, Cheng-Zhong Xu, Hui Kong

Research Collection School Of Computing and Information Systems

OpenStreetMap (OSM) has gained popularity recently in autonomous navigation due to its public accessibility, lower maintenance costs, and broader geographical coverage. However, existing methods often struggle with noisy OSM data and incomplete sensor observations, leading to inaccuracies in trajectory planning. These challenges are particularly evident in complex driving scenarios, such as at intersections or facing occlusions. To address these challenges, we propose a robust and explainable two-stage framework to learn an Orientation Field (OrField) for robot navigation by integrating LiDAR scans and OSM routes. In the first stage, we introduce a novel representation, OrField, which can provide orientations for each …

Storage Location Optimization In Automated Storage And Retrieval Systems: A Deep Reinforcement Learning Approach, Lingjun Wang, Aldy Gunawan, Pieter Vansteenwegen

Research Collection School Of Computing and Information Systems

This study investigates the optimization of storage location in automated storage and retrieval systems (AS/RS). We introduce an optimization approach based on the Deep Q-Network (DQN) algorithm to enhance warehouse task efficiency and minimize stacker travel during storage and retrieval. To accelerate the algorithm training process, we integrate a prioritized experience replay mechanism. Furthermore, we decouple action selection from value estimation within the DQN framework to address the issue of value overestimation. The proposed model is evaluated against three heuristic methods. The experimental results demonstrate that our approach significantly outperforms these baselines.

Privacy-Preserving Ridge Regression Over Encrypted Data Under Multiple Keys, Yanling Li, Junzuo Lai, Meng Sun, Beibei Song, Robert H. Deng

Research Collection School Of Computing and Information Systems

With the increase of private data being collected by data owners, it has been a trend for data owners to store the data on cloud computing platforms. The huge amounts of data in cloud servers bring fresh development opportunities to machine learning, which is applied to build a high-quality machine learning model based on a large training dataset. However, to ensure the privacy of data and facilitate retrieval, data owners often upload encrypted data under their public keys. But it creates new challenges for machine learning to learn a predictive model over these encrypted data under different keys. Most existing …

Static Analysis As A Feedback Loop: Enhancing Llm-Generated Code Beyond Correctness, Scott Blyth, Sherlock Licorish, Christoph Treude, Markus Wagner

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have demonstrated impressive capabilities in code generation, achieving high scores on benchmarks such as HumanEval and MBPP. However, these benchmarks primarily assess functional correctness and neglect broader dimensions of code quality, including security, reliability, readability, and maintainability. In this work, we systematically evaluate the ability of LLMs to generate high-quality code across multiple dimensions using the PythonSecurityEval benchmark. We introduce an iterative static analysis-driven prompting algorithm that leverages Bandit and Pylint to identify and resolve code quality issues. Our experiments with GPT-4o show substantial improvements: security issues reduced from >40% to 13%, readability violations from >80% …

Educator Perceptions Of Devops Teaching Recommendations And Their Alignment With Common Challenges, Marcelo Romulo Fernandes, Pablo Paiva, Samuel Lucas De Moura Ferino, Roberta Coelho, Christoph Treude, Eduardo Aranha, Uirá Kulesza

Research Collection School Of Computing and Information Systems

DevOps education presents unique pedagogical challenges due to the diversity of tools, rapid technological change, and the multidisciplinary nature of the field. Although previous work has proposed recommendations to address these challenges, it is unclear how educators perceive these recommendations and whether they align with the challenges encountered in practice. In this paper, we present a quantitative and qualitative methods study involving 11 DevOps educators who interacted with Improve, a tool that presents a curated set of educational challenges and recommendations derived from previous literature. Educators indicated which recommendations they already use, which they intend to use, and which challenges …

Studying Satd In Drone Systems With Human-Ai Collaboration, Leevi Rantala, Lwin Khin Shar, Mäntylä Mika V., Wei Minn, Naing Tun Yan

Research Collection School Of Computing and Information Systems

Background: Self-Admitted Technical Debt (SATD) refers to sub-optimal solutions that developers acknowledge within the source code. SATD research originated on Java projects but is expanding to other domains. We focus on SATD in drones, which are used for various critical tasks.Aims: The primary objective is to investigate SATD in drone systems. The second aim is to explore the integration of AI and human collaboration for SATD labelling and classification.Method: We conducted a sample study of SATD comments in drone systems (14 open source, 4 SDKs) to analyse the quantity and types of SATD comments present. Our study incorporates collaboration between …

Stable And Fair Cost Allocation In Platform-Enabled Lcl Consolidation, Pang Jin Tan, Shih-Fen Cheng

Research Collection School Of Computing and Information Systems

Many logistics platforms enable collaboration between agents to reduce costs, but determining fair pricing remains challenging when agents have pre-existing partnerships. This paper introduces a cooperative game theory framework to model platform-mediated collaboration, modeling the platform as an additional player. We present a novel characteristic function that distinguishes between partial collaborations (existing relationships) and full collaborations (platform-enabled). Using Shapley value, we derive fair cost allocations and platform charges that reflect each participant's contribution. We address stability concerns through an optimization model that minimizes platform subsidies while preventing profitable deviations. The framework is demonstrated through an application in freight forwarding for …

Dgl: Dynamic Global-Local Information Aggregation For Scalable Vrp Generalization With Self-Improvement Learning, Yubin Xiao, Yuesong Wu, Rui Cao, Di Wang, Zhiguang Cao, Xuan Wu, Peng Zhao, Yuanshu Li, You Zhou, Yuan Jiang

Research Collection School Of Computing and Information Systems

The Vehicle Routing Problem (VRP) is a critical combinatorial optimization problem with wide-reaching real-world applications, particularly in logistics, transportation. While neural network-based VRP solvers have shown impressive results on test instances similar to training data, their performance often degrades when faced with varying scales and unseen distributions, limiting their practical applicability. To overcome these limitations, we introduce DGL (Dynamic Global-Local Information Aggregation), a novel model that combines global and local information to effectively solve VRPs. DGL dynamically adjusts local node selections within a localized range, capturing local invariance across problems of different scales and distributions, thereby enhancing generalization. At the …

Coupling Category Alignment For Graph Domain Adaptation, Nan Yin, Xiao Teng, Zhiguang Cao, Mengzhu Wang

Research Collection School Of Computing and Information Systems

Graph domain adaptation (GDA), which transfers knowledge from a labeled source domain to an unlabeled target graph domain, attracts considerable attention in numerous fields. However, existing methods commonly employ message-passing neural networks (MPNNs) to learn domain-invariant representations by aligning the entire domain distribution, inadvertently neglecting category-level distribution alignment and potentially causing category confusion. To address the problem, we propose an effective framework named Coupling Category Alignment (CoCA) for GDA, which effectively addresses the category alignment issue with theoretical guarantees. CoCA incorporates a graph convolutional network branch and a graph kernel network branch, which explore graph topology in implicit and explicit …

Detecting Defi Fraud With A Graph-Transformer Language Model, Wei Ma, Junjie Shi, Jiaxi Qiu, Cong Wu, Jing Chen, Lingxiao Jiang, Shangqing Liu, Yang Liu, Yang Xiang

Research Collection School Of Computing and Information Systems

With the rapid development of blockchain technology, the widespread adoption of smart contracts—particularly in decentralized finance (DeFi) applications—has introduced significant security challenges, such as reentrancy attacks, phishing, and Sybil attacks. To address these issues, we propose a novel model called TrxGNNBERT, which combines Graph Neural Network (GNN) and the Transformer architecture to effectively handle both graph-structured and textual data. This combination enhances the detection of suspicious transactions and accounts on blockchain platforms like Ethereum. TrxGNNBERT was pre-trained using a masked language model (MLM) on a dataset of 60,000 Ethereum transactions by randomly masking the attributes of nodes and edges, thereby …

Towards Multimodal Emotional Support Conversation Systems, Yuqi Chu, Lizi Liao, Zhiyuan Zhou, Chong-Wah Ngo, Richang Hong

Research Collection School Of Computing and Information Systems

The integration of conversational artificial intelligence (AI) into mental health care promises a new horizon for therapist-client interactions, aiming to closely emulate the depth and nuance of human conversations. Despite the potential, the current landscape of conversational AI is markedly limited by its reliance on single-modal data, constraining the systems’ ability to empathize and provide effective emotional support. This limitation stems from a paucity of resources that encapsulate the multimodal nature of human communication essential for therapeutic counseling. To address this gap, we introduce the Multimodal Emotional Support Conversation (MESC) dataset, a first-of-its-kind resource enriched with comprehensive annotations across text, …

Guiding Multiple Remote Users In Physical Tasks With Language-Driven Robotic Telepresence, Ruyi Li, Jingfei Guo, Xinyi Zhang, Xuji Zhang, Zeqing Li, Jiannan Li, Jiangtao Gong

Research Collection School Of Computing and Information Systems

Remote assistance through robotic telepresence could involve both control and memory challenges, particularly in one expert to multiple workers situation. In this work, we proposed a novelty language-driven interface to facilitate remote collaboration through telepresence robots. Through operations and maintenance expert interviews and a scenario simulation study, we identified key pain points in executing one-expert-multiple-workers remote guidance using the telepresence robot and proposed two design goals, which together consist of five sub-design goals with corresponding features. These features were integrated into a standard telepresence robot, resulting in the development of a Collaborative LLM-based Embodied Assistant Robot, named CLEAR Robot. A …

Implementing Slack-Free Custom Penalty Function For Qubo On Gate-Based Quantum Computers, Xin Wei Lee, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Solving NP-hard constrained combinatorial optimization problems using quantum algorithms remains a challenging yet promising avenue toward quantum advantage. Variational Quantum Algorithms (VQAs), such as the Variational Quantum Eigensolver (VQE), typically require constrained problems to be reformulated as unconstrained ones using penalty methods. A common approach introduces slack variables and quadratic penalties in the QUBO formulation to handle inequality constraints. However, this leads to increased qubit requirements and often distorts the optimization landscape, making it harder to find high-quality feasible solutions. To address these issues, we explore a slack-free formulation that directly encodes inequality constraints using custom penalty functions, specifically the …

Rethinking Cognitive Complexity For Unit Tests: Toward A Readability-Aware Metric Grounded In Developer Perception, Wendkûuni C. Ouédraogo, Yinghua Li, Xueqi Dang, Xin Zhou, Anil Koyuncu, Jacques Klein, David Lo, Tegawendé F. Bissyandé

Research Collection School Of Computing and Information Systems

Automatically generated unit tests-from searchbased tools like EvoSuite or LLMs-vary significantly in structure and readability. Yet most evaluations rely on metrics like Cyclomatic Complexity and Cognitive Complexity, designed for functional code rather than test code. Recent studies have shown that SonarSource's Cognitive Complexity metric assigns nearzero scores to LLM-generated tests, yet its behavior on EvoSuitegenerated tests and its applicability to test-specific code structures remain unexplored. We introduce CCTR, a Test-Aware Cognitive Complexity metric tailored for unit tests. CCTR integrates structural and semantic features like assertion density, annotation roles, and test composition patterns-dimensions ignored by traditional complexity models but critical for …

Flow, Immersion, And Presence: Creating Virtual Reality And Engagement In The Era Of Ubiquitous And Intelligent Technologies, Yi Maggie Guo, Fiona Fui-Hoon Nah, Nannan Xi, Marshall Scott Poole

Research Collection School Of Computing and Information Systems

Researchers use the concepts of flow, immersion, and presence to explain the usage of and engagement (e.g., cognitive absorption) with information technology. In this special issue, we showcase four papers on empirical investigations of flow and immersion, their antecedents, and their outcomes. These papers address research questions that range from investigating the antecedents and consequences of immersion in head-mounted displays of virtual reality, designing for the flow experience in extended reality, studying factors influencing user engagement in the metaverse, and identifying adverse effects of work-related flow. We also provide directions and suggestions for future research.

Vibemus: Proactive Agentic System For Music Personalization, Zhiliang Guo, Teng Tu, Yunshan Ma, Xun Yang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) enable diverse forms of AI-assisted creation, yet they often struggle to bridge the preference-articulation gap: users may provide incomplete or vague intentions or lack the vocabulary to specify what they want, yielding outputs misaligned with true preferences. To address this gap and facilitate music creation in a vibe-centric environment, we introduce VibeMus, a proactive agentic system built on open-source components. The system engages in multi-turn dialogue to progressively determine the music’s emotion, genre, lyrics, and other aspects before generation. Simulated evaluations show that proactive clarification improves alignment with users’ intended nuances. Our approach is training-free, leveraging …

Improving Co-Decoding Based Security Hardening Of Code Llms Leveraging Knowledge Distillation, Dong Li, Shanfu Shu, Meng Yan, Zhongxin Liu, Chao Liu, Xiaohong Zhang, David Lo

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have been widely adopted by developers in software development. However, the massive pretraining code data is not rigorously filtered, allowing LLMs to learn unsafe coding patterns. Several prior studies have demonstrated that code LLMs tend to generate code with potential vulnerabilities. The widespread adoption of intelligent programming assistants poses a significant threat to the software development process. Existing approaches to mitigating this risk primarily involve constructing secure data that are free of vulnerabilities and then retraining or fine-tuning the models. However, such an effort is resource intensive and requires significant manual supervision. When the model parameters …

Building A Novel Question-Answering System Using Retrieval-Augmented Generation For The California Fair Political Practices Commission, Saanvi Dua

Master's Theses

The California Fair Political Practices Commission (FPPC) receives a high volume of inquiries via email from public officials, the general public, and other agencies, which currently requires staff to manually search through informational documents and manuals to provide timely responses. This process is both labor- and time-intensive.

To address this challenge, we design a question-answering (QA) system that drafts responses to emailed questions by retrieving relevant information from the FPPC’s manuals using a retrieval-augmented generation (RAG) framework. Although the current implementation focuses on a single manual, the system is designed to be adaptable to the broader set of FPPC documents. …

Exact And Approximate Conformal Inference For Multi-Output Regression, Chancellor Johnstone, Eugene Ndiaye

Faculty Publications

It is common in machine learning to estimate a response y given covariate information x . However, these predictions alone do not quantify any uncertainty associated with said predictions. One way to overcome this deficiency is with conformal inference methods, which construct a set containing the unobserved response with a prescribed probability. Unfortunately, even with a one-dimensional response, conformal inference is computationally expensive despite recent encouraging advances. In this paper, we explore multi-output regression, delivering exact derivations of conformal inference p-values when the predictive model can be described as a linear function of y . Additionally, we introduce a multivariate …