Digital Commons Network^™

Interpretable Multimodal Zero Shot Ecg Diagnosis Via Structured Clinical Knowledge Alignment, Jialu Tang, Hung Manh Pham, Ignace De Lathauwer, Henk S. Schipper, Yuan Lu, Dong Ma, Aaqib Saeed

Research Collection School Of Computing and Information Systems

Electrocardiogram (ECG) interpretation is essential for cardiovascular disease diagnosis, but current automated systems often struggle with transparency and generalization to unseen conditions. To address this, we introduce ZETA, a zero-shot multimodal framework designed for interpretable ECG diagnosis aligned with clinical workflows. ZETA uniquely compares ECG signals against structured positive and negative clinical observations, which are curated through an LLM-assisted, expertvalidated process, thereby mimicking differential diagnosis. Our approach leverages a pre-trained multimodal model to align ECG and text embeddings without disease-specific fine-tuning. Empirical evaluations demonstrate ZETA’s competitive zero-shot classification performance and, importantly, provide qualitative and quantitative evidence of enhanced interpretability, grounding …

Compendia: Automated Visual Storytelling Generation From Online Article Collection, Manusha Imesh Karunathilaka Gamage Vidana, Litian Lei, Yiming Gao, Yong Wang, Jiannan Li

Research Collection School Of Computing and Information Systems

In the digital age, readers value quantitative journalism that is clear, concise, analytical, and humancentred. To understand complex topics, they often piece together scattered facts from multiple articles. Visual storytelling can transform fragmented information into clear, engaging narratives, yet its use with unstructured online articles remains largely unexplored. To fill this gap, we present Compendia, an automated system that analyzes online articles in response to a user’s query and generates a coherent data story tailored to the user’s informational needs. Compendia addresses key challenges of storytelling from unstructured text through two modules covering: Online Article Retrieval, which gathers relevant articles; …

Interpretable Machine Learning For In-Home Mild Cognitive Impairment Detection, Budhitama Subagdja, Shanthoshigaa D, Ah-Hwee Tan, Iris Rawtaer

Research Collection School Of Computing and Information Systems

This paper introduces a novel system for in-home cognitive health assessment using ambient sensors and a machine learning technology that can robustly detect mild cognitive impairment (MCI) despite limited available data. The learned model can explain the aspects of individuals’ daily lives led to the prediction, while reliably predicting MCI, providing more insights to healthcare workers for further clinical interventions. We developed the robust transparent machine learning model, based on fusion adaptive resonance theory (Fusion ART) neural network to learn individuals’ daily patterns of activity from continuous sensor data in terms of a suite of digital biomarkers reflecting four key …

Ai Systems For Physicians: A Review From Socio-Technical And Human-Computer Interaction Perspectives, Wu Jiaqi Young, Fiona Fui-Hoon Nah

Research Collection School Of Computing and Information Systems

The adoption of artificial intelligence (AI) in healthcare is accelerating, yet successful implementations of physician-facing AI systems remain limited and uneven. This paper presents a literature review of 40 peer-reviewed studies published between November 2022 and November 2024, spanning clinical, technical, and human-computer interaction (HCI) domains. Anchored in a socio-technical perspective, the review examines our existing understanding of how technical design, user expertise, and organizational factors shape the effectiveness of AI systems in real-world clinical settings. Our analysis identifies two meta-themes: (1) context as a dynamic, multi-level influence that actively reshapes AI system behavior, and (2) trust as an emergent …

Leveraging Large Language Models For Career Mobility Analysis: A Study Of Gender, Race, And Job Change Using Us Online Resume Profiles, Palakorn Achananuparp, Ye Xu, Yao Lu, Xavier Jayaraj Siddarth Ashok, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

We present a large-scale analysis of career mobility of college-educated U.S. workers using online resume profiles to investigate how gender, race, and job change options are associated with upward mobility. This study addresses key research questions of how the job changes affect their upward career mobility, and how the outcomes of upward career mobility differ by gender and race. We address data challenges – such as missing demographic attributes, missing wage data, and noisy occupation labels – through various data processing and Artificial Intelligence (AI) methods. In particular, we develop a large language models (LLMs) based occupation classification method known …

Dual-Lora And Quality-Enhanced Pseudo Replay For Multimodal Continual Food Learning, Xinlan Wu, Bin Zhu, Feng Han, Pengkun Jiao, Jingling Chen

Research Collection School Of Computing and Information Systems

Food analysis has become increasingly critical for health-related tasks such as personalized nutrition and chronic disease prevention. However, existing large multimodal models (LMMs) in food analysis suffer from catastrophic forgetting when learning new tasks, requiring costly retraining from scratch. To address this, we propose a novel continual learning framework for multimodal food learning, integrating a Dual-LoRA architecture with Quality-Enhanced Pseudo Replay. We introduce two complementary low-rank adapters for each task: a specialized LoRA that learns task-specific knowledge with orthogonal constraints to previous tasks’ subspaces, and a cooperative LoRA that consolidates shared knowledge across tasks via pseudo replay. To improve the …

Benchmarking Gaslighting Negation Attacks Against Reasoning Models, Bin Zhu, Hailong Yin, Jingjing Chen, Yu Gang Jiang

Research Collection School Of Computing and Information Systems

Recent advances in reasoning-centric models promise improved robustness through mechanisms such as chain-of-thought prompting and test-time scaling. However, their ability to withstand gaslighting negation attacks—adversarial prompts that confidently deny correct answers—remains underexplored. In this paper, we conduct a systematic evaluation of three state-of-the-art reasoning models, i.e., OpenAI’s o4-mini, Claude-3.7-Sonnet and Gemini-2.5-Flash, across three multimodal benchmarks: MMMU, MathVista, and CharXiv. Our evaluation reveals significant accuracy drops (25–29% on average) following gaslighting negation attacks, indicating that even top-tier reasoning models struggle to preserve correct answers under manipulative user feedback. Built upon the insights of the evaluation and to further probe this vulnerability, …

Integrating Symbolic And Waveform Music Into Large Language Models, Teng Tu, Xiaohao Liu, Yunshan Ma, Ji Qi, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Music, as a unique and integral element of human life, is characterized by its complex structures, intricate details, and the fusion of multimodal information. Recent study advance music understanding by leveraging knowledge and reasoning capabilities derived from Large Language Models (LLMs). However, they often lack compatibility and fail to fully utilize the complementary strengths of diverse representations (e.g., ABC, MIDI, Waveform). To address these limitations, we propose a unified music-language model framework, named UniMuLM, transitioning from single-representation approaches to the integration of multiple music representations for LLM. Unifying different music representation formats poses challenges such as patch integrity and boundary …

Paid Search Marketing Vs. Search Engine Optimization: Analytical Models Of Search Marketing Based On Search Engine Quality, Kai Li, Chunyang Shen, Mei Lin, Zhangxi Lin

Research Collection School Of Computing and Information Systems

As search engines are leading revenue growth in online marketing, search marketing has become a popular area of academic research. Although search engine advertising has interested researchers for decades and much has been learned, one thing that puzzles scholars is why search engine optimization companies are tolerated rather than excluded from the market, even though they capture a significant share of the advertising market. In this paper, we shed light on this phenomenon and establish an analytical model based on organic search quality. Through analysis of the model, we were able to draw several intriguing conclusions. First, there is no …

Learning-Based Graph Shrinking For Quantum Optimization Of Constrained Combinatorial Problems, Monit Sharma, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Graph shrinking has recently emerged as a powerful preprocessing technique for hybrid classical–quantum optimization, enabling variable and constraint reduction before quantum solving. Conventional approaches rely on Semi-Definite Programming (SDP) relaxations to compute vertex correlations, but these methods suffer from high computational overhead, instance-specific tuning, and limited generalizability. In this work, we replace the handcrafted SDP correlation stage with a reinforcement learning (RL) based correlation estimator, trained to predict merge quality directly from graph structure. We reformulate the graph shrinking process as a Markov Decision Process (MDP), design a Graph Neural Network (GNN) policy to guide vertex merging, and integrate the …

Hybrid Learning And Optimization Methods For Solving Capacitated Vehicle Routing Problem, Monit Sharma, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

We propose a hybrid quantum–classical framework for the Capacitated Vehicle Routing Problem (CVRP) that integrates the Augmented Lagrangian Method (ALM) with deep reinforcement learning (RL). Directly solving CVRP via Variational Quantum Eigensolver (VQE) requires a slack-based QUBO formulation, where converting inequalities to equalities greatly increases the qubit count. To circumvent this, we employ an ALM-based reformulation that enforces constraints through Lagrange terms instead of slack variables, drastically reducing quantum resource demands. An RL agent, trained with Soft Actor–Critic, adaptively tunes the Lagrange penalties to improve convergence and feasibility. Experiments show that RL-Q-ALM outperforms static-penalty and plain VQE baselines in both …

Food Recognition With Visual Language Models: Search Re-Ranking Or Retrieval-Augmented Generation?, Kian Yu Gan, Phuong Anh Nguyen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Despite the rapid advances in Visual Language Models (VLMs), these models struggle to recognize culture-specific food items. While VLMs are effective in recognizing popular cultural dishes, their performance is suboptimal for dishes that are unique but not widely known internationally. Specifically, VLMs often generate either generic labels or hallucinated names for dishes that are localized to a particular culture. As a result, retrieval-augmented generation (RAG), which retrieves relevant recipes as references for VLMs, emerges as a promising approach. Nevertheless, recipe retrieval, which is itself imperfect, could mislead VLMs into generating inaccurate or culturally inappropriate dish names. This paper presents a …

Towards Secure Program Partitioning For Smart Contracts With Llm’S In-Context Learning, Ye Liu, Yuqing Niu, Chengyan Ma, Ruidong Han, Wei Ma, Yi Li, Debin Gao, David Lo

Research Collection School Of Computing and Information Systems

Smart contracts are highly susceptible to manipulation attacks due to the leakage of sensitive information. Addressing manipulation vulnerabilities is particularly challenging because they stem from inherent data confidentiality issues rather than straightforward implementation bugs. To tackle this by preventing sensitive information leakage, we present PARTITIONGPT, the first LLM-driven approach that combines static analysis with the in-context learning capabilities of large language models (LLMs) to partition smart contracts into critical (privileged) and normal codebases, guided by a few annotated sensitive data variables. We evaluated PARTITIONGPT on 18 annotated smart contracts containing 99 sensitive functions. The results demonstrate that PARTITIONGPT successfully generates …

Ai Tips And Traps, Patrick Barry

Books

Based on a series of popular courses and workshops that Professor Patrick Barry has created for students, professionals, and anyone else interested in taking a skills-based approach to artificial intelligence, this book gives you a chance to engage with important AI concepts, experiment with exploratory AI exercises, and then ultimately develop your own customized list of AI traps to try as well as AI traps to avoid.

Multimodal Representation Learning For Face Understanding: From Caption Supervision To Foundation Model Adaptation, Md Mahedi Hasan

Graduate Theses, Dissertations, and Problem Reports (ETD)

The rapid advancement of intelligent surveillance systems and the increasing demand for reliable biometric identification in border security, public safety, and digital forensics require robust face understanding under unconstrained conditions, including low resolution, pose variation, and occlusion. While Vision Transformer (ViT)-based foundation models have greatly improved visual representation learning, their patch-based tokenization and lack of spatial inductive bias limit their ability to capture fine-grained details in low-resolution inputs. This dissertation investigates multimodal representation learning for face understanding through natural language supervision, large-scale face-caption pre-training, and parameter-efficient foundation model adaptation. It hypothesizes that textual supervision provides complementary semantic cues that improve …

Llm-Driven Weekly Newsletter To Assess Open Source Software Project Github Health, Christian Novalski, Christopher Chavez, Ghalian Fayyadh, Kostadin Damevski

UROP Posters

Open Source Software (OSS) projects increasingly depend on a diverse set of contributors, including episodic participants who contribute intermittently. Episodic contributors represent a large portion of OSS communities, yet projects often struggle to retain them, leading to decreased project health and continuity. While dashboards and real-time communication tools support continuously active contributors, they often fail to serve the unique needs of episodic participants, who may struggle to remain informed and re-engage with project activity after periods of absence. In this study, we examine the effect of a weekly, email-based newsletter intervention designed to improve awareness and engagement among episodic OSS …

Artem: Enhancing Large Language Model Agents With Spatial-Temporal Episodic Memory, Cassandra Hui Ming Tan, Budhitama Subagdja, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Current large language models (LLMs) exhibit significant deficiencies in episodic memory tasks including encoding, storing, and retrieving specific information from temporally dependent events over a long period of time. Recent approaches to handle memory tasks in LLMs, such as in-context learning, retrieval-augmented generation (RAG), and fine-tuning, may resolve the long-term retention issues, but are still inadequate to handle tasks requiring chronological awareness of the stored information. We introduce Agentic Retrieval with Temporal-Episodic Memory (ARTEM), a hybrid LLM-based agent architecture integrating LLMs with a self-organizing neural network named Spatial-Temporal Episodic Memory (STEM), designed to handle episodic memory tasks. Our approach employs …

Llamoco: Instruction Tuning Of Large Language Models For Optimization Code Generation, Zeyuan Ma, Yue-Jiao Gong, Hongshu Guo, Jiacheng Chen, Yining Ma, Zhiguang Cao

Research Collection School Of Computing and Information Systems

Recently, combining the strength of large language models (LLMs) and Evolutionary Computation (EC) has shown promising results for addressing optimization problems. It typically involves either iterative next-step solution seeking or directly prompting LLMs to generate critical optimization codes. However, these methods often suffer from low computational efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. LLaMoCo features a comprehensive instruction set that includes code-style problem descriptions as input prompts and robust optimization codes from expert EC optimizers as …

Realign: Text-To-Motion Generation Via Step-Aware Reward-Guided Alignment, Wanjiang Weng, Xiaofeng Tan, Junbo Wang, Guo-Sen Xie, Pan Zhou, Hongsong Wang

Research Collection School Of Computing and Information Systems

Text-to-motion generation, which synthesizes 3D human motions from text inputs, holds immense potential for applications in gaming, film, and robotics. Recently, diffusion-based methods have been shown to generate more diversity and realistic motion. However, there exists a misalignment between text and motion distributions in diffusion models, which leads to semantically inconsistent or low-quality motions. To address this limitation, we propose Reward-guided sampling Alignment (ReAlign), comprising a step-aware reward model to assess alignment quality during the denoising sampling and a reward-guided strategy that directs the diffusion process toward an optimally aligned distribution. This reward model integrates step-aware tokens and combines a …

Cross-Modal Proxy Evolving For Ood Detection With Vision-Language Models, Hao Tang, Yu Liu, Shuanglin Yan, Fei Shen, Shengfeng He, Jing Qin

Research Collection School Of Computing and Information Systems

Reliable zero-shot detection of out-of-distribution (OOD) inputs is critical for deploying vision-language models in open-world settings. However, the lack of labeled negatives in zero-shot OOD detection necessitates proxy signals that remain effective under distribution shift. Existing negative-label methods rely on a fixed set of textual proxies, which (i) sparsely sample the semantic space beyond in-distribution (ID) classes and (ii) remain static while only visual features drift, leading to cross-modal misalignment and unstable predictions. In this paper, we propose CoEvo, a training- and annotation-free test-time framework that performs bidirectional, sample-conditioned adaptation of both textual and visual proxies. Specifically, CoEvo introduces a …

Portfoliopilot: An Agentic Platform For Financial Portfolio Management Algorithm Development And Evaluation, Jared Chan Xu Yang, Haokai Ma, Yunshan Ma

Research Collection School Of Computing and Information Systems

Developing new portfolio-management algorithms typically demands substantial programming effort, limiting rapid experimentation and excluding finance professionals without coding skills. Current robo-advisory tools offer pre-built but rigid strategies, restricting customization and experimentation. We introduce PortfolioPilot, an open-source, agentic platform that enables users to generate bespoke portfolio through natural-language descriptions. Leveraging the Anthropic Claude API, PortfolioPilot dynamically synthesizes executable TypeScript algorithms that run in the frontend with security validation. The system integrates real-time backtesting with historical market data, classical optimization algorithms (Markowitz, LSTM, ARIMA), and interactive performance visualizations.

Look, Compare And Draw: Differential Query Transformer For Automatic Oil Painting, Lingyu Liu, Yaxiong Wang, Li Zhu, Lizi Liao, Zhedong Zheng

Research Collection School Of Computing and Information Systems

This work introduces a new approach to automatic oil painting that emphasizes the creation of dynamic and expressive brushstrokes. A pivotal challenge lies in mitigating the duplicate and common-place strokes, which often lead to less aesthetic outcomes. Inspired by the human painting process, i.e., observing, comparing, and drawing, we incorporate differential image analysis into a neural oil painting model, allowing the model to effectively concentrate on the incremental impact of successive brushstrokes. To operationalize this concept, we propose the Differential Query Transformer (DQ-Transformer), a new architecture that leverages differentially derived image representations enriched with positional encoding to guide the stroke …

Nondeterministic Polynomial-Time Problem Challenge: An Ever-Scaling Reasoning Benchmark For Llms, Chang Yang, Ruiyu Wang, Junzhe Jiang, Qi Jiang, Qinggang Zhang, Yanchen Deng, Shuxin Li, Shuyue Hu, Bo Li, Florian T. Pokorny, Xiao Huang, Xinrun Wang

Research Collection School Of Computing and Information Systems

Reasoning is the fundamental capability of large language models (LLMs). Due to the rapid progress of LLMs, there are two main issues of current benchmarks: i) these benchmarks can be crushed in a short time (less than 1 year), and ii) these benchmarks may be easily hacked. To handle these issues, we propose the ever-scalingness for building the benchmarks which are scaling over complexity against crushing, instance against hacking and exploitation, oversight for easy verification, and coverage for real-world relevance. This paper presents Nondeterministic Polynomial-time Problem Challenge (NPPC), an ever-scaling reasoning benchmark for LLMs. Specifically, the NPPC has three main …

An Investigation Into The Mechanisms, Barriers, Degree And Sphere Of Risk Influence In Corporate Security, Nicola Lockhart

Theses: Doctorates and Masters

This study investigates the sphere of corporate security risk influence within organisations, addressing the conceptual and practical ambiguity surrounding the activity’s capacity to shape organisational decisions, behaviours, and risk priorities. While corporate security’s protective role is widely recognised, its broader organisational risk influence remains under-theorised. The study defines the sphere of risk influence as the range of organisational stakeholders and environments with which the corporate security activity interacts, and within which it may engage, persuade, and mobilise action. This sphere is analytically constituted through the intersection of three dimensions: the mechanisms through which influence is attempted, the barriers that constrain …

Qualitative Study For Llm-Assisted Design Study Process: Strategies, Challenges, And Roles, Shaolun Ruan, Rui Sheng, Xiaolin Wen, Jiachen Wang, Tianyi Zhang, Yong Wang, Tim Dwyer, Jiannan Li

Research Collection School Of Computing and Information Systems

Design studies aim to develop visualization solutions for real-world problems across various application domains. Recently, the emergence of large language models (LLMs) has introduced new opportunities to enhance the design study process, providing capabilities such as creative problem-solving, data handling, and insightful analysis. However, despite their growing popularity, there remains a lack of systematic understanding of how LLMs can effectively assist researchers in visualization-specific design studies. In this paper, we conducted a rnulti-stage qualitative study to fill this gap, which involved 30 design study researchers from diverse backgrounds and expertise levels. Through in-depth interviews and carefully-designed questionnaires, we investigated strategies …

Neuroevolution Of Compact Search Heuristics In Sokoban, Yeonghun Lee

Computer Science Honors Papers

This thesis investigates whether neuroevolution can produce heuristic functions for A* search on Sokoban that are drastically more parameter-efficient than a conventional gradient-trained baseline. Sokoban is a combinatorial planning puzzle that is both NPhard and PSPACE-complete, and solving it at scale requires an accurate cost-to-go heuristic to guide search. The standard approach fixes a neural network architecture and trains it by gradient descent; this thesis investigates whether evolutionary search over network topology can find heuristics that are competitive in accuracy while using orders of magnitude fewer parameters.

Four heuristic approaches are implemented and compared on the medium-difficulty split of the …

Sparse Gradient Training For Recommender Systems, Yunke Qu, Liang Qu, Tong Chen, Xiangyu Zhao, Jianxin Li, Hongzhi Yin

Research outputs 2022 to 2026

Recommender systems are widely applied in numerous online platforms such as shopping and social media platforms. They typically utilize large embedding tables that map users and items to dense vectors of uniform sizes. As the number of users and items continues to grow, this design leads to significant memory consumption and computational inefficiencies. This challenge is particularly pronounced in scenarios such as federated learning, where model parameters are updated locally on edge devices with limited computational resources before being transmitted to a central server for aggregation. Numerous approaches have been proposed to address this issue, among which embedding pruning methods …

A Comprehensive Review Of Cyber Security And Current Practices In Global Mining Critical Infrastructure, Abu Barkat Ullah, Wanli Ma, Mohiuddin Ahmed, Bazlur Rashid, Munir Ahmad Saeed, Omer Arshad, Utkarsh Raghav

Research outputs 2022 to 2026

The purpose of the study is to explore the reasons behind the low uptake of Information Security Management Standards (ISMS), Asset Management, and Business Continuity Plans despite increasing cyber threats to the mining sector. Mining companies need to modernize and automate to keep up with the ‘Fourth Industrial Revolution’, driven by disruptive technology, forcing systems and technologies to become more integrated, increasing cyber attack threats. To address this, we conducted a literature review analyzing the mining industry across various regions. The research is based on a qualitative analysis of diversified literature. The results highlighted factors behind the low uptake of …

A Knowledge-Driven, Ai-Assisted Cyber Defence Framework For Iomt Remote Patient Monitoring, Kulsoom S. Bughio, David M. Cook, Abdul M. Unar

Research outputs 2022 to 2026

The rapid adoption of Internet Medical Things (IoMT) technologies in remote patient monitoring has reshaped healthcare delivery by enabling continuous, real-time clinical observation outside traditional care settings. However, this shift has also expanded the cyber-attack surface across heterogeneous, resource-constrained medical devices, wireless networks, cloud services, and third-party platforms. In cyber warfare, healthcare has become an incorporated target of geopolitics, with hospitals, remote monitoring systems, and emergency health systems being used to broaden the attack surface for adversaries to exploit. Existing security approaches for IoMT environments remain largely manual, fragmented, and reactive, limiting their effectiveness in dynamically assessing vulnerabilities and supporting …

Enhancing Healthcare Security: Manifold-Aware Machine Learning For Robust Adversarial Attack Detection In Iomt Networks, Mohmmad Al-Fawa’Reh, Mohammed Kaosar

Research outputs 2022 to 2026

The widespread adoption of Internet of Medical Things (IoMT) devices and the increasing movement towards telehealth have revolutionized healthcare delivery but also introduced significant security challenges. Tiny Machine Learning (TinyML) models deployed on resource-constrained medical devices are vulnerable to adversarial attacks that can compromise patient data and device functionality, posing risks to patient safety. To address these critical security concerns, this paper proposes MARD (Manifold-Aware Robust Defense), a defense mechanism designed to enhance the robustness of TinyML models. MARD trains a compact student model by transferring knowledge from a teacher model that incorporates Graph-based Manifold Regularization (GMR) and Manifold Mixup …