Digital Commons Network^™

Grounding Is All You Need? Dual Temporal Grounding For Video Dialog, You Qin, Wei Ji, Xinze Lan, Hao Fei, Xun Yang, Dan Guo, Roger Zimmermann, Lizi Liao

Research Collection School Of Computing and Information Systems

In the realm of video dialog response generation, capturing both the essence of video content and the temporal nuances of conversation history is crucial. While some approaches rely on large-scale pretrained visual-language models, often neglecting temporal dynamics, others emphasize spatial-temporal relationships within videos but demand intricate object trajectory pre-extractions and overlook dialog temporal dynamics. This paper introduces the Dual Temporal Grounding-enhanced Video Dialog model (DTGVD), designed to bridge the gap between these two approaches. DTGVD uniquely integrates the strengths of both by emphasizing dual temporal relationships. It achieves this by predicting dialog turn-specific temporal regions, selectively filtering video content, and …

Mm-Attackg: A Multimodal Approach To Attack Graph Construction With Large Language Models, Yongheng Zhang, Xinyun Zhao, Yunshan Ma, Haokai Ma, Yingxiao Guan, Guozheng Yang, Yuliang Lu, Xiang Wang

Research Collection School Of Computing and Information Systems

Cyber Threat Intelligence (CTI) parsing aims to extract key threat information from massive data, transform it into actionable intelligence, enhance threat detection and defense efficiency, including attack graph construction, intelligence fusion, and indicator extraction. Among these research topics, Attack Graph Construction (AGC) is essential for visualizing and understanding the potential attack paths of threat events from CTI reports. Existing approaches primarily construct the attack graphs purely from the textual data to reveal the logical threat relationships between entities within the attack behavioral sequence. However, they typically overlook the specific threat information inherent in visual modalities, which preserves key threat details …

Obstructive Sleep Apnea Prediction: A Comprehensive Review And Comparative Study, Thi Khanh Chi Huynh, Amonae Dabbs-Brown, Anna Jurek-Loughrey, James Mulhall, Tuan Dung Pham, Ngoc Phu Doan, Viet Hung Tran, Zichi Zhang, Xuan Hoang Nguyen, Yimeng An, Peixin Li, Phi Hung Nguyen, Thi Linh Hoang, Xinming Shi, Hans Vandierendonck, Sebastien Bailly, Jean-Louis Pépin, Thai Son Mai

Research Collection School Of Computing and Information Systems

Obstructive Sleep Apnea (OSA) is a highly prevalent sleep disorder linked to considerable public health burdens and comorbidities. However, its heterogeneous presentation and the limited accessibility of traditional diagnostic tools such as polysomnography (PSG) lead to widespread underdiagnosis. As a result, artificial intelligence (AI) approaches, including machine learning (ML) and deep learning (DL) models, have attracted attention as an alternative pathway to detection. This paper first provides a comprehensive review of AI-driven OSA diagnosis, covering different diagnosis problems, input-data types, data biases, pre-processing techniques, and model performance. We then leverage the largest clinical dataset used in OSA prediction to date, …

A Runtime-Adaptive Transformer Neural Network Accelerator On Fpgas, Ehsan Kabir, Jason D. Bakos, David Andrews, Miaoqing Huang

Electrical Engineering and Computer Science Faculty Publications and Presentations

Transformer neural networks (TNN) excel in natural language processing (NLP), machine translation, and computer vision (CV) without relying on recurrent or convolutional layers. However, they have high computational and memory demands, particularly on resource constrained devices like FPGAs. Moreover, transformer models vary in processing time across applications, requiring custom models with specific parameters. Designing custom accelerators for each model is complex and time-intensive. Some custom accelerators exist with no runtime adaptability, and they often rely on sparse matrices to reduce latency. However, hardware designs become more challenging due to the need for application-specific sparsity patterns. This paper introduces ADAPTOR, a …

Hybrid Server–Ai Architecture For Persistent Generative Game Worlds: Achieving Scalable, Consistent, And Low-Latency Interactive Environments, Jay Ratican, James Hutson

Faculty Scholarship

Generative artificial intelligence has demonstrated remarkable capabilities in real-time content creation for interactive entertainment, yet current implementations struggle with the persistence, consistency, and scalability demanded by modern multiplayer and long-form gaming environments. This paper presents a hybrid server–AI architecture that fuses the deterministic reliability of authoritative multiplayer server frameworks with the creative flexibility of state-aware generative systems. The proposed three-tier design consists of (1) a deterministic server backend leveraging technologies such as Unity Netcode for GameObjects, Unreal Engine 5’s dedicated servers, and Amazon GameLift to maintain authoritative and persistent world state; (2) a state-aware generative layer responsible for producing real-time …

The Case For Ai Authorship In Copyright Law, Cheng Lim Saw, Duncan Lim

Research Collection Yong Pung How School Of Law

Today, with generative AI, literary and artistic works can be created almost effortlessly. There is at present intense debate as to whether works generated by AI – broadly categorised as “AI-assisted” and “AI-generated” works – ought to attract copyright protection. AI-assisted works are those that involve some degree of human intervention. Where AI-generated works are concerned, however, such works are created autonomously by the AI itself with minimal (de minimis) input from an identifiable human being. Presently, it is generally accepted that AI-generated works do not attract copyright protection for want of a human author. This article examines whether it …

Learnable Game-Theoretic Policy Optimization For Data-Centric Self-Explanation Rationalization, Yunxiao Zhao, Zhiqiang Wang, Xingtong Yu, Xiaoli Li, Jiye Liang, Ru Li

Research Collection School Of Computing and Information Systems

Rationalization, a data-centric framework, aims to build self-explanatory models to explain the prediction outcome by generating a subset of human-intelligible pieces of the input data. It involves a cooperative game model where a generator generates the most human-intelligible parts of the input (i.e., rationales), followed by a predictor that makes predictions based on these generated rationales. Conventional rationalization methods typically impose constraints via regularization terms to calibrate or penalize undesired generation. However, these methods are suffering from a problem called mode collapse, in which the predictor produces correct predictions yet the generator consistently outputs rationales with collapsed patterns. Moreover, existing …

The Gains Do Not Make Up For The Losses: A Comprehensive Evaluation For Safety Alignment Of Large Language Models Via Machine Unlearning, Weixiang Zhao, Yulin Hu, Xingyu Sui, Zhuojun Li, Yang Deng, Yanyan Zhao, Bing Qin, Wanxiang Che

Research Collection School Of Computing and Information Systems

Machine Unlearning (MU) has emerged as a promising technique for aligning large language models (LLMs) with safety requirements to steer them forgetting specific harmful contents. Despite the significant progress in previous studies, we argue that the current evaluation criteria, which solely focus on safety evaluation, are actually impractical and biased, leading to concerns about the true effectiveness of MU techniques. To address this, we propose to comprehensively evaluate LLMs after MU from three aspects: safety, over-safety, and general utility. Specifically, a novel benchmark MuBench with 18 related datasets is first constructed, where the safety is measured with both vanilla harmful …

Efficient Function Orchestration For Large Language Models, Xiaoxia Liu, Peng Di, Cong Li, Jun Sun, Jingyi Wang

Research Collection School Of Computing and Information Systems

Function calling is a fundamental capability of today's large language models, but sequential function calling posed efficiency problems. Recent studies have proposed to request function calls with parallelism support in order to alleviate this issue. However, they either delegate the concurrent function calls to users for execution which are conversely executed sequentially, or overlook the relations among various function calls, rending limited efficiency. This paper introduces LLMOrch, an advanced framework for automated, parallel function calling in large language models. The key principle behind LLMOrch is to identify an available processor to execute a function call while preventing any single processor …

Exploring Jvm Garbage Collector Testing With Event-Coverage, Kai Zheng, Yingquan Zhao, Junjie Chen, Hanmo You, Haoyu Wang, Haoyu Wang, Tianchang Gao

Research Collection School Of Computing and Information Systems

Garbage Collection (GC) in the Java Virtual Machine (JVM) serves as an automatic memory management mechanism, efficiently reclaiming unused memory space in different production scenarios. To optimize JVM performance, developers typically fine-tune the garbage collector by identifying an optimal set of GC configurations for specific scenarios. Despite the sophisticated design of garbage collectors, they still have the potential for bugs in different settings, and these bugs can result in more severe consequences. Hence, comprehensive testing of these garbage collectors is imperative before their release. Code coverage criteria are typically employed to assess the comprehensiveness of a test suite. However, traditional …

Less Is More: Docstring Compression In Code Generation, Guang Yang, Yu Zhou, Wei Cheng, Xiangyu Zhang, Xiang Chen, Terry Yue Zhuo, Xin Zhou, Ke Liu, David Lo, Taolue Chen

Research Collection School Of Computing and Information Systems

The widespread use of Large Language Models (LLMs) in software engineering has intensified the need for improved model and resource efficiency. In particular, for neural code generation, LLMs are used to translate function/method signature and DocString to executable code. DocStrings, which capture user requirements for the code and are typically used as the prompt for LLMs, often contain redundant information. Recent advancements in prompt compression have shown promising results in Natural Language Processing (NLP), but their applicability to code generation remains uncertain. Our empirical study shows that the state-ofthe-art prompt compression methods achieve only about 10% reduction, as further reductions …

Defending Code Language Models Against Backdoor Attacks With Deceptive Cross-Entropy Loss, Guang Yang, Yu Zhou, Xiangyu Zhang, Xiang Chen, Terry Yue Zhuo, David Lo, Taolue Chen

Research Collection School Of Computing and Information Systems

Code Language Models (CLMs), particularly those leveraging deep learning, have achieved significant success in code intelligence domain. However, the issue of security, particularly backdoor attacks, is often overlooked in this process. The previous research has focused on designing backdoor attacks for CLMs, but effective defenses have not been adequately addressed. In particular, existing defense methods from natural language processing, when directly applied to CLMs, are not effective enough and lack generality, working well in some models and scenarios but failing in others, thus fall short in consistently mitigating backdoor attacks. To bridge this gap, we first confirm the phenomenon of …

Fcghunter: Towards Evaluating Robustness Of Graph-Based Android Malware Detection, Shiwen Song, Xiaofei Xie, Ruitao Feng, Qi Guo, Sen Chen

Research Collection School Of Computing and Information Systems

Graph-based detection methods leveraging Function Call Graph (FCG) have shown promise for Android malware detection (AMD) due to their semantic insights. However, the deployment of malware detectors in dynamic and hostile environments raises significant concerns about their robustness. While recent approaches evaluate the robustness of FCG-based detectors using adversarial attacks, their effectiveness is constrained by the vast perturbation space, particularly across diverse models and features. To address these challenges, we introduce FCGHunter, a novel robustness testing framework for FCG-based AMD systems. Specifically, FCGHunter employs innovative techniques to enhance exploration and exploitation within this huge search space. Initially, it identifies critical …

Quantum Chebyshev Transform-Based Graph Neural Networks For Financial Fraud Detection, Minrui Xu, Bingyan Guan, Bethel Hui Ting Loke, Paul R. Griffin

Research Collection School Of Computing and Information Systems

Financial fraud detection is a critical challenge requiring accurate identification of anomalous patterns in complex transaction networks. Graph Neural Networks (GNNs) have emerged as powerful tools for fraud detection by capturing relational structures among entities. Meanwhile, quantum computing offers new possibilities to enhance machine learning through high-dimensional Hilbert spaces and parallelism. In this paper, we propose a hybrid classical-quantum model called QCTGNN (Quantum Chebyshev Transform-based Graph Neural Network) for financial fraud detection. The QCTGNN integrates a classical graph neural network component based on Simplified Graph Convolutions (SGConv) with a quantum component that performs a Chebyshev polynomial-based transform via variational quantum …

Lagrangian Motion Fields For Long-Term Motion Generation, Yifei Yang, Zikai Huang, Chenshu Xu, Shengfeng He

Research Collection School Of Computing and Information Systems

Long-term motion generation is a challenging task that requires producing coherent and realistic sequences over extended durations. Current methods primarily rely on framewise motion representations, which capture only static spatial details and overlook temporal dynamics. This approach leads to significant redundancy across the temporal dimension, complicating the generation of effective long-term motion. To overcome these limitations, we introduce the novel concept of Lagrangian Motion Fields, specifically designed for long-term motion generation. By treating each joint as a Lagrangian particle with uniform velocity over short intervals, our approach condenses motion representations into a series of "supermotions" (analogous to superpixels). This method …

Zero-Shot Video Translation Via Token Warping, Haiming Zhu, Yangyang Xu, Jun Yu, Shengfeng He

Research Collection School Of Computing and Information Systems

With the revolution of generative AI, video-related tasks have been widely studied. However, current state-of-the-art video models still lag behind image models in visual quality and user control over generated content. In this paper, we introduce TokenWarping, a novel framework for temporally coherent video translation. Existing diffusion-based video editing approaches rely solely on key and value patches in self-attention to ensure temporal consistency, often sacrificing the preservation of local and structural regions. Critically, these methods overlook the significance of the query patches in achieving accurate feature aggregation and temporal coherence. In contrast, TokenWarping leverages complementary token priors by constructing temporal …

Light Cone Cancellation For Variational Quantum Eigensolver In Solving Noisy Max-Cut, Xinwei Lee, Xinjian Yan, Ningyi Xie, Yoshiyuki Saito, Leo Kurosawa, Nobuyoshi Asai, Dongsheng Cai, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

Variational Quantum Eigensolver (VQE) is a quantum-classical hybrid algorithm used to estimate the ground energy of a given Hamiltonian. It consists of a parameterized quantum circuit, which the parameters are optimized using a classical optimizer. With the increasing need in solving large-scale problems in real-world applications, solving those large problems with fewer qubits and fewer gates becomes essential, so that we reduce the simulation difficulty and mitigate the effect of noise in real quantum hardware. In this study, we applied the Light Cone Cancellation (LCC) method to reduce the number of qubits and gates required in a two-local ansatz. LCC …

Cyber Science Education Meets Healthcare Technology, Angela Spencer

Journal of Cybersecurity Education, Research and Practice

The research investigates how cyber science education combines with healthcare technology during the digital age to resolve a fundamental research gap in these two advancing areas. A combined approach utilizing extensive surveys and detailed interviews evaluates the functionality of learning platforms as well as cybersecurity measures and potential uses of emerging virtual reality (VR) and augmented reality (AR) tools to improve both educational and clinical environments. The research document describes its methodologies thoroughly while. The research documents multiple quantitative and qualitative results before performing its analysis, which leads to strategy development for digit. The researchers worked to find ways that …

Deep Learning Style Transfer For Enhanced Smoke Plume Visibility: A Standardized False Color Composite (Sfcc) In Gems Satellite Imagery, Yemin Jeong, Seung Hee Kim, Menas Kafatos, Jeong-Ah Yu, Kyoung-Hee Sung, Seung-Yeon Kim, Goo Kim, Jae-Jin Kim, Yangwon Lee

Institute for ECHO Articles and Research

Wildfire smoke visualization using geostationary satellite imagery is essential for real-time monitoring and atmospheric analysis; however, inconsistencies in color tone across Geostationary Environment Monitoring Spectrometer (GEMS) images hinder reliable interpretation and model training. This study proposes a Standardized False Color Composite (SFCC) framework based on deep learning style transfer to enhance the visual consistency and interpretability of wildfire smoke scenes. Four tone-standardization methods were compared: the statistical Empirical Cumulative Distribution Function (ECDF) correction and three neural approaches—ReHistoGAN, StyTr2, and Style Injection Diffusion Model (SI-DM). Each model was evaluated visually and quantitatively using six metrics (SSIM, LPIPS, FID, histogram similarity, ArtFID, …

How Consistent Friendlike Conversation With Ai Companions Influences Our Attitudes And Perceptions Toward Ai: An Exploratory Experiment, Qi Hui Jerlyn Ho, Meilan Hu, Adalia Yin Hui Goh, Emma Jane Pragasam, Andree Hartanto

Research Collection School of Social Sciences

Despite skepticism and distrust in artificial intelligence (AI), it is increasingly integrated into daily life, with its potential benefits drawing interest. Yet little is known about the attitudinal and psychological effects of human–AI interactions, and whether consistent interactions with AI chatbots can change users’ attitudes and perceptions. Our within-subjects experiment (N = 52) investigated how five days of socially oriented, friendlike interactions with an AI chatbot, versus a journaling control, influenced changes in attitudes and perceptions of AI. Participants’ attitudes towards AI, trust, perceived empathy, anthropomorphism, animacy, likeability, perceived intelligence and safety, dependency, and exploratory well-being indicators were recorded. Results …

6d Rigid Object Pose Estimation Using Deep Learning, Zhujun Li

Dissertations, Theses, and Capstone Projects

6D object pose estimation is the task of determining an object’s 3D rotation and translation with respect to a camera, and plays a critical role in applications such as robotic manipulation, autonomous navigation, and augmented reality. While recent advances in deep learning have substantially improved performance, many existing methods still face limitations in learning robust and generalizable representations. Factors such as variations in object appearance, occlusion, sensor noise, and domain shifts can degrade model accuracy, highlighting the need for more effective representation learning strategies that capture rich geometric and semantic cues for reliable pose estimation across diverse conditions.

This dissertation …

New Fast Polynomial Root-Finders, Soo Go

Dissertations, Theses, and Capstone Projects

Univariate polynomial root-finding has been studied for four millennia and very intensively in the last decades. Our {\em black box root-finder} involves no coefficients and works for a black box polynomial, defined by an oracle (that is, black box subroutine) for its evaluation. Such root-finders have various benefits, e.g., are particularly efficient where a polynomial can be evaluated fast, say, is a sum of a small number of shifted monomials (x-c)^a.

Our root-finder approximates all d complex zeros of a dth degree polynomial p(x) (aka roots of equation p(x)=0) by using Las Vegas expected number of bit-operations within a factor …

Typeface: Machine-Viewing Gentrification On Storefront Imagery In Bedford-Stuyvesant, Brooklyn, Alexander Mcquilkin

Dissertations, Theses, and Capstone Projects

Gentrification—broadly, the replacement of a less powerful group by a more powerful one in an urban context—is oft-discussed in the popular press, but its definition is much-debated in the urban planning literature. Furthermore, academic treatments of displacement understandably focus on measurable yet fairly abstract indicators like changes in rent or income, whereas neighborhood change is often registered by residents on the ground using visual, but difficult-to-quantify markers like retail turnover. This project uses image recognition technology on a set of storefront photos to index the visual streetscape of a neighborhood, as well as to track changes to that portrait over …

Mitigating Malware Prevalence In Networks With Arbitrary Topologies: A Flip-It Cyber Game Approach Integrated With Epidemic Modeling, Mousa Tayseer Jafar, Lu Xing Yang, Gang Li, Robin Doss, Kon Mouzakis, Rajesh Vasa, Helge Janicke, Ahmed Ibrahim, Ahmed Mohsin, Iqbal H. Sarker, Kristen Moore, Seyit Camtepe, Diksha Goel

Research outputs 2022 to 2026

Cyber threats have evolved in complexity, aiming at a wide range of sectors using advanced methods and tools. This evolving threat landscape challenges existing cybersecurity frameworks, many of which lack the adaptability to counteract the complex tactics of sophisticated adversaries. Developing robust cyber defense strategies requires simulating dynamic interactions between attackers and defenders across high, moderate, and low-impact scenarios. The Flip-It cyber game serves as an intelligent framework for simulating these interactions, enabling the analysis of adaptive strategies in cybersecurity. This paper aims to address the problem of mitigating malware prevalence with full consideration of attack/defense capabilities in arbitrary network …

Efficient Privacy-Preserving Conjunctive Searchable Encryption For Cloud-Iot Healthcare Systems, Jiadi Ma, Tianqi Peng, Gong Bei, Muhammad Waqas, Hisham Alasmary, Sheng Chen

Research outputs 2022 to 2026

In cloud-Internet of Things (IoT) healthcare systems, private medical data leakage is a serious concern as the cloud server is not fully trusted. Dynamic searchable symmetric encryption (DSSE), with necessary forward and backward privacy security properties, enables doctors to retrieve ciphertexts while guaranteeing data privacy. However, existing forward and backward private DSSE schemes are not well-suited for cloud-IoT healthcare systems with attribute-value type databases. To this end, we propose an efficient privacy-preserving conjunctive searchable encryption scheme for cloud-IoT healthcare systems, called PC-SE. It is the first conjunctive DSSE scheme designed for attribute-value type databases. Specifically, we design flexible search capabilities …

Removed: When Taxi Drivers Meet Dynamic Pricing: A Lesson From Singapore's Justgrab Program, Shih-Fen Cheng, Wen-Tai Hsu, Jing Li

Research Collection School Of Economics

This paper studies how dynamic pricing influences taxi drivers’ behaviors using a unique event, the inception of the JustGrab program in Singapore in 2017, which introduces dynamic pricing to some, but not all, taxi drivers. This is the first time in history that traditional taxi drivers have access to dynamic pricing. Using data covering the universe of taxi trips before and after the inception of JustGrab, we find that there is spatial reallocation that directs more taxi drivers to the previously less-served areas, that there is also a temporal reallocation that directs more taxi drivers to rush hours, as well …

G-Trac: Graph-Textual Representations Alignment For Cold-Start Recommendations, Li Yang Chang, Yuan Fang, Ming Feng Tsai, Chuan Ju Wang

Research Collection School Of Computing and Information Systems

The cold-start problem remains a significant challenge in recommendation systems, particularly for new users or unseen items with little to no historical data. Existing methods, including graph neural networks, often struggle in such scenarios. Inspired by the success of transformer models in natural language processing, we propose G-TRAC (Graph-Textual Representations Alignment for Cold-start Recommendations), a novel approach that integrates transformer-based textual modeling with graph neural networks. By effectively leveraging both textual and structural information, G-TRAC addresses cold-start challenges more effectively. Extensive experiments demonstrate its ability to enhance recommendation quality and generalize well across diverse scenarios.

Vercation: Precise Vulnerable Open-Source Software Version Identification Based On Static Analysis And Llm, Yiran Cheng, Ting Zhang, Lwin Khin Shar, Shouguo Yang, Chaopeng Dong, David Lo, Shichao Lv, Zhiqiang Shi, Limin Sun

Research Collection School Of Computing and Information Systems

Open-source software (OSS) has experienced a surge in popularity, attributed to its collaborative development model and cost-effective nature. However, the adoption of specific software versions in development projects may introduce security risks when these versions bring along vulnerabilities. Current methods of identifying vulnerable versions typically analyze and extract the code features involved in vulnerability patches using static analysis with pre-defined rules. They then use code clone detection to identify the vulnerable versions. These methods are hindered by imprecision due to (1) the exclusion of vulnerability- irrelevant code in the analysis and (2) the inadequacy of code clone detection. This paper …

Machine Learning In Peak Demand Forecasting: Foundations, Trends, And Insights, Shuang Dai, Fanlin Meng, Hongsheng Dai, Qian Wang, Xizhong Chen, Wenlei Bai, Peizhi Shi, Richard Allmendinger, Yuchen Zhang, Jian Liu

Electrical and Computer Engineering Faculty Research & Creative Works

Peak demand forecasting involves predicting the maximum electricity demand within a specific period, which plays a key role in maintaining the efficiency and stability of power systems. The rapid evolution of power systems, driven by advanced metering infrastructure, local energy applications such as electric vehicles, and the increasing adoption of intermittent renewable energy, has introduced greater randomness and reduced predictability in peak demand. Given the pressing need to address more diverse implementation requirements across different contexts, accurate and reliable peak demand forecasting has become increasingly important. To the best of our knowledge, this study is the first to provide a …

Predicting Water Quality Using Quantum Machine Learning: The Case Of The Umgeni Catchment (U20a) Study Region, Jamal Al-Karaki, Muhammad Al Zafar Khan, Amjad Gawanmeh, Marwan Omar

All Works

The assessment of water quality has become increasingly vital for maintaining the ecological balance and ensuring public safety across global water systems. This study examines the application of Quantum Machine Learning (QML) techniques in a real-world setting to predict water quality in the U20A region of the Umgeni Catchment, Durban, South Africa. We implemented the Quantum Support Vector Classifier (QSVC) and Quantum Neural Network (QNN) on a field-collected dataset. Our results demonstrate that the QSVC is more practical to implement and yields superior performance, achieving 75 % accuracy with polynomial and radial basis function kernels. In contrast, the QNN encountered …