Digital Commons Network^™

Seeing Is Fixing: Cross-Modal Reasoning With Multimodal Llms For Visual Software Issue Fixing, Kai Huang, Jian Zhang, Xiaofei Xie, Chunyang Chen

Research Collection School Of Computing and Information Systems

Large language model (LLM)-based automated program repair (APR) techniques have shown promising results in resolving real-world github issue tasks. Existing APR systems are primarily evaluated in unimodal settings (e.g., SWE-bench), relying solely on textual issue descriptions and source code. However, these autonomous systems struggle to resolve multimodal problem scenarios (e.g., SWE-bench M) due to limitations in interpreting and leveraging visual information. In multimodal scenarios, LLMs need to rely on visual information in the graphical user interface (GUI) to understand bugs and generate fixes. To bridge this gap, we propose GUIRepair, a cross-modal reasoning approach for resolving multimodal issue scenarios by …

From Personas To Talks: Revisiting The Impact Of Personas On Llm-Synthesized Emotional Support Conversations, Shenghan Wu, Yimo Zhu, Wynne Hsu, Mong-Li Lee, Yang Deng

Research Collection School Of Computing and Information Systems

The rapid advancement of Large Language Models (LLMs) has revolutionized the generation of emotional support conversations (ESC), offering scalable solutions with reduced costs and enhanced data privacy. This paper explores the role of personas in the creation of ESC by LLMs. Our research utilizes established psychological frameworks to measure and infuse persona traits into LLMs, which then generate dialogues in the emotional support scenario. We conduct extensive evaluations to understand the stability of persona traits in dialogues, examining shifts in traits post-generation and their impact on dialogue quality and strategy distribution. Experimental results reveal several notable findings: 1) LLMs can …

Adasteer: Your Aligned Llm Is Inherently An Adaptive Jailbreak Defender, Weixiang Zhao, Jiahe Guo, Yulin Hu, Yang Deng, An Zhang, Xingyu Sui, Xinyang Han, Yanyan Zhao, Bing Qin, Tat-Seng Chua, Ting Liu

Research Collection School Of Computing and Information Systems

Despite extensive efforts in safety alignment, large language models (LLMs) remain vulnerable to jailbreak attacks. Activation steering offers a training-free defense method but relies on fixed steering coefficients, resulting in suboptimal protection and increased false rejections of benign inputs. To address this, we propose AdaSteer, an adaptive activation steering method that dynamically adjusts model behavior based on input characteristics. We identify two key properties: Rejection Law (R-Law), which shows that stronger steering is needed for jailbreak inputs opposing the rejection direction, and Harmfulness Law (H-Law), which differentiates adversarial and benign inputs. AdaSteer steers input representations along both the Rejection Direction …

Chain Of Strategy Optimization Makes Large Language Models Better Emotional Supporter, Weixiang Zhao, Xingyu Sui, Xinyang Han, Yang Deng, Yulin Hu, Jiahe Guo, Libo Qin, Qianyun Du, Shijin Wang, Yanyan Zhao, Bing Qin, Ting Liu

Research Collection School Of Computing and Information Systems

The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to users’ emotional needs. Existing supervised fine-tuning (SFT) struggles to address these issues, as it rigidly trains models on single gold-standard responses without modeling nuanced strategy trade-offs. To overcome these limitations, we propose a novel two-stage framework that optimizes strategy selection preferences at each dialogue turn. We first leverage Monte Carlo Tree Search to construct ESC-Pro, a high-quality …

Exploring Autonomous Agents: A Closer Look At Why They Fail When Completing Tasks, Ruofan Lu, Yichen Li, Yintong Huo

Research Collection School Of Computing and Information Systems

Autonomous agent systems powered by Large Language Models (LLMs) have demonstrated promising capabilities in automating complex tasks. However, current evaluations largely rely on success rates without systematically analyzing the interactions, communication mechanisms, and failure causes within these systems. To bridge this gap, we present a benchmark of 34 representative programmable tasks designed to rigorously assess autonomous agents. Using this benchmark, we evaluate three popular open-source agent frameworks combined with two LLM backbones, observing a task completion rate of approximately 50%. Through in-depth failure analysis, we develop a three-tier taxonomy of failure causes aligned with task phases, highlighting planning errors, task …

Envisioning Future Interactive Web Development: Editing Webpage With Natural Language, Truong Hai Dang, Jingyu Xiao, Yintong Huo

Research Collection School Of Computing and Information Systems

The evolution of web applications relies on iterative code modifications, a process that is traditionally manual and time-consuming. While Large Language Models (LLMs) can generate UI code, their ability to edit existing code from new design requirements (e.g., ”center the logo”) remains a challenge. This is largely due to the absence of large-scale, high-quality tuning data to align model performance with human expectations. In this paper, we introduce a novel, automated data generation pipeline that uses LLMs to synthesize a high-quality fine-tuning dataset for web editing, named Instruct4Edit. Our approach generates diverse instructions, applies the corresponding code modifications, and performs …

Damslnet: Dual-Attention Multi-Scale Lightweight Network For Plant Disease Classification, Linfan Deng, Juan Qin, Kun Li, Jinhua Zhu, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Accurately identifying crop diseases plays a crucial role in advancing intelligent and modern agricultural production. Deep learning techniques have performed robust performance in classifying plant disease images. However, current studies face the challenge that many plant disease datasets are generated in controlled environments, leading to reduced model performance in real-world agricultural settings. This paper aims to provide a lightweight model that can accurately classify plant diseases in natural environments. Specifically, this paper investigates the Dual-Attention Multi-Scale Lightweight Network (DAMSLNet), which combines dual-attention-based multi-scale feature extraction and deep information fusion, to classify plant diseases. At the front end, the model employs …

Context-Aware Hierarchical Taxonomy Generation For Scientific Papers Via Llm-Guided Multi-Aspect Clustering, Kun Zhu, Lizi Liao, Yuxuan Gu, Lei Huang, Xiaocheng Feng, Bing Qin

Research Collection School Of Computing and Information Systems

The rapid growth of scientific literature demands efficient methods to organize and synthesize research findings. Existing taxonomy construction methods, leveraging unsupervised clustering or direct prompting of large language models (LLMs), often lack coherence and granularity. We propose a novel context-aware hierarchical taxonomy generation framework that integrates LLM-guided multi-aspect encoding with dynamic clustering. Our method leverages LLMs to identify key aspects of each paper (e.g., methodology, dataset, evaluation) and generates aspect-specific paper summaries, which are then encoded and clustered along each aspect to form a coherent hierarchy. In addition, we introduce a new evaluation benchmark of 156 expert-crafted taxonomies encompassing 11.6k …

Why Stop At One Error? Benchmarking Llms As Data Science Code Debuggers For Multi-Hop And Multi-Bug Errors, Zhiyu Yang, Shuo Wang, Yukun Yan, Yang Deng

Research Collection School Of Computing and Information Systems

LLMs are transforming software development, yet current code generation and code repair benchmarks mainly assess syntactic and functional correctness in simple, single-error cases. LLMs’ capabilities to autonomously find and fix runtime logical errors in complex data science code remain largely unexplored. To address this gap, we introduce DSDBench: the Data Science Debugging Benchmark, the first benchmark for systematic evaluation of LLMs on multi-hop error tracing and multi-bug detection in data science code debugging. DSDBench adapts datasets from existing data science task benchmarks, such as DABench and MatPlotBench, featuring realistic data science debugging tasks with automatically synthesized multi-hop, multi-bug code snippets. …

Temporal And Heterogeneous Graph Neural Network For Remaining Useful Life Prediction, Zhihao Wen, Yuan Fang, Pengcheng Wei, Fayao Liu, Zhenghua Chen, Min Wu

Research Collection School Of Computing and Information Systems

Predicting remaining useful life (RUL) plays a crucial role in the prognostics and health management of industrial systems that involve a variety of interrelated sensors. Given a constant stream of time-series sensory data from such systems, deep learning (DL) models have risen to prominence at identifying complex, nonlinear temporal dependencies in these data. In addition to the temporal dependencies of individual sensors, spatial dependencies emerge as important correlations among these sensors, which can be naturally modeled by a temporal graph that describes time-varying spatial relationships. However, the majority of existing studies have relied on capturing discrete snapshots of this temporal …

Does Generative Ai Facilitate Investor Trading? Early Evidence From Chatgpt Outages, Qiang Cheng, Pengkai Lin, Yue Zhao

Research Collection School Of Accountancy

In this paper, we use ChatGPT outages to provide early evidence on whether investors rely on generative artificial intelligence (GenAI) to perform professional tasks and the associated impact on stock price informativeness. We document a significant decline in stock trading volume during ChatGPT outages. The effect is stronger for firms with corporate news released immediately before or during the outages and for firms with higher ownership held by transient institutional investors. We then document declines in short-run price impact and return variance during the outage periods, consistent with reduced informed trading. Lastly, we document a positive effect of GenAI-assisted trading …

Branch-And-Cut-And-Price For Agile Earth Observation Satellite Scheduling, Guansheng Peng, Jianjiang Wang, Guopeng Song, Aldy Gunawan, Lining Xing, Pieter Vansteenwegen

Research Collection School Of Computing and Information Systems

The Agile Earth Observation Satellite scheduling selects and sequences satellite observations of possible targets on the Earth’s surface, each with a specific profit and multiple time windows. The objective is to maximize the collected profit of all observations completed under some operational constraints. The problem can be modeled as a variant of the Team Orienteering Problem with Time Windows (TOPTW). The key differences with the regular TOPTW are twofold: first, a time-dependent transition time is required for each pair of consecutive observations to adjust the camera’s look angles. Second, the time windows of each target vary during different observation cycles, …

Image Captioning Through The Lens Of The Gricean Maxims: Generating Meaningful And Relevant Image Descriptions, Annika Lindh

Doctoral

Image captioning models enable us to automatically generate natural language image descriptions for previously unseen images. It combines the two fields of computer vision and natural language generation, allowing models to interpret the con tent of an image and communicate that knowledge through natural language text.

Research into image captioning has the potential benefit of reducing the gap in digital information availability between fully sighted individuals and those who are visually impaired. However, automatically generated captions often fail to provide the required level of detail and specificity to achieve this goal. Furthermore, current standard evaluation methods are insufficient at measuring …

Defects4c: Benchmarking Large Language Model Repair Capability With C/C++ Bugs, Jian Wang, Xiaofei Xie, Qiang Hu, Shangqing Liu, Jiongchi Yu, Jiaolong Kong, Yi Li

Research Collection School Of Computing and Information Systems

Automated Program Repair (APR) plays a critical role in enhancing the quality and reliability of software systems. While substantial progress has been made in Java-based APR, largely facilitated by benchmarks like Defects4J, there remains a significant gap in research on C/C++ program repair, despite the widespread use of C/C++ and the prevalence of associated vulnerabilities. This gap is primarily due to the lack of high-quality, open-source benchmarks tailored for C/C++. To address this issue, we introduce Defects4C, a comprehensive and executable benchmark specifically designed for C/C++ program repair. Our dataset is constructed from real-world C/C++ repositories and includes a large …

Mmlu-Prox: A Multilingual Benchmark For Advanced Large Language Model Evaluation, Weihao Xuan, Et. Al.

Research Collection School Of Computing and Information Systems

Existing large language model (LLM) evaluation benchmarks primarily focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. This dual limitation makes it challenging to assess LLMs’ performance in the multilingual setting comprehensively. To fill this gap, we introduce MMLU-ProX, a comprehensive benchmark covering 29 languages, built on an English benchmark. Each language version consists of 11,829 identical questions, enabling direct cross-lingual comparisons. Additionally, to meet efficient evaluation needs, we provide a lite version containing 658 questions per language. To ensure the high quality of MMLU-ProX, we employ a rigorous development process that involves …

Interaction2code: Benchmarking Mllm-Based Interactive Webpage Code Generation From Interactive Prototyping, Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang, Xinyi Xu, Wenxuan Wang, Zhiyao Xu, Yuhang Wang, Michael R. Lyu

Research Collection School Of Computing and Information Systems

Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance on the design-to-code task, i.e., generating UI code from UI mock-ups. However, existing benchmarks only contain static web pages for evaluation and ignore the dynamic interaction, limiting the practicality, usability and user engagement of the generated webpages. To bridge these gaps, we present the first systematic investigation of MLLMs in generating interactive webpages. Specifically, we formulate the Interaction-to-Code task and establish the Interaction2Code benchmark, encompassing 127 unique webpages and 374 distinct interactions across 15 webpage types and 31 interaction categories. Through comprehensive experiments utilizing state-of-theart (SOTA) MLLMs, evaluated via both automatic …

Distillcaps: Enhancing Audio-Language Alignment In Captioning Via Retrieval-Augmented Knowledge Distillation, Thinh Pham, Nghiem Diep, Lizi Liao, Binh Nguyen

Research Collection School Of Computing and Information Systems

Automated audio captioning (AAC) benefits from incorporatingexternal context to interpret complex sounds, but doing so withretrieval-augmented generation (RAG) at inference is sometimesinfeasible due to data availability or incurs significant latency andcomplexity. We propose DistillCaps, a novel training-time frame-work that leverages RAG to guide knowledge distillation for im-proved audio-language alignment, while lessening the relianceon retrieval during inference. In our framework, a RAG-equippedteacher model retrieves relevant textual information (e.g., simi-lar captions) for each audio clip and uses it for training to gener-ate context-enriched captions. Simultaneously, a student model istrained to imitate this teacher, learning to produce high-qualitycaptions from audio alone. We further …

Designing For Novice Debuggers: A Pilot Study On An Ai-Assisted Debugging Tool, Oka Kurniawan, Erick Chandra, Christopher M. Poskitt, Yannic Noller, Kenny T.W. Choo, Cyrille Jegourel

Research Collection School Of Computing and Information Systems

Debugging is a fundamental skill that novice programmers must develop. Numerous tools have been created to assist novice programmers in this process. Recently, large language models (LLMs) have been integrated with automated program repair techniques to generate fixes for students' buggy code. However, many of these tools foster an over-reliance on AI and do not actively engage students in the debugging process. In this work, we aim to design an intuitive debugging assistant, CodeHinter, that combines traditional debugging tools with LLM-based techniques to help novice debuggers fix semantic errors while promoting active engagement in the debugging process. We present findings …

Instructors’ Strategies In Creating And Implementing Constructivist Llm-Based Learning Activities, Emily Aurelia, Shun Yi Yeo, Michelle Lui, Effie Lai-Chong Law, Anthony Tang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) are increasingly being integrated into educational settings, enabling more adoption of constructivist teaching and learning approaches in classrooms. This paper explores the strategies instructors are currently using to incorporate LLMs into learning activities that align with constructivist principles, which emphasize that learners actively construct their own knowledge. Through interviews with nine instructors who have designed eleven distinct LLM-based activities and using reflexive thematic analysis, this study identifies various types of learning activities with respect to four different aspects of the constructivist learning theory. The strategies employed and challenges faced to foster constructivist student-LLM interaction were also …

Enhancing The Acceptability Of Decision-Making System Using Xai Case: Crime Profiling System, Mohamed Saeed Al Shamsi

Theses

In the current world, we need to place more emphasis on how easily interpretable, accurate, and acceptable data analysis results are, given that essential operations in law enforcement, among other sectors, are backed up by the use of complex computing systems. Crime profiling systems that use crime data for profiling encounter major problems because they depend on algorithm-based methods. These methods can be ambiguous and inaccurate, leading to low public acceptability. The study investigates major problems with Complex Crime profiling systems (CPS) because their unexplained algorithms result in system performance issues and public scepticism. XAI provides a solution to handle …

Advancing Security Safeguards In Large Language Models Through Multi-Agent Systems, Mohammed Rashed Alnuaimi

Theses

This thesis focused on enhancing the safe use of Large Language Model (LLM) through the innovative use of a Multi-Agent System (MAS). As LLMs like ChatGPT became essential to our everyday interactions, the need to maintain the safe use of these systems increased. This research thoroughly assessed the current security measures in place for LLM, pointed out their limitations and developed new and more effective security strategies. The core of the proposed solution was a MAS designed to ensure that all data processed by LLM met guidelines including Privacy, Confidentiality, and Ethical standards before reaching the user. The system involved …

Security Modelling For Cyber-Physical Systems: A Systematic Literature Review, Shao Fei Huang, Christopher M. Poskitt, Lwin Khin Shar

Research Collection School Of Computing and Information Systems

Cyber-physical systems are at the intersection of digital technology and engineering domains, rendering them high-value targets of sophisticated and well-funded cybersecurity threat actors. Prominent cybersecurity attacks on CPS have brought attention to the vulnerability of these systems and the inherent weaknesses of critical infrastructure reliant on them. Security modelling for CPS is an important mechanism to systematically identify and assess vulnerabilities, threats, and risks throughout system life cycles, and to ultimately ensure system resilience, safety, and reliability. This survey delves into state-of-the-art research on CPS security modelling, encompassing both threat and attack modelling. While these terms are sometimes used interchangeably, …

Simulated Interactive Debugging, Yannic Noller, Erick Chandra, Srinidhi Chandrashekar, Kenny Choo, Cyrille Jegourel, Oka Kurniawan, Christopher M. Poskitt

Research Collection School Of Computing and Information Systems

Debugging software, i.e., the localization of faults and their repair, is a key activity in software engineering. Therefore, effective and efficient debugging is one of the core skills a software engineer must develop. However, the teaching of debugging techniques is usually very limited or only taught in indirect ways, e.g., during software projects. As a result, most Computer Science (CS) students learn debugging only in an ad-hoc and unstructured way. In this work, we present our approach called Simulated Interactive Debugging that interactively guides students along the debugging process. The guidance aims to empower the students to repair their solutions …

Towards Goal-Oriented Intelligent Tutoring Systems In Online Education, Yang Deng, Zifeng Ren, An Zhang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Interactive Intelligent Tutoring Systems (ITSs) enhance the learning experience in online education byfostering effective learning through interactive problem-solving. However, many current ITS models do notfully incorporate proactive engagement strategies that optimize educational resources through thoughtfulplanning and assessment. In this work, we propose a novel and practical task of Goal-oriented IntelligentTutoring Systems (GITS), designed to help students achieve proficiency in specific concepts through a tailoredsequence of exercises and evaluations. We introduce a novel graph-based reinforcement learning framework,named Planning-Assessment-Interaction (PAI), to tackle the challenges of goal-oriented policy learningwithin GITS. This framework utilizes cognitive structure information to refine state representation andguide the selection …

A Fake Friend? Ai Companions Are Exactly That, Seow Hon Tan

Research Collection Yong Pung How School Of Law

In a commentary, SMU Associate Professor of Law Tan Seow Hon discussed how AI companions, which promise emotionally intelligent companionship, have blurred the line between human and machine relationships by mimicking empathy, memory, and affection. She suggested that while such technologies may ease loneliness, they risk fostering narcissism, diminishing real human connection, and replacing authentic friendship with comforting illusions that erode the capacity for love and community.

Deep Reinforcement Learning For Solving The Stochastic E-Waste Collection Problem, Dang Viet Anh Nguyen, Aldy Gunawan, Mustafa Misir, Kwan Hui Lim, Pieter Vansteenwegen

Research Collection School Of Computing and Information Systems

With the growing influence of the internet and information technology, Electrical and Electronic Equipment (EEE) has become a gateway to technological innovations. However, discarded devices, also called e-waste, pose a significant threat to the environment and human health if not properly treated, disposed of, or recycled. In this study, we extend a novel model for the e-waste collection in an urban context: the Heterogeneous VRP with Multiple Time Windows and Stochastic Travel Times (HVRP-MTWSTT). We propose a solution method that employs deep reinforcement learning to guide local search heuristics (DRL-LSH). The contributions of this paper are as follows: (1) HVRP-MTWSTT …

Explainable Sentiment Analysis With Deepseek-R1: Performance, Efficiency, And Few-Shot Learning, Donghao Huang, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have transformed sentiment analysis, yet balancing accuracy, efficiency, and explainability remains a critical challenge. This study presents the first comprehensive evaluation of DeepSeek-R1—an open-source reasoning model—against OpenAI’s GPT-4o and GPT-4o-mini. We test the full 671B model and its distilled variants, systematically documenting few-shot learning curves. Our experiments show DeepSeek-R1 achieves a 91.39% F1 score on 5-class sentiment and 99.31% accuracy on binary tasks with just 5 shots, an eightfold improvement in few-shot efficiency over GPT-4o. Architecture-specific distillation effects emerge, where a 32B Qwen2.5-based model outperforms the 70B Llama-based variant by 6.69 percentage points. While its reasoning …

Generative Ai And Empirical Software Engineering: A Paradigm Shift, Christoph Treude, Margaret-Anne Storey

Research Collection School Of Computing and Information Systems

The widespread adoption of generative AI in software engineering marks a paradigm shift, offering new opportunities to design and utilize software engineering tools while influencing both developers and the artifacts they create. Traditional empirical methods in software engineering, including quantitative, qualitative, and mixed-method approaches, are well established. However, this paradigm shift introduces novel data types and redefines many concepts in the software engineering process. The roles of developers, users, agents, and researchers increasingly overlap, blurring the distinctions between these social and technical actors within the field. This paper examines how integrating AI into software engineering challenges traditional research paradigms. It …

Predict Social Economic Outcomes By Transferred Knowledge With Satellite Imagery, Yang Tang, Shih-Fen Cheng, Yunqiang Zhu, Yichen Yang, Zhiqiang Zou

Research Collection School Of Computing and Information Systems

Traditional deep learning methods and econometric models have played a crucial role in the field of data mining, particularly in the prediction of socioeconomic outcomes. However, socio-economic information is unable to be directly extracted from remote sensing data. So, in this paper, we propose a method to leverage transfer learning to predict socioeconomic indicators (outcomes) through satellite imagery. Specifically, we use road network types as a proxy for socioeconomic factors, which is more effective and stable than using nightlight. We have extracted eleven distinct road topological features to generate reasonable road network types. Given the unique characteristics of road networks, …

Usefulness And Diminishing Returns: Evaluating Social Information In Recommender Systems, Qing Meng, Huiyu Min, Ming Shan Hee, Roy Ka-Wei Lee, Bing Tian Dai, Shuai Xu

Research Collection School Of Computing and Information Systems

Social recommendation, which leverages users’ social information to predict users’ preferences, is a popular branch of recommender systems. Many existing studies have attempted to advance the performance of collaborative filtering methods by leveraging the user-user matrix to enhance user embedding learning with user’s social connections. While the existing social recommender systems have demonstrated good performance in various recommendation tasks, the extent of social information usefulness in recommender systems remains unclear. This paper addresses the research gap by designing experiments to answer three research questions: (i) How useful is social information in varying user-item data sparsity? (ii) How much social information …