Open Access. Powered by Scholars. Published by Universities.®

Programming Languages and Compilers Commons™

Open Access. Powered by Scholars. Published by Universities.®

1,842 Full-Text Articles 3,360 Authors 797,974 Downloads 137 Institutions

All Articles in Programming Languages and Compilers

Faceted Search

1,842 full-text articles. Page 4 of 79.

Evowiki: Evaluating Llms On Evolving Knowledge, Wei TANG, Yixin CAO, Yang DENG, Jiahao YING, Bo WANG, Yizhe YANG, Yuyue ZHAO, Qi ZHANG, Xuanjing HUANG, Yu-Gang JIANG, Yong LIAO 2025 Singapore Management University

Evowiki: Evaluating Llms On Evolving Knowledge, Wei Tang, Yixin Cao, Yang Deng, Jiahao Ying, Bo Wang, Yizhe Yang, Yuyue Zhao, Qi Zhang, Xuanjing Huang, Yu-Gang Jiang, Yong Liao

Research Collection School Of Computing and Information Systems

Knowledge utilization is a critical aspect of LLMs, and understanding how they adapt to evolving knowledge is essential for their effective deployment. However, existing benchmarks are predominantly static, failing to capture the evolving nature of LLMs and knowledge, leading to inaccuracies and vulnerabilities such as contamination. In this paper, we introduce EvoWiki, an evolving dataset designed to reflect knowledge evolution by categorizing information into stable, evolved, and uncharted states. EvoWiki is fully auto-updated, enabling precise evaluation of continuously changing knowledge and newly released LLMs. Through experiments with Retrieval-Augmented Generation (RAG) and Continual Learning (CL), we evaluate how effectively LLMs adapt …


Fact-Audit: An Adaptive Multi-Agent Framework For Dynamic Fact-Checking Evaluation Of Large Language Models, Hongzhan LIN, Yang DENG, Yuxuan GU, Wenxuan ZHANG, Jing MA, See-Kiong NG, Tat-Seng CHUA 2025 Singapore Management University

Fact-Audit: An Adaptive Multi-Agent Framework For Dynamic Fact-Checking Evaluation Of Large Language Models, Hongzhan Lin, Yang Deng, Yuxuan Gu, Wenxuan Zhang, Jing Ma, See-Kiong Ng, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have significantly advanced the fact-checking studies. However, existing automated fact-checking evaluation methods rely on static datasets and classification metrics, which fail to automatically evaluate the justification production and uncover the nuanced limitations of LLMs in fact-checking. In this work, we introduce FACT-AUDIT, an agent-driven framework that adaptively and dynamically assesses LLMs’ fact-checking capabilities. Leveraging importance sampling principles and multi-agent collaboration, FACT-AUDIT generates adaptive and scalable datasets, performs iterative model-centric evaluations, and updates assessments based on model-specific responses. By incorporating justification production alongside verdict prediction, this framework provides a comprehensive and evolving audit of LLMs’ factual reasoning …


Knowledge Boundary Of Large Language Models: A Survey, Moxin LI, Yong ZHAO, Wenxuan ZHANG, Shuaiyi LI, Wenya XIE, See-Kiong NG, Tat-Seng CHUA, Yang DENG 2025 Singapore Management University

Knowledge Boundary Of Large Language Models: A Survey, Moxin Li, Yong Zhao, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, See-Kiong Ng, Tat-Seng Chua, Yang Deng

Research Collection School Of Computing and Information Systems

Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses. This highlights the critical need to understand the knowledge boundary of LLMs, a concept that remains inadequately defined in existing research. In this survey, we propose a comprehensive definition of the LLM knowledge boundary and introduce a formalized taxonomy categorizing knowledge into four distinct types. Using this foundation, we systematically review the field through three key lenses: the motivation for studying LLM knowledge boundaries, …


Beware Of Your Po! Measuring And Mitigating Ai Safety Risks In Role-Play Fine-Tuning Of Llms, Weixiang ZHAO, Yulin HU, Yang DENG, Jiahe GUO, Xingyu SUI, Xinyang HAN, An ZHANG, Yanyan ZHAO, Bing QIN, Tat-Seng CHUA, Ting LIU 2025 Singapore Management University

Beware Of Your Po! Measuring And Mitigating Ai Safety Risks In Role-Play Fine-Tuning Of Llms, Weixiang Zhao, Yulin Hu, Yang Deng, Jiahe Guo, Xingyu Sui, Xinyang Han, An Zhang, Yanyan Zhao, Bing Qin, Tat-Seng Chua, Ting Liu

Research Collection School Of Computing and Information Systems

Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses. This highlights the critical need to understand the knowledge boundary of LLMs, a concept that remains inadequately defined in existing research. In this survey, we propose a comprehensive definition of the LLM knowledge boundary and introduce a formalized taxonomy categorizing knowledge into four distinct types. Using this foundation, we systematically review the field through three key lenses: the motivation for studying LLM knowledge boundaries, …


Browsing Like Human: A Multimodal Web Agent With Experiential Fast-And-Slow Thinking, Haohao LUO, Jiayi KUANG, Wei LIU, Ying SHEN, Jian LUAN, Yang DENG 2025 Singapore Management University

Browsing Like Human: A Multimodal Web Agent With Experiential Fast-And-Slow Thinking, Haohao Luo, Jiayi Kuang, Wei Liu, Ying Shen, Jian Luan, Yang Deng

Research Collection School Of Computing and Information Systems

Automating web navigation which aims to build a web agent that follows user instructions to complete tasks like booking flights by interacting with websites, has received increasing attention due to its practical value. Although existing web agents are mostly equipped with visual perception, planning, and memory abilities, their reasoning process are still deviate from human cognition. In this work, we study the human thought pattern to empower agent with more human-like abilities in web navigation. To tackle this problem, we propose a novel multimodal web agent framework called WebExperT, which is designed to emulate the human planning process of “thinking …


Mpo: Multilingual Safety Alignment Via Reward Gap Optimization, Weixiang ZHAO, Yulin HU, Yang DENG, Tongtong WU, Wenxuan ZHANG, Jiahe GUO, An ZHANG, Yanyan ZHAO, Bing QIN, Tat-Seng CHUA, Ting LIU 2025 Singapore Management University

Mpo: Multilingual Safety Alignment Via Reward Gap Optimization, Weixiang Zhao, Yulin Hu, Yang Deng, Tongtong Wu, Wenxuan Zhang, Jiahe Guo, An Zhang, Yanyan Zhao, Bing Qin, Tat-Seng Chua, Ting Liu

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have become increasingly central to AI applications worldwide, necessitating robust multilingual safety alignment to ensure secure deployment across diverse linguistic contexts. Existing preference learning methods for safety alignment, such as RLHF and DPO, are primarily monolingual and struggle with noisy multilingual data. To address these limitations, we introduce Multilingual reward gaP Optimization (MPO), a novel approach that leverages the well-aligned safety capabilities of the dominant language (e.g., English) to improve safety alignment across multiple languages. MPO directly minimizes the reward gap difference between the dominant language and target languages, effectively transferring safety capabilities while preserving the …


Leveraging Reviewer Experience In Code Review Comment Generation, Hong Yi LIN, Patanamon THONGTANUNAM, Christoph TREUDE, Michael W. GODFREY, Chunhua LIU, Wachiraphan CHAROENWET 2025 Singapore Management University

Leveraging Reviewer Experience In Code Review Comment Generation, Hong Yi Lin, Patanamon Thongtanunam, Christoph Treude, Michael W. Godfrey, Chunhua Liu, Wachiraphan Charoenwet

Research Collection School Of Computing and Information Systems

Modern code review is a ubiquitous software quality assurance process aimed at identifying and resolving potential issues (e.g., functional, evolvability) within newly written code. Despite its effectiveness, the process demands large amounts of effort from the human reviewers involved. To help alleviate this workload, researchers have trained various deep learning based language models to imitate human reviewers in providing natural language code reviews for submitted code. Formally, this automation task is known as code review comment generation. Prior work has demonstrated improvements in code review comment generation by leveraging machine learning techniques and neural models, such as transfer learning and …


Debate, Reflect, And Distill: Multi-Agent Feedback With Tree-Structured Preference Optimization For Efficient Language Model Enhancement, Xiaofeng ZHOU, Heyan HUANG, Lizi LIAO 2025 Singapore Management University

Debate, Reflect, And Distill: Multi-Agent Feedback With Tree-Structured Preference Optimization For Efficient Language Model Enhancement, Xiaofeng Zhou, Heyan Huang, Lizi Liao

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) continue to set new standards in knowledge-intensive and complex reasoning tasks, yet their high computational demands limit widespread adoption. While distilling large models into smaller ones offers a sustainable solution, current techniques—such as static knowledge distillation, resource-intensive reinforcement learning from human feedback, or limited self-reflection—struggle to yield substantial and lasting performance gains. In this paper, we present a novel Debate and Reflect (D&R) framework that orchestrates multi-turn debates between smaller models and stronger teacher models, eliciting actionable feedback (e.g., error analysis, corrective strategies) to guide student models. Further, we introduce Tree-structured Direct Preference Optimization (T-DPO) to …


Memotune: A Measure And Moment-Driven Fine-Tuning Framework For Quantized Large Language Models, Yun ZHANG, Xue GENG, Lizi LIAO, Jintong SUN, Minghe YU, Ge YU 2025 Singapore Management University

Memotune: A Measure And Moment-Driven Fine-Tuning Framework For Quantized Large Language Models, Yun Zhang, Xue Geng, Lizi Liao, Jintong Sun, Minghe Yu, Ge Yu

Research Collection School Of Computing and Information Systems

Quantizing large language models (LLMs) is essential for reducing memory and computational costs in natural language processing. Existing methods combine quantization with parameter-efficient fine-tuning but often fail to meet practical performance requirements. This paper introduces MeMoTune, a novel fine-tuning framework for quantized LLMs. By employing a measure and moment approach within a low-rank approximation framework in probability measure space, MeMoTune optimizes the objective function for superior fine-tuning results. The update process is further refined through scaled gradient, enhancing convergence efficiency and noise robustness. Experiments on tasks like text generation, summarization, and understanding show MeMoTune significantly outperforms state-of-the-art methods, e.g. fine-tuning …


Xfinbench: Benchmarking Llms In Complex Financial Problem Solving And Reasoning, Zhihan ZHANG, Yixin CAO, Lizi LIAO 2025 Singapore Management University

Xfinbench: Benchmarking Llms In Complex Financial Problem Solving And Reasoning, Zhihan Zhang, Yixin Cao, Lizi Liao

Research Collection School Of Computing and Information Systems

Solving financial problems demands complex reasoning, multimodal data processing, and a broad technical understanding, presenting unique challenges for current large language models (LLMs). We introduce **XFinBench**, a novel benchmark with 4,235 examples designed to evaluate LLM’s ability in solving comple**X**, knowledge-intensive **Fin**ancial problems across diverse graduate-level finance topics with multi-modal context. We identify five core capabilities of LLMs using XFinBench, i.e., _terminology understanding_, _temporal reasoning_, _future forecasting_, _scenario planning_, and _numerical modelling_. Upon XFinBench, we conduct extensive experiments on 18 leading models. The result shows that o1 is the best-performing text-only model with an overall accuracy of 67.3%, but still …


The Impact Of Artificial Intelligence As An Intervening Variable Between The Digital Government Strategy And Competency Development "An Applied Study At Sharjah Police Sciences Academy ", Elsayed Kamal Risha, Abd al-Rahman al-Naqbi 2025 Sharjah Police Sciences Academy

The Impact Of Artificial Intelligence As An Intervening Variable Between The Digital Government Strategy And Competency Development "An Applied Study At Sharjah Police Sciences Academy ", Elsayed Kamal Risha, Abd Al-Rahman Al-Naqbi

Journal of Police and Legal Sciences

The study aimed to determine the impact of the digital government strategy on competencies development, through artificial intelligence as an intervening variable, and to achieve the objectives, the study relied on the quantitative approach and the questionnaire was used as the main tool for collecting data. The study community represented officers, non-commissioned officers and individuals at the Sharjah Academy for Police Sciences, and the study sample amounted to 30 affiliates, i.e. the total number of employees in the Competency Development Department at the Academy. The study reached a set of results, the most prominent of which are:

- The existence …


Rattler Python, Samer Jabor 2025 St. Mary's University

Rattler Python, Samer Jabor

Systems Manuals - 2026

The Rattler Python project is an interactive game-based learning system that intends to teach the basic concepts of Python programming through guided instruction, gameplay challenges, and review-based assessments. The document contains a proposal for this system consisting of problem definition, background research, existing solutions, and the proposed product, together with the system scope, assumptions, and the organization of the remainder of this document.


Query Understanding In Llm-Based Conversational Information Seeking, Yifei YUAN, Zahra ABBASIANTAEB, Mohammad ALIANNEJADI, Yang DENG 2025 Singapore Management University

Query Understanding In Llm-Based Conversational Information Seeking, Yifei Yuan, Zahra Abbasiantaeb, Mohammad Aliannejadi, Yang Deng

Research Collection School Of Computing and Information Systems

Query understanding in CIS involves accurately interpreting user intent through context-aware interactions. This includes resolving ambiguities, refining queries, and adapting to evolving information needs. LLM enhance this process by interpreting nuanced language and adapting dynamically, improving the relevance and precision of search results in real-time. In this tutorial, we explore advanced techniques to enhance query understanding in LLM-based CIS systems. We delve into LLM-driven methods for developing robust evaluation metrics to assess query understanding quality in multi-turn interactions, strategies for building more interactive systems, and applications like proactive query management and query reformulation. We also discuss key challenges in integrating …


Llm-Based Multi-Agent Systems For Software Engineering: Literature Review, Vision And The Road Ahead, Junda HE, Christoph TREUDE, David LO 2025 Singapore Management University

Llm-Based Multi-Agent Systems For Software Engineering: Literature Review, Vision And The Road Ahead, Junda He, Christoph Treude, David Lo

Research Collection School Of Computing and Information Systems

Integrating Large Language Models (LLMs) into autonomous agents marks a significant shift in the research landscape by offering cognitive abilities that are competitive with human planning and reasoning. This paper explores the transformative potential of integrating Large Language Models into Multi-Agent (LMA) systems for addressing complex challenges in software engineering (SE). By leveraging the collaborative and specialized abilities of multiple agents, LMA systems enable autonomous problem-solving, improve robustness, and provide scalable solutions for managing the complexity of real-world software projects. In this paper, we conduct a systematic review of recent primary studies to map the current landscape of LMA applications …


Crow: Eliminating Backdoors From Large Language Models Via Internal Consistency Regularization, Nay Myat MIN, Long H. PHAM, Yige LI, Jun SUN 2025 Singapore Management University

Crow: Eliminating Backdoors From Large Language Models Via Internal Consistency Regularization, Nay Myat Min, Long H. Pham, Yige Li, Jun Sun

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) are vulnerable to backdoor attacks that manipulate outputs via hidden triggers. Existing defense methods—designed for vision/text classification tasks—fail for text generation. We propose Internal Consistency Regularization (CROW), a defense leveraging the observation that backdoored models exhibit unstable layer-wise hidden representations when triggered, while clean models show smooth transitions. CROW enforces consistency across layers via adversarial perturbations and regularization during finetuning, neutralizing backdoors without requiring clean reference models or trigger knowledge—only a small clean dataset. Experiments across Llama-2 (7B, 13B), CodeLlama (7B, 13B), and Mistral-7B demonstrate CROW’s effectiveness: it achieves significant reductions in attack success rates across …


Hps: Hard Preference Sampling For Human Preference Alignment, Xiandong ZOU, Wanyu LIN, Yuchen LI, Pan ZHOU 2025 Singapore Management University

Hps: Hard Preference Sampling For Human Preference Alignment, Xiandong Zou, Wanyu Lin, Yuchen Li, Pan Zhou

Research Collection School Of Computing and Information Systems

Aligning Large Language Model (LLM) responses with human preferences is vital for building safe and controllable AI systems. While preference optimization methods based on PlackettLuce (PL) and Bradley-Terry (BT) models have shown promise, they face challenges such as poor handling of harmful content, inefficient use of dispreferred responses, and, specifically for PL, high computational costs. To address these issues, we propose Hard Preference Sampling (HPS), a novel framework for robust and efficient human preference alignment. HPS introduces a training loss that prioritizes the most preferred response while rejecting all dispreferred and harmful ones. It emphasizes “hard” dispreferred responses — those …


Milpbench: A Large-Scale Benchmark Test Suite For Mixed Integer Linear Programming Problems, Huigen YE, Yaoyang CHENG, Hua XU, Zhiguang CAO, Hanzhang QIN 2025 Singapore Management University

Milpbench: A Large-Scale Benchmark Test Suite For Mixed Integer Linear Programming Problems, Huigen Ye, Yaoyang Cheng, Hua Xu, Zhiguang Cao, Hanzhang Qin

Research Collection School Of Computing and Information Systems

Mixed-integer linear programming (MILP) is a cornerstone of optimization with applications across numerous domains. However, the development and evaluation of MILP-solving algorithms are hindered by existing benchmark datasets, which are often limited in scale, lack diversity, and are poorly structured, making them inadequate for systematic testing across different solving approaches, especially for machine learning (ML)-based methods. To address these issues, we introduce MILPBench, a large-scale benchmark suite comprising 100,000 MILP instances organized into 60 well-categorized classes. Using structural properties and embedding similarity metrics, we developed a novel classification framework to ensure both intra-class homogeneity and inter-class diversity. In addition to …


Developing Sky Plots Of Rocket Launches From Gps Scintillation Data, Ishaan Dey, Kshitija Deshpande 2025 Embry-Riddle Aeronautical University

Developing Sky Plots Of Rocket Launches From Gps Scintillation Data, Ishaan Dey, Kshitija Deshpande

Beyond: Undergraduate Research Journal

Sky plots displaying GPS satellite and rocket launch trajectories are developed to determine the spatial correlation between satellites that display ionospheric scintillations and heavy thrust-producing rockets. The trajectories of three major Falcon Heavy and Artemis 1 rocket launches are used within this paper. Python code is utilized to compute and plot Ionospheric Pierce Point (IPP) coordinates which are then used to produce satellite trajectories from the receiver's point of view. Rocket latitude, longitude, and altitude data is integrated within the code to provide extensive detail into the location of the rocket in relation to GPS satellites that displayed scintillations from …


Learning And Optimization Under Human-Centric Considerations, Qian SHAO 2025 Singapore Management University

Learning And Optimization Under Human-Centric Considerations, Qian Shao

Dissertations and Theses Collection (Open Access)

This dissertation investigates learning and optimization problems shaped by humancentric considerations, such as preferences, demonstrations, behavioral patterns, and resource constraints. As real-world decision-making increasingly involves interaction with human agents, data, and limitations, modeling these factors becomes critical for building practical, adaptive, and robust systems.

The research spans four domains. First, we study preference-aware delivery routing by learning implicit practitioner preferences and incorporating them into a hierarchical route optimization framework. Second, we develop imitation learning methods for cost-constrained settings, enabling agents to mimic expert behavior while respecting safety and resource limitations. Third,we explore early rumor detection in data-limited environments, integrating large …


Large Language Models For Logical Fallacy Detection, NICOLE ANNE HUI-YING TEO, Donghao HUANG, Erik CAMBRIA, Zhaoxia WANG 2025 Singapore Management University

Large Language Models For Logical Fallacy Detection, Nicole Anne Hui-Ying Teo, Donghao Huang, Erik Cambria, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Identifying logical fallacies is essential for maintaining log-ical reasoning and reducing false information in a variety of domains, such as the media, law, and education. We present an extensive study on the use of large language models (LLMs) for logical fallacy detection and provide a comparative overview of model performance across various fallacy classes. We evaluate the logical fallacy detection capabilities of multiple state-of-the-art models (LLaMA, Qwen, Gemma, Phi) utilizing accuracy, precision, recall, and F1-score as assessment measures. Accord-ing to our findings, our models do well on simple fallacies like “circular reasoning,” but they have trouble with more interpretive reasoning …


Digital Commons powered by bepress