Digital Commons Network^™

How Behavioral Science Can Improve The Return On Ai Investments, David De Cremer, Shane Schweitzer, Jack Mcguire, Devesh Narayanan

Research Collection Lee Kong Chian School Of Business

Many AI projects fail because leaders treat adoption as a tech purchase instead of a behavioral change problem. People resist tools that disrupt routines, overreact to visible AI errors, and prefer familiar human judgment. As a result, even good systems fail to gain purchase. Leaders can address this problem by applying “Behavioral Human-Centered AI” across the AI adoption cycle. In the design phrase, companies should co-design with diverse users, add purposeful friction where it improves scrutiny, require beta tests with subgroup results and behavioral input. During adoption, they should frame AI as an augmenter, disclose limits and safeguards, use explainability …

Seeing Culture: A Benchmark For Visual Reasoning And Grounding, Burak Satar, Zhixin Ma, Patrick Amadeus Irrawan, Wilfried Ariel Mulyawan, Jing Jiang, Ee-Peng Lim, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Multimodal vision-language models (VLMs) have made substantial progress in various tasks that require a combined understanding of visual and textual content, particularly in cultural understanding tasks, with the emergence of new cultural datasets. However, these datasets frequently fall short of providing cultural reasoning while underrepresenting many cultures.In this paper, we introduce the Seeing Culture Benchmark (SCB), focusing on cultural reasoning with a novel approach that requires VLMs to reason on culturally rich images in two stages: i) selecting the correct visual option with multiple-choice visual question answering (VQA), and ii) segmenting the relevant cultural artifact as evidence of reasoning. Visual …

From Personas To Talks: Revisiting The Impact Of Personas On Llm-Synthesized Emotional Support Conversations, Shenghan Wu, Yimo Zhu, Wynne Hsu, Mong-Li Lee, Yang Deng

Research Collection School Of Computing and Information Systems

The rapid advancement of Large Language Models (LLMs) has revolutionized the generation of emotional support conversations (ESC), offering scalable solutions with reduced costs and enhanced data privacy. This paper explores the role of personas in the creation of ESC by LLMs. Our research utilizes established psychological frameworks to measure and infuse persona traits into LLMs, which then generate dialogues in the emotional support scenario. We conduct extensive evaluations to understand the stability of persona traits in dialogues, examining shifts in traits post-generation and their impact on dialogue quality and strategy distribution. Experimental results reveal several notable findings: 1) LLMs can …

Adasteer: Your Aligned Llm Is Inherently An Adaptive Jailbreak Defender, Weixiang Zhao, Jiahe Guo, Yulin Hu, Yang Deng, An Zhang, Xingyu Sui, Xinyang Han, Yanyan Zhao, Bing Qin, Tat-Seng Chua, Ting Liu

Research Collection School Of Computing and Information Systems

Despite extensive efforts in safety alignment, large language models (LLMs) remain vulnerable to jailbreak attacks. Activation steering offers a training-free defense method but relies on fixed steering coefficients, resulting in suboptimal protection and increased false rejections of benign inputs. To address this, we propose AdaSteer, an adaptive activation steering method that dynamically adjusts model behavior based on input characteristics. We identify two key properties: Rejection Law (R-Law), which shows that stronger steering is needed for jailbreak inputs opposing the rejection direction, and Harmfulness Law (H-Law), which differentiates adversarial and benign inputs. AdaSteer steers input representations along both the Rejection Direction …

Chain Of Strategy Optimization Makes Large Language Models Better Emotional Supporter, Weixiang Zhao, Xingyu Sui, Xinyang Han, Yang Deng, Yulin Hu, Jiahe Guo, Libo Qin, Qianyun Du, Shijin Wang, Yanyan Zhao, Bing Qin, Ting Liu

Research Collection School Of Computing and Information Systems

The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to users’ emotional needs. Existing supervised fine-tuning (SFT) struggles to address these issues, as it rigidly trains models on single gold-standard responses without modeling nuanced strategy trade-offs. To overcome these limitations, we propose a novel two-stage framework that optimizes strategy selection preferences at each dialogue turn. We first leverage Monte Carlo Tree Search to construct ESC-Pro, a high-quality …

Exploring Autonomous Agents: A Closer Look At Why They Fail When Completing Tasks, Ruofan Lu, Yichen Li, Yintong Huo

Research Collection School Of Computing and Information Systems

Autonomous agent systems powered by Large Language Models (LLMs) have demonstrated promising capabilities in automating complex tasks. However, current evaluations largely rely on success rates without systematically analyzing the interactions, communication mechanisms, and failure causes within these systems. To bridge this gap, we present a benchmark of 34 representative programmable tasks designed to rigorously assess autonomous agents. Using this benchmark, we evaluate three popular open-source agent frameworks combined with two LLM backbones, observing a task completion rate of approximately 50%. Through in-depth failure analysis, we develop a three-tier taxonomy of failure causes aligned with task phases, highlighting planning errors, task …

Envisioning Future Interactive Web Development: Editing Webpage With Natural Language, Truong Hai Dang, Jingyu Xiao, Yintong Huo

Research Collection School Of Computing and Information Systems

The evolution of web applications relies on iterative code modifications, a process that is traditionally manual and time-consuming. While Large Language Models (LLMs) can generate UI code, their ability to edit existing code from new design requirements (e.g., ”center the logo”) remains a challenge. This is largely due to the absence of large-scale, high-quality tuning data to align model performance with human expectations. In this paper, we introduce a novel, automated data generation pipeline that uses LLMs to synthesize a high-quality fine-tuning dataset for web editing, named Instruct4Edit. Our approach generates diverse instructions, applies the corresponding code modifications, and performs …

Context-Aware Hierarchical Taxonomy Generation For Scientific Papers Via Llm-Guided Multi-Aspect Clustering, Kun Zhu, Lizi Liao, Yuxuan Gu, Lei Huang, Xiaocheng Feng, Bing Qin

Research Collection School Of Computing and Information Systems

The rapid growth of scientific literature demands efficient methods to organize and synthesize research findings. Existing taxonomy construction methods, leveraging unsupervised clustering or direct prompting of large language models (LLMs), often lack coherence and granularity. We propose a novel context-aware hierarchical taxonomy generation framework that integrates LLM-guided multi-aspect encoding with dynamic clustering. Our method leverages LLMs to identify key aspects of each paper (e.g., methodology, dataset, evaluation) and generates aspect-specific paper summaries, which are then encoded and clustered along each aspect to form a coherent hierarchy. In addition, we introduce a new evaluation benchmark of 156 expert-crafted taxonomies encompassing 11.6k …

Why Stop At One Error? Benchmarking Llms As Data Science Code Debuggers For Multi-Hop And Multi-Bug Errors, Zhiyu Yang, Shuo Wang, Yukun Yan, Yang Deng

Research Collection School Of Computing and Information Systems

LLMs are transforming software development, yet current code generation and code repair benchmarks mainly assess syntactic and functional correctness in simple, single-error cases. LLMs’ capabilities to autonomously find and fix runtime logical errors in complex data science code remain largely unexplored. To address this gap, we introduce DSDBench: the Data Science Debugging Benchmark, the first benchmark for systematic evaluation of LLMs on multi-hop error tracing and multi-bug detection in data science code debugging. DSDBench adapts datasets from existing data science task benchmarks, such as DABench and MatPlotBench, featuring realistic data science debugging tasks with automatically synthesized multi-hop, multi-bug code snippets. …

Image Captioning Through The Lens Of The Gricean Maxims: Generating Meaningful And Relevant Image Descriptions, Annika Lindh

Doctoral

Image captioning models enable us to automatically generate natural language image descriptions for previously unseen images. It combines the two fields of computer vision and natural language generation, allowing models to interpret the con tent of an image and communicate that knowledge through natural language text.

Research into image captioning has the potential benefit of reducing the gap in digital information availability between fully sighted individuals and those who are visually impaired. However, automatically generated captions often fail to provide the required level of detail and specificity to achieve this goal. Furthermore, current standard evaluation methods are insufficient at measuring …

Mmlu-Prox: A Multilingual Benchmark For Advanced Large Language Model Evaluation, Weihao Xuan, Et. Al.

Research Collection School Of Computing and Information Systems

Existing large language model (LLM) evaluation benchmarks primarily focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. This dual limitation makes it challenging to assess LLMs’ performance in the multilingual setting comprehensively. To fill this gap, we introduce MMLU-ProX, a comprehensive benchmark covering 29 languages, built on an English benchmark. Each language version consists of 11,829 identical questions, enabling direct cross-lingual comparisons. Additionally, to meet efficient evaluation needs, we provide a lite version containing 658 questions per language. To ensure the high quality of MMLU-ProX, we employ a rigorous development process that involves …

Interaction2code: Benchmarking Mllm-Based Interactive Webpage Code Generation From Interactive Prototyping, Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang, Xinyi Xu, Wenxuan Wang, Zhiyao Xu, Yuhang Wang, Michael R. Lyu

Research Collection School Of Computing and Information Systems

Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance on the design-to-code task, i.e., generating UI code from UI mock-ups. However, existing benchmarks only contain static web pages for evaluation and ignore the dynamic interaction, limiting the practicality, usability and user engagement of the generated webpages. To bridge these gaps, we present the first systematic investigation of MLLMs in generating interactive webpages. Specifically, we formulate the Interaction-to-Code task and establish the Interaction2Code benchmark, encompassing 127 unique webpages and 374 distinct interactions across 15 webpage types and 31 interaction categories. Through comprehensive experiments utilizing state-of-theart (SOTA) MLLMs, evaluated via both automatic …

Distillcaps: Enhancing Audio-Language Alignment In Captioning Via Retrieval-Augmented Knowledge Distillation, Thinh Pham, Nghiem Diep, Lizi Liao, Binh Nguyen

Research Collection School Of Computing and Information Systems

Automated audio captioning (AAC) benefits from incorporatingexternal context to interpret complex sounds, but doing so withretrieval-augmented generation (RAG) at inference is sometimesinfeasible due to data availability or incurs significant latency andcomplexity. We propose DistillCaps, a novel training-time frame-work that leverages RAG to guide knowledge distillation for im-proved audio-language alignment, while lessening the relianceon retrieval during inference. In our framework, a RAG-equippedteacher model retrieves relevant textual information (e.g., simi-lar captions) for each audio clip and uses it for training to gener-ate context-enriched captions. Simultaneously, a student model istrained to imitate this teacher, learning to produce high-qualitycaptions from audio alone. We further …

Instructors’ Strategies In Creating And Implementing Constructivist Llm-Based Learning Activities, Emily Aurelia, Shun Yi Yeo, Michelle Lui, Effie Lai-Chong Law, Anthony Tang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) are increasingly being integrated into educational settings, enabling more adoption of constructivist teaching and learning approaches in classrooms. This paper explores the strategies instructors are currently using to incorporate LLMs into learning activities that align with constructivist principles, which emphasize that learners actively construct their own knowledge. Through interviews with nine instructors who have designed eleven distinct LLM-based activities and using reflexive thematic analysis, this study identifies various types of learning activities with respect to four different aspects of the constructivist learning theory. The strategies employed and challenges faced to foster constructivist student-LLM interaction were also …

Enhancing The Acceptability Of Decision-Making System Using Xai Case: Crime Profiling System, Mohamed Saeed Al Shamsi

Theses

In the current world, we need to place more emphasis on how easily interpretable, accurate, and acceptable data analysis results are, given that essential operations in law enforcement, among other sectors, are backed up by the use of complex computing systems. Crime profiling systems that use crime data for profiling encounter major problems because they depend on algorithm-based methods. These methods can be ambiguous and inaccurate, leading to low public acceptability. The study investigates major problems with Complex Crime profiling systems (CPS) because their unexplained algorithms result in system performance issues and public scepticism. XAI provides a solution to handle …

Explainable Sentiment Analysis With Deepseek-R1: Performance, Efficiency, And Few-Shot Learning, Donghao Huang, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have transformed sentiment analysis, yet balancing accuracy, efficiency, and explainability remains a critical challenge. This study presents the first comprehensive evaluation of DeepSeek-R1—an open-source reasoning model—against OpenAI’s GPT-4o and GPT-4o-mini. We test the full 671B model and its distilled variants, systematically documenting few-shot learning curves. Our experiments show DeepSeek-R1 achieves a 91.39% F1 score on 5-class sentiment and 99.31% accuracy on binary tasks with just 5 shots, an eightfold improvement in few-shot efficiency over GPT-4o. Architecture-specific distillation effects emerge, where a 32B Qwen2.5-based model outperforms the 70B Llama-based variant by 6.69 percentage points. While its reasoning …

Generative Ai And Empirical Software Engineering: A Paradigm Shift, Christoph Treude, Margaret-Anne Storey

Research Collection School Of Computing and Information Systems

The widespread adoption of generative AI in software engineering marks a paradigm shift, offering new opportunities to design and utilize software engineering tools while influencing both developers and the artifacts they create. Traditional empirical methods in software engineering, including quantitative, qualitative, and mixed-method approaches, are well established. However, this paradigm shift introduces novel data types and redefines many concepts in the software engineering process. The roles of developers, users, agents, and researchers increasingly overlap, blurring the distinctions between these social and technical actors within the field. This paper examines how integrating AI into software engineering challenges traditional research paradigms. It …

Intentionframe: A Semi-Structured, Multi-Aspect Framework For Fine-Grained Conversational Intention Understanding, Zailong Tian, Zhuoheng Han, Lizi Liao, Lizi Liao

Research Collection School Of Computing and Information Systems

Understanding user intentions in multi-turn dialogues is critical for conversational AI, yet existing approaches—relying on rigid slot-value structures or unstructured free-text—fail to fully capture conversational complexity. In this paper, we propose IntentionFrame, a semi-structured framework inspired by psychological and cognitive intention theories, which organizes conversational intents into four interrelated aspects: situation, emotion, action, and knowledge. This design not only retains interpretability but also provides LLMs with a rich context to accurately parse and respond to nuanced user inputs. To efficiently scale IntentionFrame annotations, we introduce a Weakly-supervised Reinforced Generation (WeRG) method that leverages a small set of high-quality human annotations …

Attention Mapping For Hallucination Reduction In Arabic Financial Using Retrieval-Augmented Generation Systems, Hasan Abdulameer Hasan, Khaldoun H. Al-Hussayni, Ali Z. K. Matloob

Journal of Intelligent Informatics, Networking, and Cybersecurity

This research introduces a customized attention visualization framework for mitigating hallucinations in Arabic Retrieval-Augmented Generation (RAG) systems tailored for financial document analysis. The proposed architecture extends MarBERT with a dual-stage attention supervision mechanism and a hallucination-aware loss formulation, trained on a newly constructed dataset of 7,000 annotated Arabic financial query-context pairs. A grounding alignment score is computed over attended tokens, and generated responses are rejected when falling below a dynamically adjusted precision-aware threshold,serving as the core decision-making approach for hallucination detection. The system achieves 95.04% classification accuracy, 95.84% precision, 94.12% recall, and an F1 score of 94.97%, outperforming AraELECTRA and …

A Comparative Study For Classical Machine Learning Models For Swahili Social Media Sentiment Analysis, Mahadia Tunga, Davis David

Tanzania Journal of Science

Despite sentiment analysis being one of the most popular applications in Natural Language Processing (NLP), most studies are skewed towards languages with a rich corpus (language database). Less emphasis has been placed on low-resource languages like Swahili. Swahili is the official language of the African Union and of 4 countries in East Africa, and is spoken by many people on the African continent. This study performed sentiment analysis using 3,000 tweets hosted on the Zindi Africa platform. Data was processed using a term frequency-inverse document frequency vectorization method, and five classical machine learning algorithms (RandomForest, XgBoost, and CatBoost, HistogramGradientBoost, LightGradientBoos) …

Towards Mitigation Of Hallucination For Llm-Empowered Agents: Progressive Generalization Bound Exploration And Watchdog Monitor, Siyuan Liu, Wenjing Liu, Zhiwei Xu, Xin Wang, Bo Chen, Tao Li

Michigan Tech Publications

Empowered by large language models (LLMs), intelligent agents have become a popular paradigm for interacting with open environments to facilitate AI deployment. However, hallucinations generated by LLMs - where outputs are inconsistent with facts - pose a significant challenge, undermining the credibility of intelligent agents. Only if hallucinations can be mitigated, the intelligent agents can be used in real-world without any catastrophic risk. Therefore, effective detection and mitigation of hallucinations are crucial to ensure the dependability of agents. Unfortunately, the related approaches either depend on white-box access to LLMs or fail to accurately identify hallucinations. To address the challenge posed …

Large Language Models As Information Providers For Appropriate Antimicrobial Use: Computational Text Analysis And Expert-Rated Comparison Of Chatgpt, Claude And Gemini, Marcello Di Pumpo, Maria Rosaria Gualano, Danilo Buonsenso, Francesca Raffaelli, Daniele Donà, Vittorio Maio, Patrizia Laurenti, Walter Ricciardi, Leonardo Villani

College of Population Health Faculty Papers

OBJECTIVES: Antimicrobial resistance is a critical public health threat. Large language models (LLMs) show great capability for providing health information. This study evaluates the effectiveness of LLMs in providing information on antibiotic use and infection management.

METHODS: Using a mixed-method approach, responses to healthcare expert-designed scenarios from ChatGPT 3.5, ChatGPT 4.0, Claude 2.0 and Gemini 1.0, in both Italian and English, were analysed. Computational text analysis assessed readability, lexical diversity and sentiment, while content quality was assessed by three experts via DISCERN tool.

RESULTS: 16 scenarios were developed. A total of 101 outputs and 5454 Likert-scale (1-5) scores were obtained …

Graphrag-Enabled Local Large Language Model For Gestational Diabetes Mellitus: Development Of A Proof-Of-Concept, Edmund Evangelista, Fathima Ruba, Salman Bukhari, Amril Nazir, Ravishankar Sharma

All Works

Background: Gestational diabetes mellitus (GDM) is a prevalent chronic condition that affects maternal and fetal health outcomes worldwide, increasingly in underserved populations. While generative artificial intelligence (AI) and large language models (LLMs) have shown promise in health care, their application in GDM management remains underexplored. Objective: This study aimed to investigate whether retrieval-augmented generation techniques, when combined with knowledge graphs (KGs), could improve the contextual relevance and accuracy of AI-driven clinical decision support. For this, we developed and validated a graph-based retrieval-augmented generation (GraphRAG)–enabled local LLM as a clinical support tool for GDM management, assessing its performance against open-source LLM …

Cv: Teresa Bixby (Chemistry), Teresa Bixby

Chemistry Department Faculty Curricula Vitae

No abstract provided.

Zero Trust - The Future Of Cybersecurity, Danial Haider, Shougfta Mushtaq, Hasnat Ali, Mazliham Mohd Su’Ud

Journal of Informatics and Web Engineering

The recent Zero-Trust Architecture (ZTA) is progressively adopted to the develop network security by assuming no implicit trust within or outside an organization’s boundary. Though, ZTA faces substantial challenges in detecting sophisticated and developing cyber threats, particularly due to its trust on traditional security mechanisms that struggle to manage internal threats and sophisticated attack techniques. To report these shortcomings, the proposed study discovers the combination of advanced machine learning (ML) and deep learning (DL) performances to improve the anomaly detection proficiencies within ZTA environments. The study develops the CICIDS2017 dataset, which contains diverse and realistic network traffic patterns, to assess …

The Role Of Generative Ai In E-Commerce Recommender Systems: Methods, Trends And Insights, Kai-Ze Liau, Heru Agus Santoso

Journal of Informatics and Web Engineering

Recommender systems have existed for decades, shaping how people consume digital content, receive information, and engage in day-to-day activities, among others. Undoubtably, recommender systems also play a crucial role in e-commerce applications as well, with industry players like Amazon, AliBaba, eBay using recommender systems within their ecosystems to give suitable and value-driven insights. However, recommender systems face some main concerns such as data sparsity, cold-start problems and so on. As a result, research is currently ongoing to solve these issues and provide high-quality recommendations to consumers. This review aims to identify prevailing gaps surrounding these issues by analysing existing research …

Research Article Recommender Systems: A Comprehensive Review Of Models, Approaches And Evaluation Metrics, Sir-Yuean Lim, Noramiza Hashim, Lanh Le Thanh

Journal of Informatics and Web Engineering

With the advent of the current digital era, individuals across the developed world are commonly equipped with devices that can access vast amounts of information at their fingertips. What was considered an impossible feat was realized through remarkable technological advancements. This positive transformation has had a profound impact on education, where traditional knowledge management, such as libraries, are no longer a primary determinant of a student’s academic success. Instead, it has been replaced by the internet as a medium for learning, practicing, and topic exploration. However, the sheer volume of the ever-increasing information available online can easily overwhelm a user, …

Lightweight String Similarity Approaches For Duplicate Detection In Academic Titles, Fahrudin Mukti Wibowo, Muhammad Zidny Nafan, Muhamad Azrino Gustalika, Harinda Fernando, Muhammad Hussain, Nur Afiqah Sahadun

Journal of Informatics and Web Engineering

This study addresses the critical challenge of detecting duplicate final year project (FYP) titles in academic institutions, where minor variations like reordering, synonyms, and paraphrasing often obscure plagiarism. We systematically evaluate four string similarity algorithms - Jaro-Winkler, Levenshtein Edit Distance, TF-IDF with Cosine Similarity, and Jaccard Similarity - using a synthetic dataset of 250 title pairs representing common duplication patterns. Our experiments reveal that character-based methods (Jaro-Winkler and Edit Distance) achieve perfect detection (F1-score=1.0) for literal matches, including typographical variations and phrase reordering. At the same time, TF-IDF demonstrates strong semantic capability (F1-score=0.95), albeit with some false positives. Jaccard Similarity …

Needs Analysis Of Interactive Learning Media For Teachingheat Transfer In Physics: Students’ And Teachers’Perspectives, Rizki Eka Shintya, Ika Mustika Sari, Ida Kaniawati

Jurnal Pendidikan Sains

Heat transfer material in physics is known to be abstract. This study specifically analyzes the conceptual difficulties, and visualization needs specific to this material, which is different from the analysis of physics needs in general. This analysis does not only look for the need for traditional media but explicitly looks for the need for interactive media (simulation, animation, virtual laboratory). Therefore, this study aims to identify the need for interactive learning media in physics education, specifically in the teaching of heat transfer, from the perspective of students and teachers in West Java, Indonesia. Using a quantitative descriptive survey, data were …

Development Of Digital Comic Learning Media Based On Mobile Learning On Media: Making Media Materials In Plant Tissue Culture, Sufina Intan Pertiwi, Fauziyah Harahap, Aline Gusti Yolanda Harahap, Marsya Amira Indrawati, Aline Jelita Christiani Simanullang, Yanda Febry Gracea Br Tarigan, Maureen Efriyanti Sitinjak

Jurnal Pendidikan Sains

his research was conducted based on student needs analysis, analysis of learner characteristics, and curriculum analysis, namely the lack of visualization and presentation of concepts and the lack of utilization of interactive technology in the learning process on media material and media making in tissue culture for college students. This study intends to develop digital comic media based on mobile learning and making media materials in tissue culture. This research method used research and development (R&D) with the ADDIE model: analyze, design, develop, implement, and evaluate. The subjects of this study were students of the Department of Biology, Universitas Negeri …