Open Access. Powered by Scholars. Published by Universities.®

Artificial Intelligence and Robotics Commons™

Open Access. Powered by Scholars. Published by Universities.®

Research Collection School Of Computing and Information Systems

Discipline
Keyword
Publication Year
File Type

Articles 391 - 420 of 1664

Full-Text Articles in Artificial Intelligence and Robotics

Agentstudio: A Toolkit For Building General Virtual Agents, Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, Shuicheng Yan Apr 2025

Agentstudio: A Toolkit For Building General Virtual Agents, Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, Shuicheng Yan

Research Collection School Of Computing and Information Systems

General virtual agents need to handle multimodal observations, master complex action spaces, and self-improve in dynamic, open-domain environments. However, existing environments are often domain-specific and require complex setups, which limits agent development and evaluation in real-world settings. As a result, current evaluations lack in-depth analyses that decompose fundamental agent capabilities. We introduce AgentStudio, a trinity of environments, tools, and benchmarks to address these issues. AgentStudio provides a lightweight, interactive environment with highly generic observation and action spaces, e.g., video observations and GUI/API actions. It integrates tools for creating online benchmark tasks, annotating GUI elements, and labeling actions in videos. Based …

Go to article

On Minimizing Adversarial Counterfactual Error In Adversarial Reinforcement Learning, Roman Belaire, Arunesh Sinha, Pradeep Varakantham Apr 2025

On Minimizing Adversarial Counterfactual Error In Adversarial Reinforcement Learning, Roman Belaire, Arunesh Sinha, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

Deep Reinforcement Learning (DRL) policies are highly susceptible to adversarial noise in observations, which poses significant risks in safety-critical scenarios. The challenge inherent to adversarial perturbations is that by altering the information observed by the agent, the state becomes only partially observable. Existing approaches address this by either enforcing consistent actions across nearby states or maximizing the worst-case value within adversarially perturbed observations. However, the former suffers from performance degradation when attacks succeed, while the latter tends to be overly conservative, leading to suboptimal performance in benign settings. We hypothesize that these limitations stem from their failing to account for …

Go to article

Semantic Loss-Guided Data-Efficient Supervised Fine-Tuning For Safe Responses In Llms, Yuxiao Lu, Pradeep Varakantham, Pradeep Varakantham Apr 2025

Semantic Loss-Guided Data-Efficient Supervised Fine-Tuning For Safe Responses In Llms, Yuxiao Lu, Pradeep Varakantham, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) generating unsafe responses to toxic prompts is a significant issue in their applications. While various efforts aim to address this safety concern, previous approaches often demand substantial human data collection or rely on the less dependable option of using another LLM to generate corrective data. In this paper, we aim to take this problem and overcome limitations of requiring significant high-quality human data. Our method requires only a small set of unsafe responses to toxic prompts, easily obtained from the unsafe LLM itself. By employing a semantic cost combined with a negative Earth Mover Distance (EMD) …

Go to article

Bootstrapping Language Models With Dpo Implicit Rewards, Changyu Chen, Zichen Liu, Chao Du, Tianyu Pang, Qian Liu, Arunesh Sinha, Pradeep Varakantham, Min Lin Apr 2025

Bootstrapping Language Models With Dpo Implicit Rewards, Changyu Chen, Zichen Liu, Chao Du, Tianyu Pang, Qian Liu, Arunesh Sinha, Pradeep Varakantham, Min Lin

Research Collection School Of Computing and Information Systems

Human alignment in large language models (LLMs) is an active area of research. A recent groundbreaking work, direct preference optimization (DPO), has greatly simplified the process from past work in reinforcement learning from human feedback (RLHF) by bypassing the reward learning stage in RLHF. DPO, after training, provides an implicit reward model. In this work, we make a novel observation that this implicit reward model can by itself be used in a bootstrapping fashion to further align the LLM. Our approach is to use the rewards from a current LLM model to construct a preference dataset, which is then used …

Go to article

Mimic: Ai And Ar-Enhanced Multi-Modal, Immersive, Relative Instruction Comprehension, Dhanuja Wanniarachchi, Archan Misra Mar 2025

Mimic: Ai And Ar-Enhanced Multi-Modal, Immersive, Relative Instruction Comprehension, Dhanuja Wanniarachchi, Archan Misra

Research Collection School Of Computing and Information Systems

We present a multimodal instruction comprehension framework, called MImIC, that utilizes visual sensing (including LIDAR and 2D RGB sensing) & AI spatial reasoning capabilities to support more seamless and immersive interaction between humans and AI-driven situated assistive agents. MImIC's key new capability is to support disambiguation of a wider set of relative spatial references that users naturally employ while issuing spatially-situated instructions. To support enhanced visual grounding via a combination of both fully-qualified and relative attribute references, MImIC uses (a) a fine-tuned transformer-based language translation DNN to accurately convert natural verbal commands into a structured set of machine understandable constraints …

Go to article

Explainable Neural Networks With Guarantee: A Sparse Estimation Approach, Antoine Ledent, Peng Liu Mar 2025

Explainable Neural Networks With Guarantee: A Sparse Estimation Approach, Antoine Ledent, Peng Liu

Research Collection School Of Computing and Information Systems

Balancing predictive power and interpretability has long been a challenging research area, particularly in powerful yet complex models like neural networks, where nonlinearity obstructs direct interpretation. This paper introduces a novel approach to constructing an explainable neural network that harmonizes predictiveness and explainability. Our model is designed as a linear combination of a sparse set of jointly learned features, each derived from a different trainable function applied to a single 1-dimensional input feature. Leveraging the ability to learn arbitrarily complex relationships, our neural network architecture enables automatic selection of a sparse set of important features, with the final prediction being …

Go to article

Divide-And-Conquer: Confluent Triple-Flow Network For Rgb-T Salient Object Detection, Hao Tang, Zechao Li, Dong Zhang, Shengfeng He, Jinhui Tang Mar 2025

Divide-And-Conquer: Confluent Triple-Flow Network For Rgb-T Salient Object Detection, Hao Tang, Zechao Li, Dong Zhang, Shengfeng He, Jinhui Tang

Research Collection School Of Computing and Information Systems

RGB-Thermal Salient Object Detection (RGB-T SOD) aims to pinpoint prominent objects within aligned pairs of visible and thermal infrared images. A key challenge lies in bridging the inherent disparities between RGB and Thermal modalities for effective saliency map prediction. Traditional encoder-decoder architectures, while designed for cross-modality feature interactions, may not have adequately considered the robustness against noise originating from defective modalities, thereby leading to suboptimal performance in complex scenarios. Inspired by hierarchical human visual systems, we propose the ConTriNet, a robust Confluent Triple-Flow Network employing a "Divide-and-Conquer"strategy. This framework utilizes a unified encoder with specialized decoders, each addressing different subtasks …

Go to article

Respear: Earable-Based Robust Respiratory Rate Monitoring, Yang Liu, Kayla-Jade Butkow, Jake Stuchbury-Wass, Adam Pullin, Dong Ma, Cecilia Masolo Mar 2025

Respear: Earable-Based Robust Respiratory Rate Monitoring, Yang Liu, Kayla-Jade Butkow, Jake Stuchbury-Wass, Adam Pullin, Dong Ma, Cecilia Masolo

Research Collection School Of Computing and Information Systems

Respiratory rate (RR) monitoring is integral to understanding physical and mental health and tracking fitness. Existing studies have demonstrated the feasibility of RR monitoring under specific user conditions (e.g., while remaining still, or while breathing heavily). Yet, performing accurate, continuous and non-obtrusive RR monitoring across diverse daily routines and activities remains challenging. In this work, we present RespEar, an earable-based system for robust RR monitoring. By leveraging the unique properties of in-ear microphones in earbuds, RespEar enables the use of Respiratory Sinus Arrhythmia (RSA) and Locomotor Respiratory Coupling (LRC), physiological couplings between cardiovascular activity, gait and respiration, to indirectly determine …

Go to article

Imageinthat: Manipulating Images To Convey User Instructions To Robots, Karthik Mahadevan, Blaine Lewis, Jiannan Li, Bilge Mutlu, Anthony Tang, Tovi Grossman Mar 2025

Imageinthat: Manipulating Images To Convey User Instructions To Robots, Karthik Mahadevan, Blaine Lewis, Jiannan Li, Bilge Mutlu, Anthony Tang, Tovi Grossman

Research Collection School Of Computing and Information Systems

Foundation models are rapidly improving the capability of robots in performing everyday tasks autonomously such as meal preparation, yet robots will still need to be instructed by humans due to model performance, the difficulty of capturing user preferences, and the need for user agency. Robots can be instructed using various methods---natural language conveys immediate instructions but can be abstract or ambiguous, whereas end-user programming supports longer-horizon tasks but interfaces face difficulties in capturing user intent. In this work, we propose using direct manipulation of images as an alternative paradigm to instruct robots, and introduce a specific instantiation called ImageInThat which …

Go to article

Forward-Secure Hierarchical Delegable Signature For Smart Homes, Jianfei Sun, Guowen Xu, Yang Yang, Xuehuan Yang, Xiaoguo Li, Cong Wu, Zhen Liu, Guomin Yang, Robert H. Deng Mar 2025

Forward-Secure Hierarchical Delegable Signature For Smart Homes, Jianfei Sun, Guowen Xu, Yang Yang, Xuehuan Yang, Xiaoguo Li, Cong Wu, Zhen Liu, Guomin Yang, Robert H. Deng

Research Collection School Of Computing and Information Systems

Aiming to provide people with great convenience and comfort, smart home systems have been deployed in thousands of homes. In this paper, we focus on handling the security and privacy issues in such a promising system by customizing a new cryptographic primitive to provide the following security guarantees: (1) fine-grained, privacy-preserving authorization for smart home users and integrity protection of communication contents; (2) flexible self-sovereign permission delegation; (3) forward security of previous messages. To our knowledge, no previous system has been designed to consider these three security and privacy requirements simultaneously. To tackle these challenges, we put forward the first-ever …

Go to article

Adaptive Deviation Learning For Visual Anomaly Detection With Data Contamination, Aanindya Sundar Das, Guansong Pang, Monowar Bhuyan Mar 2025

Adaptive Deviation Learning For Visual Anomaly Detection With Data Contamination, Aanindya Sundar Das, Guansong Pang, Monowar Bhuyan

Research Collection School Of Computing and Information Systems

Visual anomaly detection targets to detect images that notably differ from normal pattern, and it has found extensive application in identifying defective parts within the manufacturing industry. These anomaly detection paradigms predominantly focus on training detection models using only clean, unlabeled normal samples, assuming an absence of contamination; a condition often unmet in real-world scenarios. The performance of these methods significantly depends on the quality of the data and usually decreases when exposed to noise. We introduce a systematic adaptive method that employs deviation learning to compute anomaly scores end-to-end while addressing data contamination by assigning relative importance to the …

Go to article

Generalization Analysis For Deep Contrastive Representation Learning, Minh Hieu Nong, Antoine Ledent, Yunwen Lei, Cheng Yeaw Ku Mar 2025

Generalization Analysis For Deep Contrastive Representation Learning, Minh Hieu Nong, Antoine Ledent, Yunwen Lei, Cheng Yeaw Ku

Research Collection School Of Computing and Information Systems

In this paper, we present generalization bounds for the unsupervised risk in the Deep Contrastive Representation Learning framework, which employs deep neural networks as representation functions. We approach this problem from two angles. On the one hand, we derive a parameter-counting bound that scales with the overall size of the neural networks. On the other hand, we provide a norm-based bound that scales with the norms of neural networks’ weight matrices. Ignoring logarithmic factors, the bounds are independent of k, the size of the tuples provided for contrastive learning. To the best of our knowledge, this property is only shared …

Go to article

Explainable Neural Networks With Guarantees: A Sparse Estimation Approach, Antoine Ledent, Peng Liu Mar 2025

Explainable Neural Networks With Guarantees: A Sparse Estimation Approach, Antoine Ledent, Peng Liu

Research Collection School Of Computing and Information Systems

Balancing predictive power and interpretability has long been a challenging research area, particularly in powerful yet complex models like neural networks, where nonlinearity obstructs direct interpretation. This paper introduces a novel approach to constructing an explainable neural network that harmonizes predictiveness and explainability. Our model is designed as a linear combination of a sparse set of jointly learned features, each derived from a different trainable function applied to a single 1-dimensional input feature. Leveraging the ability to learn arbitrarily complex relationships, our neural network architecture enables automatic selection of a sparse set of important features, with the final prediction being …

Go to article

Enhancing Item‑Level Bundle Representation For Bundle Recommendation, Xiaoyu Du, Kun Qian, Yunshan Ma, Xinguang Xiang Mar 2025

Enhancing Item‑Level Bundle Representation For Bundle Recommendation, Xiaoyu Du, Kun Qian, Yunshan Ma, Xinguang Xiang

Research Collection School Of Computing and Information Systems

Bundle recommendation approaches offer users a set of related items on a particular topic. The current state-of-the-art (SOTA) method utilizes contrastive learning to learn representations at both the bundle and item levels. However, due to the inherent difference between the bundle-level and item-level preferences, the item-level representations may not receive sufficient information from the bundle affiliations to make accurate predictions. In this article, we propose a novel approach, Enhanced Bundle Recommendation (EBRec), which incorporates two enhanced modules to explore inherent item-level bundle representations. First, we propose to incorporate the bundle-user-item (B-U-I) high-order correlations to explore more collaborative information, thus to …

Go to article

Attackg+: Boosting Attack Graph Construction With Large Language Models, Yongheng Zhang, Tingwen Du, Yunshan Ma, Xiang Wang, Yi Xie, Guozheng Yang, Yuliang Lu, Ee‑Chien Chang Mar 2025

Attackg+: Boosting Attack Graph Construction With Large Language Models, Yongheng Zhang, Tingwen Du, Yunshan Ma, Xiang Wang, Yi Xie, Guozheng Yang, Yuliang Lu, Ee‑Chien Chang

Research Collection School Of Computing and Information Systems

Attack graph construction seeks to convert textual cyber threat intelligence (CTI) reports into structuredrepresentations, portraying the evolutionary traces of cyber attacks. Even though previous research hasproposed various methods to construct attack graphs, they generally suffer from limited generalizationcapability to diverse knowledge types as well as requirement of expertise in model design and tuning.Addressing these limitations, we seek to utilize Large Language Models (LLMs), which have achieved enormoussuccess in a broad range of tasks given exceptional capabilities in both language understanding and zeroshot task fulfillment. Thus, we propose a fully automatic LLM-based framework to construct attack graphsnamed: AttacKG+. Our framework consists …

Go to article

Enhanced Sample Selection With Confidence Tracking: Identifying Correctly Labeled Yet Hard-To-Learn Samples In Noisy Data, Weiran Pan, Wei Wei, Feida Zhu, Yong Deng Mar 2025

Enhanced Sample Selection With Confidence Tracking: Identifying Correctly Labeled Yet Hard-To-Learn Samples In Noisy Data, Weiran Pan, Wei Wei, Feida Zhu, Yong Deng

Research Collection School Of Computing and Information Systems

We propose a novel sample selection method for image classification in the presence of noisy labels. Existing methods typically consider small-loss samples as correctly labeled. However, some correctly labeled samples are inherently difficult for the model to learn and can exhibit high loss similar to mislabeled samples in the early stages of training. Consequently, setting a threshold on per-sample loss to select correct labels results in a trade-off between precision and recall in sample selection: a lower threshold may miss many correctly labeled hard-to-learn samples (low recall), while a higher threshold may include many mislabeled samples (low precision). To address …

Go to article

Revisiting Sentiment Analysis For Software Engineering In The Era Of Large Language Models, Ting Zhang, Ivana Clairine Irsan, Thung Ferdian, David Lo Mar 2025

Revisiting Sentiment Analysis For Software Engineering In The Era Of Large Language Models, Ting Zhang, Ivana Clairine Irsan, Thung Ferdian, David Lo

Research Collection School Of Computing and Information Systems

Software development involves collaborative interactions where stakeholders express opinions across various platforms. Recognizing the sentiments conveyed in these interactions is crucial for the effective development and ongoing maintenance of software systems. For software products, analyzing the sentiment of user feedback, e.g., reviews, comments, and forum posts can provide valuable insights into user satisfaction and areas for improvement. This can guide the development of future updates and features. However, accurately identifying sentiments in software engineering datasets remains challenging.This study investigates bigger large language models (bLLMs) in addressing the labeled data shortage that hampers fine-tuned smaller large language models (sLLMs) in software …

Go to article

Multi-Uav Reconnaissance Mission Planning Via Deep Reinforcement Learning With Simulated Annealing, Mingfeng Fan, Huan Liu, Guohua Wu, Aldy Gunawan, Guillaume Sartoretti Mar 2025

Multi-Uav Reconnaissance Mission Planning Via Deep Reinforcement Learning With Simulated Annealing, Mingfeng Fan, Huan Liu, Guohua Wu, Aldy Gunawan, Guillaume Sartoretti

Research Collection School Of Computing and Information Systems

Unmanned aerial vehicles (UAVs) are widely used in reconnaissance missions due to their autonomy and flexibility. Efficient mission planning for multiple UAVs is crucial for tasks such as traffic monitoring and data collection. However, existing approaches to multi-UAV reconnaissance mission planning problem (MURMPP) often struggle with high computational demands, leading to suboptimal solutions. To overcome this challenge, we introduce a divide-and-conquer framework that splits the problem into two phases: target allocation and UAV routing, effectively reducing computational complexity. Specifically, we propose a hybrid method, SA-NNO-DRL, which combines the nearest neighbor optima-based deep reinforcement learning (NNO-DRL) approach with simulated annealing (SA). …

Go to article

Multisfl: Towards Accurate Split Federated Learning Via Multi-Model Aggregation And Knowledge Replay, Zeke Xia, Ming Hu, Dengke Yan, Ruixuan Liu, Anran Li, Xiaofei Xie, Mingsong Chen Mar 2025

Multisfl: Towards Accurate Split Federated Learning Via Multi-Model Aggregation And Knowledge Replay, Zeke Xia, Ming Hu, Dengke Yan, Ruixuan Liu, Anran Li, Xiaofei Xie, Mingsong Chen

Research Collection School Of Computing and Information Systems

Although Split Federated Learning (SFL) effectively enables knowledge sharing among resource-constrained clients, it suffers from low training performance due to the neglect of data heterogeneity and catastrophic forgetting problems. To address these issues, we propose a novel SFL approach named MultiSFL, which adopts i) an effective multimodel aggregation mechanism to alleviate gradient divergence caused by heterogeneous data and ii) a novel knowledge replay strategy to deal with the catastrophic forgetting problem. MultiSFL adopts two servers (i.e., the fed server and main server) to maintain multiple branch models for local training and an aggregated master model for knowledge sharing among branch …

Go to article

Understanding Individual Agent Importance In Multi-Agent System Via Counterfactual Reasoning, Jianming Chen, Yawen Wang, Junjie Wang, Xiaofei Xie, Jun Hu, Qing Wang, Fanjiang Xu Mar 2025

Understanding Individual Agent Importance In Multi-Agent System Via Counterfactual Reasoning, Jianming Chen, Yawen Wang, Junjie Wang, Xiaofei Xie, Jun Hu, Qing Wang, Fanjiang Xu

Research Collection School Of Computing and Information Systems

Explaining multi-agent systems (MAS) is urgent as these systems become increasingly prevalent in various applications. Previous work has provided explanations for the actions or states of agents, yet falls short in understanding the black-boxed agent's importance within a MAS and the overall team strategy. To bridge this gap, we propose EMAI, a novel agent-level explanation approach that evaluates the individual agent's importance. Inspired by counterfactual reasoning, a larger change in reward caused by the randomized action of agent indicates its higher importance. We model it as a MARL problem to capture interactions across agents. Utilizing counterfactual reasoning, EMAI learns the …

Go to article

Ragg: Retrieval-Augmented Grasp Generation Model, Zhenhua Tang, Bin Zhu, Yanbin Hao, Chong-Wah Ngo, Richang Hong Mar 2025

Ragg: Retrieval-Augmented Grasp Generation Model, Zhenhua Tang, Bin Zhu, Yanbin Hao, Chong-Wah Ngo, Richang Hong

Research Collection School Of Computing and Information Systems

Intent-based grasp generation inherently involves challenges such as manipulation ambiguity and modality gaps. To address these, we propose a novel Retrieval-Augmented Grasp Generation model (RAGG). Our key insight is that when humans manipulate new objects, they initially mimic the interaction patterns observed in similar objects, then progressively adjust hand-object contact. Consequently, we develop RAGG as a two-stage approach, encompassing retrieval-guided generation and structurally stable grasp refinement. In the first stage, we propose a Retrieval-Augmented Diffusion Model (ReDim), which identifies the most relevant interaction instance from a knowledge base to explicitly guide grasp generation, thereby mitigating ambiguity and bridging modality gaps …

Go to article

Aligning Large Language Models For Faithful Integrity Against Opposing Argument, Yong Zhao, Yang Deng, See-Kiong Ng, Tat-Seng Chua Mar 2025

Aligning Large Language Models For Faithful Integrity Against Opposing Argument, Yong Zhao, Yang Deng, See-Kiong Ng, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Large Language Models (LLMs) have demonstrated impressive capabilities in complex reasoning tasks. However, they can be easily misled by unfaithful arguments during conversations, even when their original statements are correct. To this end, we investigate the problem of maintaining faithful integrity in LLMs. This involves ensuring that LLMs adhere to their faithful statements in the face of opposing arguments and are able to correct their incorrect statements when presented with faithful arguments. In this work, we propose a novel framework, named Alignment for Faithful Integrity with Confidence Estimation (AFICE), which aims to align the LLM responses with faithful integrity. Specifically, …

Go to article

Dualopt: A Dual Divide-And-Optimize Algorithm For The Large-Scale Traveling Salesman Problem, Shipei Zhou, Yuandong Ding, Chi Zhang, Zhiguang Cao, Yan Jin Mar 2025

Dualopt: A Dual Divide-And-Optimize Algorithm For The Large-Scale Traveling Salesman Problem, Shipei Zhou, Yuandong Ding, Chi Zhang, Zhiguang Cao, Yan Jin

Research Collection School Of Computing and Information Systems

This paper proposes a dual divide-and-optimize algorithm (DualOpt) for solving the large-scale traveling salesman problem (TSP). DualOpt combines two complementary strategies to improve both solution quality and computational efficiency. The first strategy is a grid-based divide-and-conquer procedure that partitions the TSP into smaller subproblems, solving them in parallel and iteratively refining the solution by merging nodes and partial routes. The process continues until only one grid remains, yielding a high-quality initial solution. The second strategy involves a path-based divide-and-optimize procedure that further optimizes the solution by dividing it into sub-paths, optimizing each using a neural solver, and merging them back …

Go to article

Leveraging Constraint Violation Signals For Action Constrained Reinforcement Learning, Janaka Chathuranga Brahmanage, Jiajing Ling, Akshat Kumar Mar 2025

Leveraging Constraint Violation Signals For Action Constrained Reinforcement Learning, Janaka Chathuranga Brahmanage, Jiajing Ling, Akshat Kumar

Research Collection School Of Computing and Information Systems

In many RL applications, ensuring an agent’s actions adhere to constraints is crucial for safety. Most previous methods in Action-Constrained Reinforcement Learning (ACRL) employ a projection layer after the policy network to correct the action. However projection-based methods suffer from issues like the zero gradient problem and higher runtime due to the usage of optimization solvers. Recently methods were proposed to train generative models to learn a differentiable mapping between latent variables and feasible actions to address this issue. However, generative models require training using samples from the constrained action space, which itself is challenging. To address such limitations, first, …

Go to article

Offline Safe Reinforcement Learning Using Trajectory Classification, Ze Gong, Akshat Kumar, Pradeep Varakantham Mar 2025

Offline Safe Reinforcement Learning Using Trajectory Classification, Ze Gong, Akshat Kumar, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

Offline safe reinforcement learning (RL) has emerged as a promising approach for learning safe behaviors without engaging in risky online interactions with the environment. Most existing methods in offline safe RL rely on cost constraints at each time step (derived from global cost constraints) and this can result in either overly conservative policies or violation of safety constraints. In this paper, we propose to learn a policy that generates desirable trajectories and avoids undesirable trajectories. To be specific, we first partition the pre-collected dataset of state-action trajectories into desirable and undesirable subsets. Intuitively, the desirable set contains high reward and …

Go to article

Occlusion-Insensitive Talking Head Video Generation Via Facelet Compensation, Yuhui Deng, Yuqin Lu, Yangyang Xu, Yongwei Nie, Shengfeng He Mar 2025

Occlusion-Insensitive Talking Head Video Generation Via Facelet Compensation, Yuhui Deng, Yuqin Lu, Yangyang Xu, Yongwei Nie, Shengfeng He

Research Collection School Of Computing and Information Systems

Talking head video generation involves animating a still face image using facial motion cues derived from a driving video to replicate target poses and expressions. Traditional methods often rely on the assumption that the relative positions of facial keypoints remain unchanged. However, this assumption fails when keypoints are occluded or when the head is in a profile pose, leading to inconsistencies in identity and blurring in certain facial regions. In this paper, we introduce Occlusion-Insensitive Talking Head Video Generation, a novel approach that eliminates the reliance on spatial correlation of keypoints and instead leverages semantic correlation. Our method transforms facial …

Go to article

Personamagic: Stage-Regulated High-Fidelity Face Customization With Tandem Equilibrium, Xinzhe Li, Jiahui Zhan, Shengfeng He, Yangyang Xu, Junyu Dong, Huaidong Zhang, Yong Du Mar 2025

Personamagic: Stage-Regulated High-Fidelity Face Customization With Tandem Equilibrium, Xinzhe Li, Jiahui Zhan, Shengfeng He, Yangyang Xu, Junyu Dong, Huaidong Zhang, Yong Du

Research Collection School Of Computing and Information Systems

Personalized image generation has made significant strides in adapting content to novel concepts. However, a persistent challenge remains: balancing the accurate reconstruction of unseen concepts with the need for editability according to the prompt, especially when dealing with the complex nuances of facial features. In this study, we delve into the temporal dynamics of the text-to-image conditioning process, emphasizing the crucial role of stage partitioning in introducing new concepts. We present PersonaMagic, a stage-regulated generative technique designed for high-fidelity face customization. Using a simple MLP network, our method learns a series of embeddings within a specific timestep interval to capture …

Go to article

Adversarial Attacks On Event-Based Pedestrian Detectors: A Physical Approach, Guixu Lin, Muyao Niu, Qingtian Zhu, Zhengwei Yin, Zhuoxiao Li, Shengfeng He, Yinqiang Zheng Mar 2025

Adversarial Attacks On Event-Based Pedestrian Detectors: A Physical Approach, Guixu Lin, Muyao Niu, Qingtian Zhu, Zhengwei Yin, Zhuoxiao Li, Shengfeng He, Yinqiang Zheng

Research Collection School Of Computing and Information Systems

Event cameras, known for their low latency and high dynamic range, show great potential in pedestrian detection applications. However, while recent research has primarily focused on improving detection accuracy, the robustness of event-based visual models against physical adversarial attacks has received limited attention. For example, adversarial physical objects, such as specific clothing patterns or accessories, can exploit inherent vulnerabilities in these systems, leading to misdetections or misclassifications. This study is the first to explore physical adversarial attacks on event-driven pedestrian detectors, specifically investigating whether certain clothing patterns worn by pedestrians can cause these detectors to fail, effectively rendering them unable …

Go to article

An Aspect Performance-Aware Hypergraph Neural Network For Review-Based Recommendation, Junrui Liu, Tong Li, Di Wu, Zifang Tang, Yuan Fang, Zhen Yang Mar 2025

An Aspect Performance-Aware Hypergraph Neural Network For Review-Based Recommendation, Junrui Liu, Tong Li, Di Wu, Zifang Tang, Yuan Fang, Zhen Yang

Research Collection School Of Computing and Information Systems

Online reviews allow consumers to provide detailed feedback on various aspects of items. Existing methods utilize these aspects to model users' fine-grained preferences for specific item features through graph neural networks. We argue that the performance of items on different aspects is important for making precise recommendations, which has not been taken into account by existing approaches, due to lack of data. In this paper, we propose an aspect performance-aware hypergraph neural network (APH) for the review-based recommendation, which learns the performance of items from the conflicting sentiment polarity of user reviews. Specifically, APH comprehensively models the relationships among users, …

Go to article

Marginal Benefit Driven Rl Teacher For Unsupervised Environment Design, Dexun Li, Wenjun Li, Pradeep Varakantham Mar 2025

Marginal Benefit Driven Rl Teacher For Unsupervised Environment Design, Dexun Li, Wenjun Li, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

Training generally capable agents in complex environments is a challenging task that involves identifying the “right” environments at the training stage. Recent research has highlighted the potential of the Unsupervised Environment Design framework, which generates environment instances/levels adaptively at the frontier of the agent’s capabilities using regret measures. While regret approaches have shown promise in generating feasible environments, they can produce difficult environments that are challenging for an RL agent to learn from. This is because regret represents the best-case (upper bound) learning potential and not the actual learning potential of an environment. To address this, we propose an alternative …

Go to article

First Prev Next Last