Open Access. Powered by Scholars. Published by Universities.®
- Discipline
-
- Physical Sciences and Mathematics (4208)
- Computer Sciences (4205)
- Databases and Information Systems (1558)
- Software Engineering (1251)
- Artificial Intelligence and Robotics (743)
-
- Information Security (727)
- Graphics and Human Computer Interfaces (489)
- Numerical Analysis and Scientific Computing (454)
- Engineering (344)
- Programming Languages and Compilers (269)
- Computer Engineering (231)
- Social and Behavioral Sciences (213)
- Theory and Algorithms (212)
- Business (202)
- OS and Networks (148)
- Communication (103)
- Operations Research, Systems Engineering and Industrial Engineering (101)
- Social Media (83)
- Data Storage Systems (80)
- Medicine and Health Sciences (79)
- Education (65)
- Public Affairs, Public Policy and Public Administration (54)
- Computer and Systems Architecture (52)
- Health Information Technology (51)
- Transportation (50)
- Management Information Systems (49)
- Digital Communications and Networking (47)
- E-Commerce (44)
- Finance and Financial Management (40)
- Higher Education (30)
- Keyword
-
- Deep learning (71)
- Machine learning (59)
- Privacy (41)
- Empirical study (40)
- Deep Learning (35)
-
- Training (34)
- Security (33)
- Task analysis (32)
- Cloud computing (31)
- Data mining (31)
- Artificial intelligence (29)
- Reinforcement learning (29)
- Social media (29)
- Software engineering (29)
- Neural networks (28)
- Android (27)
- Blockchain (27)
- Feature extraction (27)
- Model Check (26)
- Access control (25)
- Large Language Models (25)
- Natural language processing (25)
- Authentication (24)
- Privacy-preserving (24)
- Servers (24)
- Anomaly detection (23)
- Information retrieval (23)
- Large language models (23)
- Visualization (23)
- Classification (22)
- Publication Year
Articles 361 - 390 of 4258
Full-Text Articles in Entire DC Network
Intention Is All You Need: Refining Your Code From Your Intention, Qi Guo, Xiaofei Xie, Shangqing Liu, Ming Hu, Xiaohong Li, Lei Bu
Intention Is All You Need: Refining Your Code From Your Intention, Qi Guo, Xiaofei Xie, Shangqing Liu, Ming Hu, Xiaohong Li, Lei Bu
Research Collection School Of Computing and Information Systems
Code refinement aims to enhance existing code by addressing issues, refactoring, and optimizing to improve quality and meet specific requirements. As software projects scale in size and complexity, the traditional iterative exchange between reviewers and developers becomes increasingly burdensome. While recent deep learning techniques have been explored to accelerate this process, their performance remains limited, primarily due to challenges in accurately understanding reviewers’ intents. This paper proposes an intention-based code refinement technique that enhances the conventional comment-to-code process by explicitly extracting reviewer intentions from the comments. Our approach consists of two key phases: Intention Extraction and Intention Guided Revision Generation. …
Scenario-Driven And Context-Aware Automated Accessibility Testing For Android Apps, Yuxin Zhang, Sen Chen, Xiaofei Xie, Zibo Liu, Lingling Fan
Scenario-Driven And Context-Aware Automated Accessibility Testing For Android Apps, Yuxin Zhang, Sen Chen, Xiaofei Xie, Zibo Liu, Lingling Fan
Research Collection School Of Computing and Information Systems
Mobile accessibility is increasingly important nowadays as it enables people with disabilities to use mobile applications to perform daily tasks. Ensuring mobile accessibility not only benefits those with disabilities but also enhances the user experience for all users, making applications more intuitive and user-friendly. Although numerous tools are available for testing and detecting accessibility issues in Android applications, a large number of false negatives and false positives persist due to limitations in the existing approaches, i.e., low coverage of UI scenarios and lack of consideration of runtime context. To address these problems, in this paper, we propose a scenario-driven exploration …
Acccred: Improved Accountable Anonymous Credentials With Dynamic Triple-Hiding Committees, Sijiang Xie, Rui Shi, Yang Yang, Huiqin Xie, Yingjiu Li, Robert H. Deng
Acccred: Improved Accountable Anonymous Credentials With Dynamic Triple-Hiding Committees, Sijiang Xie, Rui Shi, Yang Yang, Huiqin Xie, Yingjiu Li, Robert H. Deng
Research Collection School Of Computing and Information Systems
Accountable anonymous credentials protect user privacy while holding the accountability of ill-intentioned individuals, which is a critical feature for applications such as online payments and other financial services. Existing accountable anonymous credentials rely on a public committee of trustworthy members who are assumed not to collude and are well protected to perform privacy revocation. However, this assumption is unsound in blockchain-based cryptocurrency systems because the selected committees may involve nodes with significant stakes, and public nodes serving as committee members are vulnerable against targeted attacks from high-computing power adversaries. In this paper, we propose an improved accountable anonymous credential called …
Towards Multimodal Empathetic Response Generation: A Rich Text-Speech-Vision Avatar-Based Benchmark, Han Zhang, Zixiang Meng, Meng Luo, Hong Han, Lizi Liao, Erik Cambria, Hao Fei
Towards Multimodal Empathetic Response Generation: A Rich Text-Speech-Vision Avatar-Based Benchmark, Han Zhang, Zixiang Meng, Meng Luo, Hong Han, Lizi Liao, Erik Cambria, Hao Fei
Research Collection School Of Computing and Information Systems
Empathetic Response Generation (ERG) is one of the key tasks of the affective computing area, which aims to produce emotionally nuanced and compassionate responses to user's queries. However, existing ERG research is predominantly confined to the singleton text modality, limiting its effectiveness since human emotions are inherently conveyed through multiple modalities. To combat this, we introduce an avatar-based Multimodal ERG (MERG) task, entailing rich text, speech, and facial vision information. We first present a large-scale high-quality benchmark dataset, AvaMERG, which extends traditional text ERG by incorporating authentic human speech audio and dynamic talking-face avatar videos, encompassing a diverse range of …
Seaexam And Seabench: Benchmarking Llms With Local Multilingual Questions In Southeast Asia, Chaoqun Liu, Wenxuan Zhang, Jiahao Ying, Mahani Aljunied, Anh Tuan Luu, Lidong Bing
Seaexam And Seabench: Benchmarking Llms With Local Multilingual Questions In Southeast Asia, Chaoqun Liu, Wenxuan Zhang, Jiahao Ying, Mahani Aljunied, Anh Tuan Luu, Lidong Bing
Research Collection School Of Computing and Information Systems
This study introduces two novel benchmarks, SeaExam and SeaBench, designed to evalu ate the capabilities of Large Language Models (LLMs) in Southeast Asian (SEA) application scenarios. Unlike existing multilingual datasets primarily derived from English translations, these benchmarks are constructed based on real world scenarios from SEA regions. SeaExam draws from regional educational exams to form a comprehensive dataset that encompasses sub jects such as local history and literature. In contrast, SeaBench is crafted around multi turn, open-ended tasks that reflect daily inter actions within SEA communities. Our evalua tions demonstrate that SeaExam and SeaBench more effectively discern LLM performance on …
Augsso: Secure Threshold Single-Sign-On Authentication With Popular Password Collection, Changsong Jiang, Chunxiang Xu, Guomin Yang
Augsso: Secure Threshold Single-Sign-On Authentication With Popular Password Collection, Changsong Jiang, Chunxiang Xu, Guomin Yang
Research Collection School Of Computing and Information Systems
Single-sign-on authentication is widely deployed in mobile systems, which allows an identity server to authenticate a mobile user and issue her/him with a token, such that the user can access diverse mobile services. To address the single-point-offailure problem, threshold single-sign-on authentication (PbTA) is a feasible solution, where multiple identity servers perform user authentication and token issuance in a threshold way. However, existing PbTA schemes confront critical drawbacks. Specifically, these schemes are vulnerable to perpetual secret leakage attacks (PSLA): an adversary perpetually compromises secrets of identity servers (e.g., secret key shares or credentials) to break security. Besides, they fail to achieve …
Dissecting Global Search: A Simple Yet Effective Method To Boost Individual Discrimination Testing And Repair, Lili Quan, Tianlin Li, Xiaofei Xie, Zhenpeng Chen, Sen Chen, Lingxiao Jiang, Xiaohong Li
Dissecting Global Search: A Simple Yet Effective Method To Boost Individual Discrimination Testing And Repair, Lili Quan, Tianlin Li, Xiaofei Xie, Zhenpeng Chen, Sen Chen, Lingxiao Jiang, Xiaohong Li
Research Collection School Of Computing and Information Systems
Deep Learning (DL) has achieved significant success in socially critical decision-making applications but often exhibits unfair behaviors, raising social concerns. Among these unfair behaviors, individual discrimination-examining inequalities between instance pairs with identical profiles differing only in sensitive attributes such as gender, race, and age-is extremely socially impactful. Existing methods have made significant and commendable efforts in testing individual discrimination before deployment. However, their efficiency and effectiveness remain limited, particularly when evaluating relatively fairer models. It remains unclear which phase of the existing testing framework (global or local) is the primary bottleneck limiting performance. Facing the above issues, we first identify …
Robust Threshold Ecdsa With Online-Friendly Design In Three Rounds, Guofeng Tang, Haiyang Xue
Robust Threshold Ecdsa With Online-Friendly Design In Three Rounds, Guofeng Tang, Haiyang Xue
Research Collection School Of Computing and Information Systems
Threshold signatures, especially ECDSA, enhance key protection by addressing the single-point-of-failure issue. Threshold signing can be divided into offline and online phases, based on whether the message is required. Schemes with low-cost online phases are referred to as “online-friendly”. Another critical aspect of threshold ECDSA for real-world applications is robustness, which guarantees the successful completion of each signing execution whenever a threshold number t of semi-honest participants is met, even in the presence of misbehaving signatories. The state-of-the-art online-friendly threshold ECDSA with-out robustness was developed by Doerner et al. in S&P'24, requiring only three rounds. Recent work by Wong et …
Frame-Voyager: Learning To Query Frames For Video Large Language Models, Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang, Jiawei Wu, Hao Zhang, Qianru Sun
Frame-Voyager: Learning To Query Frames For Video Large Language Models, Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang, Jiawei Wu, Hao Zhang, Qianru Sun
Research Collection School Of Computing and Information Systems
Video Large Language Models (Video-LLMs) have made remarkable progress in video understanding tasks. However, they are constrained by the maximum length of input tokens, making it impractical to input entire videos. Existing frame selection approaches, such as uniform frame sampling and text-frame retrieval, fail to account for the information density variations in the videos or the complex instructions in the tasks, leading to sub-optimal performance. In this paper, we propose Frame-Voyager that learns to query informative frame combinations, based on the given textual queries in the task. To train Frame-Voyager, we introduce a new data collection and labeling pipeline, by …
Does Chatgpt-Permitted Assessments Help Students Generate Better Answers And Learn More?, Michelle L. F. Cheong, Yun-Chen Chen
Does Chatgpt-Permitted Assessments Help Students Generate Better Answers And Learn More?, Michelle L. F. Cheong, Yun-Chen Chen
Research Collection School Of Computing and Information Systems
We discuss our methodology and implementation of ChatGPT-permitted assessments for a university-level spreadsheets modelling module. Through our quantitative data analysis, our students rated ChatGPT’s answers to be incorrect on average and thus will not help them generate better answers directly, representing low “Perceived usefulness” (PU), while they rated ChatGPT 3.5 with relatively high “Perceived ease of use” (PE). They gave a good “Behavioural intention” (BI) rating indicating that they were motivated to use it in future as they could still learn more about this module by using ChatGPT 3.5. We found that both PU and PE affected BI positively, with …
Verification Of Bit-Flip Attacks Against Quantized Neural Networks, Yedi Zhang, Lei Huang, Pengfei Gao, Fu Song, Jun Sun, Jin Song Dong
Verification Of Bit-Flip Attacks Against Quantized Neural Networks, Yedi Zhang, Lei Huang, Pengfei Gao, Fu Song, Jun Sun, Jin Song Dong
Research Collection School Of Computing and Information Systems
In the rapidly evolving landscape of neural network security, the resilience of neural networks against bit-flip attacks (i.e., an attacker maliciously flips an extremely small amount of bits within its parameter storage memory system to induce harmful behavior), has emerged as a relevant area of research. Existing studies suggest that quantization may serve as a viable defense against such attacks. Recognizing the documented susceptibility of real-valued neural networks to such attacks and the comparative robustness of quantized neural networks (QNNs), in this work, we introduce BFAVerifier, the first verification framework designed to formally verify the absence of bit-flip attacks against …
Ada-Gen: Iterative And Incremental Generation Of Full-Stack Apps For Learning Agile/Devops Software Development Practices, Nguyen Binh Duong Ta
Ada-Gen: Iterative And Incremental Generation Of Full-Stack Apps For Learning Agile/Devops Software Development Practices, Nguyen Binh Duong Ta
Research Collection School Of Computing and Information Systems
To learn Agile/DevOps practices effectively, students need to apply them in an actual software development project. This is challenging if students are mostly from non-computing backgrounds and they do not have time in the curriculum to learn programming and related tools. Therefore, it is important to help students who do not possess programming foundations to develop fully functional software during the process of learning Agile/DevOps concepts. We noted that existing low-code/no-code app development platforms have not been designed to teach Agile/DevOps practices. On the other hand, recent AI-based tools for code generation such as GitHub Copilot have been built mainly …
Ivyapc: Auditable Generalized Payment Channels, Ming Li, Yuxian Li, Jian Weng, Yingjiu Li, Jiasi Weng, Junzuo Lai, Robert H. Deng
Ivyapc: Auditable Generalized Payment Channels, Ming Li, Yuxian Li, Jian Weng, Yingjiu Li, Jiasi Weng, Junzuo Lai, Robert H. Deng
Research Collection School Of Computing and Information Systems
Payment channels are a cornerstone of a scalable blockchain infrastructure that enables transacting parties to lock assets on the blockchain and perform rapid off-chain updates with minimal latency and overhead. These protocols dramatically reduce on-chain interaction and improve throughput, with blockchain consensus only invoked in the event of disputes or final closure. While widely adopted in single-chain settings—such as in the Lightning Network for Bitcoin—existing constructions have several limitations, in particular they suffer from at least one of the following limitations: 1. No cross-chain. They do not enable fast trading of assets that reside on multiple isolated blockchains. 2. Non-optimal …
Dual Operation Aggregation Graph Neural Networks For Solving Flexible Job-Shop Scheduling Problem With Reinforcement Learning, Peng Zhao, You Zhou, Di Wang, Zhiguang Cao, Yubin Xiao, Xuan Wu, Yuanshu Li, Hongjia Liu, Wei Du, Yuan Jiang, Liupu Wang
Dual Operation Aggregation Graph Neural Networks For Solving Flexible Job-Shop Scheduling Problem With Reinforcement Learning, Peng Zhao, You Zhou, Di Wang, Zhiguang Cao, Yubin Xiao, Xuan Wu, Yuanshu Li, Hongjia Liu, Wei Du, Yuan Jiang, Liupu Wang
Research Collection School Of Computing and Information Systems
With the widespread adoption of Internet Protocol (IP) communication technology and web-based platforms, cloud manufacturing has become a significant hallmark of Industry 4.0. Integrating graph algorithms into these web-enabled environments is crucial as they facilitate the representation and analysis of complex relationships in manufacturing processes, enabling efficient decision-making and adaptability in dynamic environments. As a key scheduling problem in cloud manufacturing, the flexible job-shop scheduling problem (FJSP) finds extensive applications in real-world scenarios. However, traditional FJSP-solving methods struggle to meet the efficiency and adaptability demands of cloud manufacturing due to generalization issues and excessive computational time, while reinforcement learning-based methods …
Alphaedit: Null-Space Constrained Knowledge Editing For Language Models, Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, Tat‑Seng Chua
Alphaedit: Null-Space Constrained Knowledge Editing For Language Models, Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, Tat‑Seng Chua
Research Collection School Of Computing and Information Systems
Large language models (LLMs) often exhibit hallucinations, producing incorrector outdated knowledge. Hence, model editing methods have emerged to enabletargeted knowledge updates. To achieve this, a prevailing paradigm is the locatingthen-editing approach, which first locates influential parameters and then edits themby introducing a perturbation. While effective, current studies have demonstrated that this perturbation inevitably disrupt the originally preserved knowledge within LLMs, especially in sequential editing scenarios. To address this, we introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters. We theoretically prove that this projection ensures the output …
Scuzer: A Scheduling Optimization Fuzzer For Tvm, Xiangxiang Chen, Xingwei Lin, Jingyi Wang, Jun Sun, Jiashui Wang, Wenhai Wang
Scuzer: A Scheduling Optimization Fuzzer For Tvm, Xiangxiang Chen, Xingwei Lin, Jingyi Wang, Jun Sun, Jiashui Wang, Wenhai Wang
Research Collection School Of Computing and Information Systems
The concept of Deep Learning (DL) compiler was proposed to deploy DL models more efficiently on diverse hardware through optimization techniques. As one of the most popular DL compilers, TVM incorporates three levels (high-level, schedule, and low-level) of optimizations, which can inadvertently introduce code logic bugs and build failure bugs. Among these optimizations, scheduling optimization is the core component of DL compilers, which ensures the acceleration of models on all devices. However, the existing works only focus on the testing of high-level and low-level optimizations in TVM, fail to take the most important and challenging intermediate scheduling optimization layer into …
Enhancing Llm-Based Coding Tools Through Native Integration Of Ide-Derived Static Context, Yichen Li, Yun Peng, Yintong Huo, R. Michael Lyu
Enhancing Llm-Based Coding Tools Through Native Integration Of Ide-Derived Static Context, Yichen Li, Yun Peng, Yintong Huo, R. Michael Lyu
Research Collection School Of Computing and Information Systems
Large Language Models (LLMs) have achieved remarkable success in code completion, as evidenced by their essential roles in developing code assistant services such as Copilot. Being trained on in-file contexts, current LLMs are quite effective in completing code for single source files. However, it is challenging for them to conduct repository-level code completion for large software projects that require cross-file information. Existing research on LLM-based repository-level code completion identifies and integrates cross-file contexts, but it suffers from low accuracy and limited context length of LLMs. In this paper, we argue that Integrated Development Environments (IDEs) can provide direct, accurate and …
Semantic Loss-Guided Data-Efficient Supervised Fine-Tuning For Safe Responses In Llms, Yuxiao Lu, Pradeep Varakantham, Pradeep Varakantham
Semantic Loss-Guided Data-Efficient Supervised Fine-Tuning For Safe Responses In Llms, Yuxiao Lu, Pradeep Varakantham, Pradeep Varakantham
Research Collection School Of Computing and Information Systems
Large Language Models (LLMs) generating unsafe responses to toxic prompts is a significant issue in their applications. While various efforts aim to address this safety concern, previous approaches often demand substantial human data collection or rely on the less dependable option of using another LLM to generate corrective data. In this paper, we aim to take this problem and overcome limitations of requiring significant high-quality human data. Our method requires only a small set of unsafe responses to toxic prompts, easily obtained from the unsafe LLM itself. By employing a semantic cost combined with a negative Earth Mover Distance (EMD) …
Democratic Training Against Universal Adversarial Perturbations, Bing Sun, Jun Sun, Wei Zhao
Democratic Training Against Universal Adversarial Perturbations, Bing Sun, Jun Sun, Wei Zhao
Research Collection School Of Computing and Information Systems
Despite their advances and success, real-world deep neural networks are known to be vulnerable to adversarial attacks. Universal adversarial perturbation, an inputagnostic attack, poses a serious threat for them to be deployed in security-sensitive systems. In this case, a single universal adversarial perturbation deceives the model on a range of clean inputs without requiring input-specific optimization, which makes it particularly threatening. In this work, we observe that universal adversarial perturbations usually lead to abnormal entropy spectrum in hidden layers, which suggests that the prediction is dominated by a small number of “feature” in such cases (rather than democratically by many …
Can Llms Replace Manual Annotation Of Software Engineering Artifacts?, Toufique Ahmed, Premkumar Devanbu, Christoph Treude, Michael Pradel
Can Llms Replace Manual Annotation Of Software Engineering Artifacts?, Toufique Ahmed, Premkumar Devanbu, Christoph Treude, Michael Pradel
Research Collection School Of Computing and Information Systems
Experimental evaluations of software engineering innovations, e.g., tools and processes, often include human-subject studies as a component of a multi-pronged strategy to obtain greater generalizability of the findings. However, human-subject studies in our field are challenging, due to the cost and difficulty of finding and employing suitable subjects, ideally, professional programmers with varying degrees of experience. Meanwhile, large language models (LLMs) have recently started to demonstrate human-level performance in several areas. This paper explores the possibility of substituting costly human subjects with much cheaper LLM queries in evaluations of code and code-related artifacts. We study this idea by applying six …
Comadice: Offline Cooperative Multi-Agent Reinforcement Learning With Stationary Distribution Shift Regularization, The Viet Bui, Tien Mai, Hong Thanh Nguyen
Comadice: Offline Cooperative Multi-Agent Reinforcement Learning With Stationary Distribution Shift Regularization, The Viet Bui, Tien Mai, Hong Thanh Nguyen
Research Collection School Of Computing and Information Systems
Offline reinforcement learning (RL) has garnered significant attention for its ability to learn effective policies from pre-collected datasets without the need for further environmental interactions. While promising results have been demonstrated in single-agent settings, offline multi-agent reinforcement learning (MARL) presents additional challenges due to the large joint state-action space and the complexity of multi-agent behaviors. A key issue in offline RL is the distributional shift, which arises when the target policy being optimized deviates from the behavior policy that generated the data. This problem is exacerbated in MARL due to the interdependence between agents' local policies and the expansive joint …
Chatcrs: Incorporating External Knowledge And Goal Guidance For Llm-Based Conversational Recommender Systems, Chuang Li, Yang Deng, Hengchang Hu, Min-Yen Kan, Haizhou Li
Chatcrs: Incorporating External Knowledge And Goal Guidance For Llm-Based Conversational Recommender Systems, Chuang Li, Yang Deng, Hengchang Hu, Min-Yen Kan, Haizhou Li
Research Collection School Of Computing and Information Systems
This paper aims to efficiently enable large language models (LLMs) to use external knowledge and goal guidance in conversational recommender system (CRS) tasks. Advanced LLMs (e.g., ChatGPT) are limited in domain-specific CRS tasks for 1) generating grounded responses with recommendation-oriented knowledge, or 2) proactively leading the conversations through different dialogue goals. In this work, we first analyze those limitations through a comprehensive evaluation, showing the necessity of external knowledge and goal guidance which contribute significantly to the recommendation accuracy and language quality. In light of this finding, we propose a novel ChatCRS framework to decompose the complex CRS task into …
Towards Understanding Why Fixmatch Generalizes Better Than Supervised Learning, Jingyang Li, Jiachun Pan, Vincent Tan, Kim-Chuan Toh, Pan Zhou
Towards Understanding Why Fixmatch Generalizes Better Than Supervised Learning, Jingyang Li, Jiachun Pan, Vincent Tan, Kim-Chuan Toh, Pan Zhou
Research Collection School Of Computing and Information Systems
Semi-supervised learning (SSL), exemplified by FixMatch (Sohn et al., 2020), has shown significant generalization advantages over supervised learning (SL), particularly in the context of deep neural networks (DNNs). However, it is still unclear, from a theoretical standpoint, why FixMatch-like SSL algorithms generalize better than SL on DNNs. In this work, we present the first theoretical justification for the enhanced test accuracy observed in FixMatch-like SSL applied to DNNs by taking convolutional neural networks (CNNs) on classification tasks as an example. Our theoretical analysis reveals that the semantic feature learning processes in FixMatch and SL are rather different. In particular, FixMatch …
Agentstudio: A Toolkit For Building General Virtual Agents, Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, Shuicheng Yan
Agentstudio: A Toolkit For Building General Virtual Agents, Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, Shuicheng Yan
Research Collection School Of Computing and Information Systems
General virtual agents need to handle multimodal observations, master complex action spaces, and self-improve in dynamic, open-domain environments. However, existing environments are often domain-specific and require complex setups, which limits agent development and evaluation in real-world settings. As a result, current evaluations lack in-depth analyses that decompose fundamental agent capabilities. We introduce AgentStudio, a trinity of environments, tools, and benchmarks to address these issues. AgentStudio provides a lightweight, interactive environment with highly generic observation and action spaces, e.g., video observations and GUI/API actions. It integrates tools for creating online benchmark tasks, annotating GUI elements, and labeling actions in videos. Based …
Prioritizing Speech Test Cases, Zhou Yang, Jieke Shi, Muhammad Hilmi Asyrofi, Bowen Xu, Xin Zhou, Donggyun Han, David Lo
Prioritizing Speech Test Cases, Zhou Yang, Jieke Shi, Muhammad Hilmi Asyrofi, Bowen Xu, Xin Zhou, Donggyun Han, David Lo
Research Collection School Of Computing and Information Systems
As Automated Speech Recognition (ASR) systems gain widespread acceptance, there is a pressing need to rigorously test and enhance their performance. Nonetheless, the process of collecting and executing speech test cases is typically both costly and time-consuming. This presents a compelling case for the strategic prioritization of speech test cases, which consist of a piece of audio and the corresponding reference text. The central question we address is: In what sequence should speech test cases be collected and executed to identify the maximum number of errors at the earliest stage? In this study, we introduce PRiOritizing sPeecH tEsT …
Bigcodebench: Benchmarking Code Generation With Diverse Function Calls And Complex Instructions, T.Y. Zhuo, M.C. Vu, J. Chim, ..., David Lo
Bigcodebench: Benchmarking Code Generation With Diverse Function Calls And Complex Instructions, T.Y. Zhuo, M.C. Vu, J. Chim, ..., David Lo
Research Collection School Of Computing and Information Systems
Task automation has been greatly empowered by the recent advances in Large Language Models (LLMs) via Python code, where the tasks ranging from software engineering development to general-purpose reasoning. While current benchmarks have shown that LLMs can solve tasks using programs like human developers, the majority of their evaluations are limited to short and self-contained algorithmic tasks or standalone function calls. Solving challenging and practical tasks requires the capability of utilizing diverse function calls as tools to efficiently implement functionalities like data analysis and web development. In addition, using multiple tools to solve a task needs compositional reasoning by accurately …
Adversarial Attacks On Event-Based Pedestrian Detectors: A Physical Approach, Guixu Lin, Muyao Niu, Qingtian Zhu, Zhengwei Yin, Zhuoxiao Li, Shengfeng He, Yinqiang Zheng
Adversarial Attacks On Event-Based Pedestrian Detectors: A Physical Approach, Guixu Lin, Muyao Niu, Qingtian Zhu, Zhengwei Yin, Zhuoxiao Li, Shengfeng He, Yinqiang Zheng
Research Collection School Of Computing and Information Systems
Event cameras, known for their low latency and high dynamic range, show great potential in pedestrian detection applications. However, while recent research has primarily focused on improving detection accuracy, the robustness of event-based visual models against physical adversarial attacks has received limited attention. For example, adversarial physical objects, such as specific clothing patterns or accessories, can exploit inherent vulnerabilities in these systems, leading to misdetections or misclassifications. This study is the first to explore physical adversarial attacks on event-driven pedestrian detectors, specifically investigating whether certain clothing patterns worn by pedestrians can cause these detectors to fail, effectively rendering them unable …
Retrieval Augmented Recipe Generation, Guoshan Liu, Hailong Yin, Bin Zhu, Jingjing Chen, Chong-Wah Ngo, Yu-Gang Jiang
Retrieval Augmented Recipe Generation, Guoshan Liu, Hailong Yin, Bin Zhu, Jingjing Chen, Chong-Wah Ngo, Yu-Gang Jiang
Research Collection School Of Computing and Information Systems
The growing interest in generating recipes from food images has drawn substantial research attention in recent years. Existing works for recipe generation primarily utilize a two-stage training method—first predicting ingredients from a food image and then generating instructions from both the image and ingredients. Large Multi-modal Models (LMMs), which have achieved notable success across a variety of vision and language tasks, shed light on generating both ingredients and instructions directly from images. Nevertheless, LMMs still face the common issue of hallu- cinations during recipe generation, leading to suboptimal performance. To tackle this issue, we propose a retrieval augmented large multimodal …
Imageinthat: Manipulating Images To Convey User Instructions To Robots, Karthik Mahadevan, Blaine Lewis, Jiannan Li, Bilge Mutlu, Anthony Tang, Tovi Grossman
Imageinthat: Manipulating Images To Convey User Instructions To Robots, Karthik Mahadevan, Blaine Lewis, Jiannan Li, Bilge Mutlu, Anthony Tang, Tovi Grossman
Research Collection School Of Computing and Information Systems
Foundation models are rapidly improving the capability of robots in performing everyday tasks autonomously such as meal preparation, yet robots will still need to be instructed by humans due to model performance, the difficulty of capturing user preferences, and the need for user agency. Robots can be instructed using various methods---natural language conveys immediate instructions but can be abstract or ambiguous, whereas end-user programming supports longer-horizon tasks but interfaces face difficulties in capturing user intent. In this work, we propose using direct manipulation of images as an alternative paradigm to instruct robots, and introduce a specific instantiation called ImageInThat which …
Neurovig: Integrating Event Cameras For Resource-Efficient Video Grounding, Dulanga Weerakoon, Vigneshwaran Subbaraju, Joo Hwee Lim, Archan Misra
Neurovig: Integrating Event Cameras For Resource-Efficient Video Grounding, Dulanga Weerakoon, Vigneshwaran Subbaraju, Joo Hwee Lim, Archan Misra
Research Collection School Of Computing and Information Systems
Spatio-Temporal Video Grounding (STVG) - the task of identifying the target object in the field-of-view that the language instruction refers to - is a fundamental vision-language task. Current STVG approaches typically utilize feeds from an RGB camera that is assumed to be always-on and process the video frames using complex neural network pipelines. As a result they often impose prohibitive system overheads (energy latency) on pervasive devices. To address this we propose NeuroViG with two key innovations: (a) leveraging on event streams from a low-power neuromorphic event camera sensor to perform selective triggering of the more energy-hungry RGB camera for …