Digital Commons Network^™

Causality Analysis For Neural Network Security, Bing Sun

Dissertations and Theses Collection (Open Access)

While neural networks are demonstrating excellent performance in a wide range of applications, there has been a growing concern on their reliability and dependability.Similar to traditional decision-making programs, neural networks inevitably have defects that need to be identified and mitigated at times. Neural networks are usually inherently black-boxes and do not provide explanations on how and why decisions are made. As a result, these defects are more ``hidden" and more challenging to eliminate. It is thus crucial to develop systematic approaches to identify and mitigate defects in a neural network in a rigorous way.

In this dissertation, we focus on …

Robust Learning With Probabilistic Relaxation Using Hypothesis-Test-Based Sampling, Zilin Wang

Dissertations and Theses Collection (Open Access)

In recent years, deep learning has been a vital tool in various tasks. The performance of a neural network is usually evaluated by empirical risk minimization. However, robustness issues have gained great concern which can be fatal in safety-critical applications. Adversarial training can mitigate the issue by minimizing the loss of worst-case perturbations of data. It is effective in improving the robustness of the model, but it is too conservative, and the plain performance of the model can be unsatisfying. Probabilistic Robust Learning (PRL) empirically balances the average- and worst-case performance while the robustness of the model is not provable …

Towards Robust, Secure, And Privacy-Aware Large Language Models Of Code, Zhou Yang

Dissertations and Theses Collection (Open Access)

The field of software engineering has witnessed a surge in large language models specifically tailored to understand and process code, which we call large language models for code (LLM4Code). The increasing popularity of LLM4Code is inseparable from three key factors: the availability of extensive datasets compiled from diverse data sources, the advancements in deep learning algorithms and computational power that facilitate the training of these powerful models, and the active engagement and collaboration within the research community fostering innovation and the rapid exchange of ideas and methodologies. As evidenced by a series of studies, LLM4Code has been experiencing rapid development …

A Data-Driven Approach For Automated Multi-Site Competitive Facility Location, Ming Hui Tan, Kar Way Tan, Hoong Chuin Lau

Research Collection School Of Computing and Information Systems

This paper addresses the challenge of optimal retail expansion in competitive urban environments through a novel approach to the Competitive Facility Location (CFL) problem. Traditional methods for solving CFL problems often struggle with large-scale scenarios, relying on manual pre-selection of candidate sites and imposing limitations on the number of new locations. Our approach leverages Adaptive Large Neighborhood Search (ALNS) enhanced with data enrichment techniques, including community detection on road networks and population weighting based on mobility data. We developed two ALNS variants: Community Geometric Centroid (CGC-ALNS) and Population Weighted Centroid (PWC-ALNS). These methods automate site selection, eliminating manual pre-selection while …

Harnessing The Power Of Ai-Instructor Collaborative Grading Approach: Topic-Based Effective Grading For Semi Open-Ended Multipart Questions, Phyo Yi Win Myint, Siaw Ling Lo, Yuhao Zhang

Research Collection School Of Computing and Information Systems

Semi open-ended multipart questions consist of multiple sub questions within a single question, requiring students to provide certain factual information while allowing them to express their opinion within a defined context. Human grading of such questions can be tedious, constrained by the marking scheme and susceptible to the subjective judgement of instructors. The emergence of large language models (LLMs) such as ChatGPT has significantly advanced the prospect of automatic grading in educational settings. This paper introduces a topic-based grading approach that harnesses LLM capabilities alongside a refined marking scheme to ensure fair and explainable assessment processes. The proposed approach involves …

Gtree: Gpu-Friendly Privacy-Preserving Decision Tree Training And Inference, Qifan Wang, Shujie Cui, Lei Zhou, Ye Dong, Jianli Bai, Yun Sing Koh, Giovanni Russello

Research Collection School Of Computing and Information Systems

Outsourcing Decision tree (DT) training and inference to cloud platforms raises privacy concerns. Recent Secure Multi-Party Computation (MPC)-based methods are hindered by heavy overhead. Few recent studies explored GPUs to improve MPC-protected deep learning, yet integrating GPUs into MPC-protected DT with massive data-dependent operations remains challenging, raising question: can MPC-protected DT training and inference fully leverage GPUs for optimal performance?We present GTree, the first scheme that exploits GPU to accelerate MPC-protected secure DT training and inference. GTree is built across 3 parties who jointly perform DT training and inference with GPUs. GTree is secure against semi-honest adversaries, ensuring that no …

Artificial Intelligence And Machine Learning In Cancer Pain: A Systematic Review, Vivian Salama, Brandon Godinich, Yimin Geng, Laia Humbert-Vidan, Laura Maule, Kareem A Wahid, Mohamed A Naser, Renjie He, Abdallah S R Mohamed, Clifton D Fuller, Amy C Moreno

Faculty, Staff and Student Publications

Background/objectives: Pain is a challenging multifaceted symptom reported by most cancer patients. This systematic review aims to explore applications of artificial intelligence/machine learning (AI/ML) in predicting pain-related outcomes and pain management in cancer.

Methods: A comprehensive search of Ovid MEDLINE, EMBASE and Web of Science databases was conducted using terms: "Cancer," "Pain," "Pain Management," "Analgesics," "Artificial Intelligence," "Machine Learning," and "Neural Networks" published up to September 7, 2023. AI/ML models, their validation and performance were summarized. Quality assessment was conducted using PROBAST risk-of-bias andadherence to TRIPOD guidelines.

Results: Forty four studies from 2006 to 2023 were included. Nineteen studies used …

Addressing Inference Time Of Machine Learning Models In Embedded Systems, Samuel Black

UNLV Theses, Dissertations, Professional Papers, and Capstones

Embedded Systems are used for a wide range of specialized computing purposes including surveyal, safety, security, and quality of life. Many areas that embedded systems are used in require the use of machine learning models. Constraints can be placed on embedded systems. Timeliness of execution, user satisfaction, security, power, and resource limitations must be considered when designing for embedded systems. Neural networks excel at complex tasks that are otherwise intractable, but their relatively high computational cost poses a challenge for inclusion in embedded systems. Neural network architectures should be optimized to reduce the total number of operations performed while maintaining …

Mechanically Cost-Effective Approach For Bipedal Walking In Robots Using Instantaneous Collision Angle, Smit R. Patel

UNLV Theses, Dissertations, Professional Papers, and Capstones

Humans, as bipedal locomotors, are effective at reducing the mechanical cost of transport (CoTmech) by adopting movement strategies and gaits that minimize energy expenditure for a given distance. By using different gaits at different speeds, leveraging their long spring-like tendons and muscle elasticity which store and release energy during movement, humans reduce the mechanical effort required for locomotion. Current locomotion solutions offered in bipedal robots, based on legacy walking and running gait models, are not great at energy efficiency unless walking at very low speeds. Additionally, the control system of robots, designed to ensure stability and adaptability, requires substantial resources, …

Reference Dependence In Queue Design And Pricing Strategies, Jian Liu, Yongpin Zhou, Jian Chen, Peng Li

Electrical and Computer Engineering Faculty Research & Creative Works

This research investigates the effect of reference dependence on waiting times in service systems which formerly used a first-in-first-out (FIFO) service but have introduced a priority line with a fee. Our model combines reference-dependent gain-loss utility with standard customer utility, and we posit that customers are pleased with shorter-than-expected waiting times, whereas longer-than-expected times lead to dissatisfaction and an increased likelihood of balking. The study explores two scenarios: a captive customer system (CCS) and a noncaptive customer system (NCCS), with a focus on optimal pricing and segmentation strategies for revenue and social welfare maximization. The results reveal that, in a …

A Full-History Network Dataset For Btc Asset Decentralization Profiling, Ling Cheng, Qian Shao, Fengzhu Zeng, Feida Zhu

Research Collection School Of Computing and Information Systems

Since its advent in 2009, Bitcoin (BTC) has garnered increasing attention from both academia and industry. However, due to the massive transaction volume, no systematic study has quantitatively measured the asset decentralization degree specifically from a network perspective.In this paper, by conducting a thorough analysis of the BTC transaction network, we first address the significant gap in the availability of full-history BTC graph and network property dataset, which spans over 15 years from the genesis block (1st March, 2009) to the 845651-th block (29, May 2024). We then present the first systematic investigation to profile BTC's asset decentralization and design …

Agchain: A Blockchain-Based Gateway For Trustworthy App Delegation From Mobile App Markets, Mengjie Chen, Xiao Yi, Daoyuan Wu, Jianliang Xu, Yingjiu Li, Debin Gao

Research Collection School Of Computing and Information Systems

The popularity of smartphones has led to the growth of mobile app markets, creating a need for enhanced transparency, global access, and secure downloading. This paper introduces AGChain, a blockchain-based gateway that enables trustworthy app delegation within existing markets. AGChain ensures that markets can continue providing services while users benefit from permanent, distributed, and secure app delegation. During its development, we address two key challenges: significantly reducing smart contract gas costs and enabling fully distributed IPFS-based file storage. Additionally, we tackle three system issues related to security and sustainability. We have implemented a prototype of AGChain on Ethereum and Polygon …

Triadic Temporal-Semantic Alignment For Weakly-Supervised Video Moment Retrieval, Jin Liu, Jialong Xie, Fengyu Zhou, Shengfeng He

Research Collection School Of Computing and Information Systems

Video Moment Retrieval (VMR) aims to identify specific event moments within untrimmed videos based on natural language queries. Existing VMR methods have been criticized for relying heavily on moment annotation bias rather than true multi-modal alignment reasoning. Weakly supervised VMR approaches inherently overcome this issue by training without precise temporal location information. However, they struggle with fine-grained semantic alignment and often yield multiple speculative predictions with prolonged video spans. In this paper, we take a step forward in the context of weakly supervised VMR by proposing a triadic temporalsemantic alignment model. Our proposed approach augments weak supervision by comprehensively addressing …

An Aggregate Matching And Pick-Up Model For Mobility-On-Demand Services, Xinwei Li, Jintao Ke, Hai Yang, Hai Wang, Yaqian Zhou

Research Collection School Of Computing and Information Systems

This paper presents an Aggregate Matching and Pick-up (AMP) model to delineate the matching and pick-up processes in mobility-on-demand (MoD) service markets by explicitly considering the matching mechanisms in terms of matching intervals and matching radii. With passenger demand rate, vehicle fleet size and matching strategies as inputs, the AMP model can well approximate drivers’ idle time and passengers’ waiting time for matching and pick-up by considering batch matching in a stationary state. Properties of the AMP model are then analyzed, including the relationship between passengers’ waiting time and drivers’ idle time, and their changes with market thickness, which is …

Towards Unified Multimodal Editing With Enhanced Knowledge Collaboration, Kaihang Pan, Zhaoyu Fan, Juncheng Li, Qifan Yu, Hao Fei, Siliang Tang, Richang Hong, Hanwang Zhang, Qianru Sun

Research Collection School Of Computing and Information Systems

The swift advancement in Multimodal LLMs (MLLMs) also presents significant challenges for effective knowledge editing. Current methods, including intrinsic knowledge editing and external knowledge resorting, each possess strengths and weaknesses, struggling to balance the desired properties of reliability, generality, and locality when applied to MLLMs. In this paper, we propose UniKE, a novel multimodal editing method that establishes a unified perspective and paradigm for intrinsic knowledge editing and external knowledge resorting. Both types of knowledge are conceptualized as vectorized key-value memories, with the corresponding editing processes resembling the assimilation and accommodation phases of human cognition, conducted at the same semantic …

Learning De-Biased Representations For Remote-Sensing Imagery, Zichen Tian, Zhaozheng Chen, Qianru Sun

Research Collection School Of Computing and Information Systems

Remote sensing (RS) imagery, requiring specialized satellites to collect and being difficult to annotate, suffers from data scarcity and class imbalance in certain spectrums. Due to data scarcity, training any large-scale RS models from scratch is unrealistic, and the alternative is to transfer pre-trained models by fine-tuning or a more data-efficient method LoRA. Due to class imbalance, transferred models exhibit strong bias, where features of the major class dominate over those of the minor class. In this paper, we propose debLoRA---a generic training approach that works with any LoRA variants to yield debiased features. It is an unsupervised learning approach …

Reinforcement Learning Based Online Request Scheduling Framework For Workload-Adaptive Edge Deep Learning Inference, Xinrui Tan, Hongjia Li, Xiaofei Xie, Lu Guo, Nirwan Ansari, Xueqing Huang, Liming Wang, Zhen Xu, Yang Liu

Research Collection School Of Computing and Information Systems

The recent advances of deep learning in various mobile and Internet-of-Things applications, coupled with the emergence of edge computing, have led to a strong trend of performing deep learning inference on the edge servers located physically close to the end devices. This trend presents the challenge of how to meet the quality-of-service requirements of inference tasks at the resource-constrained network edge, especially under variable or even bursty inference workloads. Solutions to this challenge have not yet been reported in the related literature. In the present paper, we tackle this challenge by means of workload-adaptive inference request scheduling: in different workload …

Mvgamba : Unify 3d Content Generation As State Space Sequence Modeling, Xuanyu Yi, Zike Wu, Qiuhong Shen, Qingshan Xu, Pan Zhou, Joo-Hwee Lim, Shuicheng Yan, Xinchao Wang, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Recent 3D large reconstruction models (LRMs) can generate high-quality 3D content in sub-seconds by integrating multi-view diffusion models with scalable multi-view reconstructors. Current works further leverage 3D Gaussian Splatting as 3D representation for improved visual quality and rendering efficiency. However, we observe that existing Gaussian reconstruction models often suffer from multi-view inconsistency and blurred textures. We attribute this to the compromise of multi-view information propagation in favor of adopting powerful yet computationally intensive architectures (e.g., Transformers). To address this issue, we introduce MVGamba, a general and lightweight Gaussian reconstruction model featuring a multi-view Gaussian reconstructor based on the RNN-like State …

Unsupervised Modality Adaptation With Text-To-Image Diffusion Models For Semantic Segmentation, Ruihao Xia, Yu Liang, Peng-Tao Jiang, Hao Zhang, Bo Li, Yang Tang, Pan Zhou

Research Collection School Of Computing and Information Systems

Despite their success, unsupervised domain adaptation methods for semantic segmentation primarily focus on adaptation between image domains and do not utilize other abundant visual modalities like depth, infrared and event. This limitation hinders their performance and restricts their application in real-world multimodal scenarios. To address this issue, we propose Modality Adaptation with text-toimage Diffusion Models (MADM) for semantic segmentation task which utilizes text-to-image diffusion models pre-trained on extensive image-text pairs to enhance the model’s cross-modality capabilities. Specifically, MADM comprises two key complementary components to tackle major challenges. First, due to the large modality gap, using one modal data to generate …

Lova3 : Learning To Visual Question Answering, Asking And Assessment, Henry Hengyuan Zhao, Pan Zhou, Difei Gao, Bai Shou, Mike Zheng Shou

Research Collection School Of Computing and Information Systems

Question answering, asking, and assessment are three innate human traits crucial for understanding the world and acquiring knowledge. By enhancing these capabilities, humans can more effectively utilize data, leading to better comprehension and learning outcomes. Current Multimodal Large Language Models (MLLMs) primarily focus on question answering, often neglecting the full potential of questioning and assessment skills. Inspired by the human learning mechanism, we introduce LOVA3 , an innovative framework named “Learning tO Visual question Answering, Asking and Assessment,” designed to equip MLLMs with these additional capabilities. Our approach involves the creation of two supplementary training tasks GenQA and EvalQA, aiming …

4-Bit Shampoo For Memory-Efficient Network Training, Sike Wang, Pan Zhou, Jia Li, Hua Huang

Research Collection School Of Computing and Information Systems

Second-order optimizers, maintaining a matrix termed a preconditioner, are superior to first-order optimizers in both theory and practice. The states forming the preconditioner and its inverse root restrict the maximum size of models trained by second-order optimizers. To address this, compressing 32-bit optimizer states to lower bitwidths has shown promise in reducing memory usage. However, current approaches only pertain to first-order optimizers. In this paper, we propose the first 4-bit second-order optimizers, exemplified by 4-bit Shampoo, maintaining performance similar to that of 32-bit ones. We show that quantizing the eigenvector matrix of the preconditioner in 4-bit Shampoo is remarkably better …

Unified Generative And Discriminative Training For Multi-Modal Large Language Models, Wei Chow, Juncheng Li, Kaihang Pan, Qifan Yu, Hao Fei, Zhiqi Ge, Shuai Yang, Siliang Teng, Hanwang Zhang, Qianru Sun

Research Collection School Of Computing and Information Systems

In recent times, Vision-Language Models (VLMs) have been trained under two predominant paradigms. Generative training has enabled Multimodal Large Language Models (MLLMs) to tackle various complex tasks, yet issues such as hallucinations and weak object discrimination persist. Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval, yet struggles with complex scenarios requiring fine-grained semantic differentiation. This paper addresses these challenges by proposing a unified approach that integrates the strengths of both paradigms. Considering interleaved image-text sequences as the general format of input samples, we introduce a structure-induced training strategy that imposes semantic relationships between input …

Generative Semi-Supervised Graph Anomaly Detection, Hezhe Qiao, Qingsong Wen, Xiaoli Li, Ee-Peng Lim, Guansong Pang

Research Collection School Of Computing and Information Systems

This work considers a practical semi-supervised graph anomaly detection (GAD) scenario, where part of the nodes in a graph are known to be normal, contrasting to the extensively explored unsupervised setting with a fully unlabeled graph. We reveal that having access to the normal nodes, even just a small percentage of normal nodes, helps enhance the detection performance of existing unsupervised GAD methods when they are adapted to the semi-supervised setting. However, their utilization of these normal nodes is limited. In this paper we propose a novel Generative GAD approach (namely GGAD) for the semi-supervised scenario to better exploit the …

3d Snapshot: Invertible Embedding Of 3d Neural Representations In A Single Image, Yuqin Lu, Bailin Deng, Zhixuan Zhong, Tianle Zhang, Yuhui Quan, Hongmin Cai, Shengfeng He

Research Collection School Of Computing and Information Systems

3D neural rendering enables photo-realistic reconstruction of a specific scene by encoding discontinuous inputs into a neural representation. Despite the remarkable rendering results, the storage of network parameters is not transmission-friendly and not extendable to metaverse applications. In this paper, we propose an invertible neural rendering approach that enables generating an interactive 3D model from a single image (i.e., 3D Snapshot). Our idea is to distill a pre-trained neural rendering model (e.g., NeRF) into a visualizable image form that can then be easily inverted back to a neural network. To this end, we first present a neural image distillation method …

Mimicking To Dominate: Imitation Learning Strategies For Success In Multiagent Competitive Games, The Viet Bui, Tien Mai, Hong Thanh Nguyen

Research Collection School Of Computing and Information Systems

Training agents in multi-agent games presents significant challenges due to their intricate nature. These challenges are exacerbated by dynamics influenced not only by the environment but also by strategies of opponents. Existing methods often struggle with slow convergence and instability. To address these challenges, we harness the potential of imitation learning (IL) to comprehend and anticipate actions of the opponents, aiming to mitigate uncertainties with respect to the game dynamics. Our key contributions include: (i) a new multi-agent IL model for predicting next moves of the opponents --- our model works with hidden actions of opponents and local observations; (ii) …

Sampdetox : Black-Box Backdoor Defense Via Perturbation-Based Sample Detoxification, Yanxin Yang, Chentao Jia, Dengke Yan, Ming Hu, Tianlin Li, Xiaofei Xie, Xian Wei, Mingsong Chen

Research Collection School Of Computing and Information Systems

The advancement of Machine Learning has enabled the widespread deployment of Machine Learning as a Service (MLaaS) applications. However, the untrustworthy nature of third-party ML services poses backdoor threats. Existing defenses in MLaaS are limited by their reliance on training samples or white-box model analysis, highlighting the need for a black-box backdoor purification method. In our paper, we attempt to use diffusion models for purification by introducing noise in a forward diffusion process to destroy backdoors and recover clean samples through a reverse generative process. However, since a higher noise also destroys the semantics of the original samples, it still …

Learning To Handle Complex Constraints For Vehicle Routing Problems, Jieyi Bi, Yining Ma, Jianan Zhou, Wen Song, Zhiguang Cao, Yaoxin Wu, Jie Zhang

Research Collection School Of Computing and Information Systems

Vehicle Routing Problems (VRPs) can model many real-world scenarios and often involve complex constraints. While recent neural methods excel in constructing solutions based on feasibility masking, they struggle with handling complex constraints, especially when obtaining the masking itself is NP-hard. In this paper, we propose a novel Proactive Infeasibility Prevention (PIP) framework to advance the capabilities of neural methods towards more complex VRPs. Our PIP integrates the Lagrangian multiplier as a basis to enhance constraint awareness and introduces preventative infeasibility masking to proactively steer the solution construction process. Moreover, we present PIP-D, which employs an auxiliary decoder and two adaptive …

Sprinql : Sub-Optimal Demonstrations Driven Offline Imitation Learning, Minh Huy Hoang, Tien Mai, Pradeep Varakantham

Research Collection School Of Computing and Information Systems

We focus on offline imitation learning (IL), which aims to mimic an expert's behavior using demonstrations without any interaction with the environment. One of the main challenges in offline IL is the limited support of expert demonstrations, which typically cover only a small fraction of the state-action space. While it may not be feasible to obtain numerous expert demonstrations, it is often possible to gather a larger set of sub-optimal demonstrations. For example, in treatment optimization problems, there are varying levels of doctor treatments available for different chronic conditions. These range from treatment specialists and experienced general practitioners to less …

Self-Supervised Fine-Tuning For Neural Expert Finding, Budhitama Subagdja, Dan Sanchari, Ah-Hwee Tan

Research Collection School Of Computing and Information Systems

Expert finding systems allow ones to find individuals who have expertise in specific fields or domains. Traditional expert finding are mostly based on topic modeling or keyword search methods that are limited in their capability to encode contextual knowledge from natural language. To address the limitation, this paper presents Neural Expert Finder (NEF), a novel method that takes a transfer learning approach based on transformer encoder networks to leverage the rich seman-tic and syntactic patterns of language encoded in pre-trained language models (PLMs). We propose a self-supervised learning approach utilizing contrastive training using both positive and automatically generated negative samples …

Long-Tailed Out-Of-Distribution Detection Via Normalized Outlier Distribution Adaptation, Wenjun Miao, Guansong Pang, Jin Zheng, Xiao Bai

Research Collection School Of Computing and Information Systems

Onekeychallenge in Out-of-Distribution (OOD) detection is the absence of groundtruth OOD samples during training. One principled approach to address this issue is to use samples from external datasets as outliers (i.e., pseudo OOD samples) to train OOD detectors. However, we find empirically that the outlier samples often present a distribution shift compared to the true OOD samples, especially in LongTailed Recognition (LTR) scenarios, where ID classes are heavily imbalanced, i.e., the true OOD samples exhibit very different probability distribution to the head and tailed ID classes from the outliers. In this work, we propose a novel approach, namely normalized outlier …