Open Access. Powered by Scholars. Published by Universities.®

Computer Sciences Commons™

Open Access. Powered by Scholars. Published by Universities.®

2022

Discipline
Institution
Keyword
Publication
Publication Type
File Type

Articles 691 - 720 of 3613

Full-Text Articles in Computer Sciences

Dynamic Temporal Filtering In Video Models, Fuchen Long, Zhaofan Qiu, Yingwei Pan, Ting Yao, Chong-Wah Ngo, Tao Mei Oct 2022

Dynamic Temporal Filtering In Video Models, Fuchen Long, Zhaofan Qiu, Yingwei Pan, Ting Yao, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Video temporal dynamics is conventionally modeled with 3D spatial-temporal kernel or its factorized version comprised of 2D spatial kernel and 1D temporal kernel. The modeling power, nevertheless, is limited by the fixed window size and static weights of a kernel along the temporal dimension. The pre-determined kernel size severely limits the temporal receptive fields and the fixed weights treat each spatial location across frames equally, resulting in sub-optimal solution for longrange temporal modeling in natural scenes. In this paper, we present a new recipe of temporal feature learning, namely Dynamic Temporal Filter (DTF), that novelly performs spatial-aware temporal modeling in …

Go to article

Gpu Accelerated Estimation Of A Shared Random Effect Joint Model For Dynamic Prediction, Shikun Wang, Zhao Li, Lan Lan, Jieyi Zhao, W Jim Zheng, Liang Li Oct 2022

Gpu Accelerated Estimation Of A Shared Random Effect Joint Model For Dynamic Prediction, Shikun Wang, Zhao Li, Lan Lan, Jieyi Zhao, W Jim Zheng, Liang Li

Faculty, Staff and Student Publications

In longitudinal cohort studies, it is often of interest to predict the risk of a terminal clinical event using longitudinal predictor data among subjects at risk by the time of the prediction. The at-risk population changes over time; so does the association between predictors and the outcome, as well as the accumulating longitudinal predictor history. The dynamic nature of this prediction problem has received increasing interest in the literature, but computation often poses a challenge. The widely used joint model of longitudinal and survival data often comes with intensive computation and excessive model fitting time, due to numerical optimization and …

Go to article

Pruning The Communication Bandwidth Between Reinforcement Learning Agents Through Causal Inference: An Innovative Approach To Designing A Smart Grid Power System, Xianjie Zhang, Yu Liu, Wenjun Li, Chen Gong Oct 2022

Pruning The Communication Bandwidth Between Reinforcement Learning Agents Through Causal Inference: An Innovative Approach To Designing A Smart Grid Power System, Xianjie Zhang, Yu Liu, Wenjun Li, Chen Gong

Research Collection School Of Computing and Information Systems

Electricity demands are increasing significantly and the traditional power grid system is facing huge challenges. As the desired next-generation power grid system, smart grid can provide secure and reliable power generation, and consumption, and can also realize the system’s coordinated and intelligent power distribution. Coordinating grid power distribution usually requires mutual communication between power distributors to accomplish coordination. However, the power network is complex, the network nodes are far apart, and the communication bandwidth is often expensive. Therefore, how to reduce the communication bandwidth in the cooperative power distribution process task is crucially important. One way to tackle this problem …

Go to article

Codebert For Code Clone Detection: A Replication Study, Saad Arshad, Shamsa Abid, Shafay Shamail Oct 2022

Codebert For Code Clone Detection: A Replication Study, Saad Arshad, Shamsa Abid, Shafay Shamail

Research Collection School Of Computing and Information Systems

Large pre-trained models have dramatically improved the state-of-the-art on a variety of natural language processing (NLP) tasks. CodeBERT is one such pre-trained model for natural language (NL) and programming language (PL) which captures the semantics in natural language and programming language, and produces general-purpose representations. While it has been shown to support natural language code search and code documentation generation tasks, its effectiveness for code clone detection is not explored in depth. In this paper, we aim to replicate and evaluate the performance of CodeBERT for code clone detection on multiple datasets with varying functionalities to understand (1) whether CodeBERT …

Go to article

A Comparative Analysis Of Clone Detection Techniques On Semanticclonebench, Sohaib Masood Rabbani, Nabeel Ahmad Gulzar, Saad Arshad, Shamsa Abid, Shafay Shamail Oct 2022

A Comparative Analysis Of Clone Detection Techniques On Semanticclonebench, Sohaib Masood Rabbani, Nabeel Ahmad Gulzar, Saad Arshad, Shamsa Abid, Shafay Shamail

Research Collection School Of Computing and Information Systems

Semantic code clone detection involves the detection of functionally similar code fragments which may otherwise be lexically, syntactically, or structurally dissimilar. The detection of semantic code clones has important applications in aspect mining and product line analysis. The accurate detection of semantic code clones is a challenging task and various techniques have been proposed. However, the evaluation of these techniques is performed using various datasets and we do not have a clear picture of the performance of these techniques relative to each other. Recently, SemanticCloneBench has been introduced as a benchmark for semantic clones. Now, we can use the SemanticCloneBench …

Go to article

Social Access And Representation For Autistic Adult Livestreamers, Terrance Mok, Anthony Tang, Adam Mccrimmon, Lora Oehlberg Oct 2022

Social Access And Representation For Autistic Adult Livestreamers, Terrance Mok, Anthony Tang, Adam Mccrimmon, Lora Oehlberg

Research Collection School Of Computing and Information Systems

We interviewed 10 autistic livestreamers to understand their motivations for livestreaming on Twitch. Our participants explained that streaming helped them fulfill social desires by: supporting them in making meaningful social connections with others; giving them a safe space to practice social skills like “small talk”; and empowering them to be autistic role models and to share their true selves. This work offers an early report on how autistic individuals leverage livestreaming as a beneficial social platform while struggling with audience expectations.

Go to article

Cvfnet: Real-Time 3d Object Detection By Learning Cross View Features, Jiaqi Gu, Zhiyu Xiang, Pan Zhao, Tingming Bai, Lingxuan Wang, Xijun Zhao, Zhiyuan Zhang Oct 2022

Cvfnet: Real-Time 3d Object Detection By Learning Cross View Features, Jiaqi Gu, Zhiyu Xiang, Pan Zhao, Tingming Bai, Lingxuan Wang, Xijun Zhao, Zhiyuan Zhang

Research Collection School Of Computing and Information Systems

In recent years 3D object detection from LiDAR point clouds has made great progress thanks to the development of deep learning technologies. Although voxel or point based methods are popular in 3D object detection, they usually involve time-consuming operations such as 3D convolutions on voxels or ball query among points, making the resulting network inappropriate for time critical applications. On the other hand, 2D view-based methods feature high computing efficiency while usually obtaining inferior performance than the voxel or point based methods. In this work, we present a real-time view-based single stage 3D object detector, namely CVFNet to fulfill this …

Go to article

Editing Out-Of-Domain Gan Inversion Via Differential Activations, Haorui Song, Yong Du, Tianyi Xiang, Junyu Dong, Jing Qin, Shengfeng He Oct 2022

Editing Out-Of-Domain Gan Inversion Via Differential Activations, Haorui Song, Yong Du, Tianyi Xiang, Junyu Dong, Jing Qin, Shengfeng He

Research Collection School Of Computing and Information Systems

Despite the demonstrated editing capacity in the latent space of a pretrained GAN model, inverting real-world images is stuck in a dilemma that the reconstruction cannot be faithful to the original input. The main reason for this is that the distributions between training and real-world data are misaligned, and because of that, it is unstable of GAN inversion for real image editing. In this paper, we propose a novel GAN prior based editing framework to tackle the out-of-domain inversion problem with a composition-decomposition paradigm. In particular, during the phase of composition, we introduce a differential activation module for detecting semantic …

Go to article

Pixel-Wise Energy-Biased Abstention Learning For Anomaly Segmentation On Complex Urban Driving Scenes, Yu Tian, Yuyuan Liu, Guansong Pang, Fengbei Liu, Yuanhong Chen, Gustavo Carneiro Oct 2022

Pixel-Wise Energy-Biased Abstention Learning For Anomaly Segmentation On Complex Urban Driving Scenes, Yu Tian, Yuyuan Liu, Guansong Pang, Fengbei Liu, Yuanhong Chen, Gustavo Carneiro

Research Collection School Of Computing and Information Systems

State-of-the-art (SOTA) anomaly segmentation approaches on complex urban driving scenes explore pixel-wise classification uncertainty learned from outlier exposure, or external reconstruction models. However, previous uncertainty approaches that directly associate high uncertainty to anomaly may sometimes lead to incorrect anomaly predictions, and external reconstruction models tend to be too inefficient for real-time self-driving embedded systems. In this paper, we propose a new anomaly segmentation method, named pixel-wise energy-biased abstention learning (PEBAL), that explores pixel-wise abstention learning (AL) with a model that learns an adaptive pixel-level anomaly class, and an energy-based model (EBM) that learns inlier pixel distribution. More specifically, PEBAL is …

Go to article

Physical Adversarial Attack On A Robotic Arm, Yifan Jia, Christopher M. Poskitt, Jun Sun, Sudipta Chattopadhyay Oct 2022

Physical Adversarial Attack On A Robotic Arm, Yifan Jia, Christopher M. Poskitt, Jun Sun, Sudipta Chattopadhyay

Research Collection School Of Computing and Information Systems

Collaborative Robots (cobots) are regarded as highly safety-critical cyber-physical systems (CPSs) owing to their close physical interactions with humans. In settings such as smart factories, they are frequently augmented with AI. For example, in order to move materials, cobots utilize object detectors based on deep learning models. Deep learning, however, has been demonstrated as vulnerable to adversarial attacks: a minor change (noise) to benign input can fool the underlying neural networks and lead to a different result. While existing works have explored such attacks in the context of picture/object classification, less attention has been given to attacking neural networks used …

Go to article

Locally Varying Distance Transform For Unsupervised Visual Anomaly Detection, Wen-Yan Lin, Zhonghang Liu, Siying Liu Oct 2022

Locally Varying Distance Transform For Unsupervised Visual Anomaly Detection, Wen-Yan Lin, Zhonghang Liu, Siying Liu

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection on image data is notoriously unstable. We believe this is because many classical anomaly detectors implicitly assume data is low dimensional. However, image data is always high dimensional. Images can be projected to a low dimensional embedding but such projections rely on global transformations that truncate minor variations. As anomalies are rare, the final embedding often lacks the key variations needed to distinguish anomalies from normal instances. This paper proposes a new embedding using a set of locally varying data projections, with each projection responsible for persevering the variations that distinguish a local cluster of instances from …

Go to article

What Machines Can't Do (Yet) In Real Work Settings, Thomas H. Davenport, Steven M. Miller Oct 2022

What Machines Can't Do (Yet) In Real Work Settings, Thomas H. Davenport, Steven M. Miller

Research Collection School Of Computing and Information Systems

AI systems may perform well in the research lab or under highly controlled application settings, but they still needed human help in the types of real world work settings we researched for a new book, Working With AI: Real Stories of Human-Machine Collaboration. Human workers were very much in evidence across our 30 case studies. In this article, we use those examples to illustrate our list of AI-enabled activities that still require human assistance. These are activities where organizations need to continue to invest in human capital, and where practitioners can expect job continuity for the immediate future

Go to article

Multi-Functional Job Roles To Support Operations In A Multi-Faceted Jewel Enabled By Ai And Digital Transformation, Steven M. Miller Oct 2022

Multi-Functional Job Roles To Support Operations In A Multi-Faceted Jewel Enabled By Ai And Digital Transformation, Steven M. Miller

Research Collection School Of Computing and Information Systems

In this story, we highlight the way in which the use of AI enabled support systems, together with work process digital transformation and innovative approaches to job redesign, have combined to dramatically change the nature of the work of the front-line service staff who protect and support the facility and visitors at the world’s most iconic airport mall and lifestyle destination.

Go to article

Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, Koki Kusano Oct 2022

Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, Koki Kusano

Research Collection School Of Computing and Information Systems

Inaccessible e-commerce websites and mobile applications exclude people with visual impairments (PVI) from online shopping. Customer-to-customer (C2C) marketplaces, a form of e-commerce where trading happens not between businesses and customers but between customers, could pose a unique set of challenges in the interactions that the platform brings about. Through online questionnaire and remote interviews, we investigate problems experienced by people with low vision disabilities in common C2C scenarios. Our study with low vision participants (N = 12) reveal both previously known general accessibility issues (e.g., web and mobile interface accessibility) and C2C specific accessibility issues (e.g., inability to confirm item …

Go to article

Field Experiments In Operations Management, Yang Gao, Meng Li, Shujing Sun Oct 2022

Field Experiments In Operations Management, Yang Gao, Meng Li, Shujing Sun

Research Collection School Of Computing and Information Systems

While the field experiment is a powerful and well-established method to investigate causal relationships, operations management (OM) has embraced this methodology only in recent years. This paper provides a comprehensive review of the existing OM literature leveraging field experiments and serves as a one-stop guide for future application of field experiments in the OM area. We start by recapping the characteristics that distinguish field experiments from other common types of experiments and organizing the relevant OM studies by topic. Corresponding to the commonly overlooked issues in field experiment-based OM studies, we then provide a detailed roadmap, ranging from experimental design …

Go to article

Interactive Video Corpus Moment Retrieval Using Reinforcement Learning, Zhixin Ma, Chong-Wah Ngo Oct 2022

Interactive Video Corpus Moment Retrieval Using Reinforcement Learning, Zhixin Ma, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Known-item video search is effective with human-in-the-loop to interactively investigate the search result and refine the initial query. Nevertheless, when the first few pages of results are swamped with visually similar items, or the search target is hidden deep in the ranked list, finding the know-item target usually requires a long duration of browsing and result inspection. This paper tackles the problem by reinforcement learning, aiming to reach a search target within a few rounds of interaction by long-term learning from user feedbacks. Specifically, the system interactively plans for navigation path based on feedback and recommends a potential target that …

Go to article

Equivariance And Invariance Inductive Bias For Learning From Insufficient Data, Tan Wang, Qianru Sun, Sugiri Pranata, Karlekar Jayashree, Hanwang Zhang Oct 2022

Equivariance And Invariance Inductive Bias For Learning From Insufficient Data, Tan Wang, Qianru Sun, Sugiri Pranata, Karlekar Jayashree, Hanwang Zhang

Research Collection School Of Computing and Information Systems

We are interested in learning robust models from insufficient data, without the need for any externally pre-trained model checkpoints. First, compared to sufficient data, we show why insufficient data renders the model more easily biased to the limited training environments that are usually different from testing. For example, if all the training "swan" samples are "white", the model may wrongly use the "white" environment to represent the intrinsic class "swan". Then, we justify that equivariance inductive bias can retain the class feature while invariance inductive bias can remove the environmental feature, leaving only the class feature that generalizes to any …

Go to article

Class Is Invariant To Context And Vice Versa: On Learning Invariance For Out-Of-Distribution Generalization, Jiaxin Qi, Kaihua Tang, Qianru Sun, Xian-Sheng Hua, Hanwang Zhang Oct 2022

Class Is Invariant To Context And Vice Versa: On Learning Invariance For Out-Of-Distribution Generalization, Jiaxin Qi, Kaihua Tang, Qianru Sun, Xian-Sheng Hua, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Out-Of-Distribution generalization (OOD) is all about learning invariance against environmental changes. If the context in every class is evenly distributed, OOD would be trivial because the context can be easily removed due to an underlying principle: class is invariant to context. However, collecting such a balanced dataset is impractical. Learning on imbalanced data makes the model bias to context and thus hurts OOD. Therefore, the key to OOD is context balance.We argue that the widely adopted assumption in prior work—the context bias can be directly annotated or estimated from biased class prediction—renders the context incomplete or even incorrect. In contrast, …

Go to article

Soci: A Toolkit For Secure Outsourced Computation On Integers, Bowen Zhao, Jiaming Yuan, Ximeng Liu, Yongdong Wu, Hwee Hwa Pang, Robert H. Deng Oct 2022

Soci: A Toolkit For Secure Outsourced Computation On Integers, Bowen Zhao, Jiaming Yuan, Ximeng Liu, Yongdong Wu, Hwee Hwa Pang, Robert H. Deng

Research Collection School Of Computing and Information Systems

Secure outsourced computation is a key technique for protecting data security and privacy in the cloud. Although fully homomorphic encryption (FHE) enables computations over encrypted data, it suffers from high computation costs in order to support an unlimited number of arithmetic operations. Recently, secure computations based on interactions of multiple computation servers and partially homomorphic encryption (PHE) were proposed in the literature, which enable an unbound number of addition and multiplication operations on encrypted data more efficiently than FHE and do not add any noise to encrypted data; however, these existing solutions are either limited in functionalities (e.g., computation on …

Go to article

Mando: Multi-Level Heterogeneous Graph Embeddings For Fine-Grained Detection Of Smart Contract Vulnerabilities, Huu Hoang Nguyen, Nhat Minh Nguyen, Chunyao Xie, Zahra Ahmadi, Daniel Kudenko, Thanh Nam Doan, Lingxiao Jiang Oct 2022

Mando: Multi-Level Heterogeneous Graph Embeddings For Fine-Grained Detection Of Smart Contract Vulnerabilities, Huu Hoang Nguyen, Nhat Minh Nguyen, Chunyao Xie, Zahra Ahmadi, Daniel Kudenko, Thanh Nam Doan, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Learning heterogeneous graphs consisting of different types of nodes and edges enhances the results of homogeneous graph techniques. An interesting example of such graphs is control-flow graphs representing possible software code execution flows. As such graphs represent more semantic information of code, developing techniques and tools for such graphs can be highly beneficial for detecting vulnerabilities in software for its reliability. However, existing heterogeneous graph techniques are still insufficient in handling complex graphs where the number of different types of nodes and edges is large and variable. This paper concentrates on the Ethereum smart contracts as a sample of software …

Go to article

Ffl: Fine Grained Fault Localization For Student Programs Via Syntactic And Semantic Reasoning, Thanh Dat Nguyen, Cong Thanh Le, Duc-Minh Luong, Van-Hai Duong, Xuan Bach Le, David Lo, Quyet-Thang Huynh Oct 2022

Ffl: Fine Grained Fault Localization For Student Programs Via Syntactic And Semantic Reasoning, Thanh Dat Nguyen, Cong Thanh Le, Duc-Minh Luong, Van-Hai Duong, Xuan Bach Le, David Lo, Quyet-Thang Huynh

Research Collection School Of Computing and Information Systems

Fault localization has been used to provide feedback for incorrect student programs since locations of faults can be a valuable hint for students about what caused their programs to crash. Unfortunately, existing fault localization techniques for student programs are limited because they usually consider either the program’s syntax or semantics alone. This motivates the new design of fault localization techniques that use both semantic and syntactical information of the program. In this paper, we introduce FFL (Fine grained Fault Localization), a novel technique using syntactic and semantic reasoning for localizing bugs in student programs. The novelty in FFL that allows …

Go to article

Softskip: Empowering Multi-Modal Dynamic Pruning For Single-Stage Referring Comprehension, Dulanga Weerakoon, Vigneshwaran Subbaraju, Tuan Tran, Archan Misra Oct 2022

Softskip: Empowering Multi-Modal Dynamic Pruning For Single-Stage Referring Comprehension, Dulanga Weerakoon, Vigneshwaran Subbaraju, Tuan Tran, Archan Misra

Research Collection School Of Computing and Information Systems

Supporting real-time referring expression comprehension (REC) on pervasive devices is an important capability for human-AI collaborative tasks. Model pruning techniques, applied to DNN models, can enable real-time execution even on resource-constrained devices. However, existing pruning strategies are designed principally for uni-modal applications, and suffer a significant loss of accuracy when applied to REC tasks that require fusion of textual and visual inputs. We thus present a multi-modal pruning model, LGMDP, which uses language as a pivot to dynamically and judiciously select the relevant computational blocks that need to be executed. LGMDP also introduces a new SoftSkip mechanism, whereby 'skipped' visual …

Go to article

Accurate Generation Of Trigger-Action Programs With Domain-Adapted Sequence-To-Sequence Learning, Imam Nur Bani Yusuf, Lingxiao Jiang, David Lo Oct 2022

Accurate Generation Of Trigger-Action Programs With Domain-Adapted Sequence-To-Sequence Learning, Imam Nur Bani Yusuf, Lingxiao Jiang, David Lo

Research Collection School Of Computing and Information Systems

Trigger-action programming allows end users to write event-driven rules to automate smart devices and internet services. Users can create a trigger-action program (TAP) by specifying triggers and actions from a set of predefined functions along with suitable data fields for the functions. Many trigger-action programming platforms have emerged as the popularity grows, e.g., IFTTT, Microsoft Power Automate, and Samsung SmartThings. Despite their simplicity, composing trigger-action programs (TAPs) can still be challenging for end users due to the domain knowledge needed and enormous search space of many combinations of triggers and actions. We propose RecipeGen, a new deep learning-based approach that …

Go to article

Lawbreaker: An Approach For Specifying Traffic Laws And Fuzzing Autonomous Vehicles, Yang Sun, Christopher M. Poskitt, Jun Sun, Yuqi Chen, Zijiang Yang Oct 2022

Lawbreaker: An Approach For Specifying Traffic Laws And Fuzzing Autonomous Vehicles, Yang Sun, Christopher M. Poskitt, Jun Sun, Yuqi Chen, Zijiang Yang

Research Collection School Of Computing and Information Systems

Autonomous driving systems (ADSs) must be tested thoroughly before they can be deployed in autonomous vehicles. High-fidelity simulators allow them to be tested against diverse scenarios, including those that are difficult to recreate in real-world testing grounds. While previous approaches have shown that test cases can be generated automatically, they tend to focus on weak oracles (e.g. reaching the destination without collisions) without assessing whether the journey itself was undertaken safely and satisfied the law. In this work, we propose LawBreaker, an automated framework for testing ADSs against real-world traffic laws, which is designed to be compatible with different scenario …

Go to article

Improving Knowledge-Aware Recommendation With Multi-Level Interactive Contrastive Learning, Ding Zou, Wei Wei, Ziyang Wang, Xian-Ling Mao, Feida Zhu, Rui Fang, Dangyang Chen Oct 2022

Improving Knowledge-Aware Recommendation With Multi-Level Interactive Contrastive Learning, Ding Zou, Wei Wei, Ziyang Wang, Xian-Ling Mao, Feida Zhu, Rui Fang, Dangyang Chen

Research Collection School Of Computing and Information Systems

Incorporating Knowledge Graphs (KG) into recommeder system as side information has attracted considerable attention. Recently, the technical trend of Knowledge-aware Recommendation (KGR) is to develop end-to-end models based on graph neural networks (GNNs). However, the extremely sparse user-item interactions significantly degrade the performance of the GNN-based models, from the following aspects: 1) the sparse interaction, itself, means inadequate supervision signals and limits the supervised GNN-based models; 2) the combination of sparse interactions (CF part) and redundant KG facts (KG part) further results in an unbalanced information utilization. Besides, the GNN paradigm aggregates local neighbors for node representation learning, while ignoring …

Go to article

Right To Know, Right To Refuse: Towards Ui Perception-Based Automated Fine-Grained Permission Controls For Android Apps, Vikas Kumar Malviya, Chee Wei Leow, Ashok Kasthuri, Naing Tun Yan, Lwin Khin Shar, Lingxiao Jiang Oct 2022

Right To Know, Right To Refuse: Towards Ui Perception-Based Automated Fine-Grained Permission Controls For Android Apps, Vikas Kumar Malviya, Chee Wei Leow, Ashok Kasthuri, Naing Tun Yan, Lwin Khin Shar, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

It is the basic right of a user to know how the permissions are used within the Android app’s scope and to refuse the app if granted permissions are used for the activities other than specified use which can amount to malicious behavior. This paper proposes an approach and a vision to automatically model the permissions necessary for Android apps from users’ perspective and enable fine-grained permission controls by users, thus facilitating users in making more well-informed and flexible permission decisions for different app functionalities, which in turn improve the security and data privacy of the App and enforce apps …

Go to article

Adding Context To Source Code Representations For Deep Learning, Fuwei Tian, Christoph Treude Oct 2022

Adding Context To Source Code Representations For Deep Learning, Fuwei Tian, Christoph Treude

Research Collection School Of Computing and Information Systems

Deep learning models have been successfully applied to a variety of software engineering tasks, such as code classification, summarisation, and bug and vulnerability detection. In order to apply deep learning to these tasks, source code needs to be represented in a format that is suitable for input into the deep learning model. Most approaches to representing source code, such as tokens, abstract syntax trees (ASTs), data flow graphs (DFGs), and control flow graphs (CFGs) only focus on the code itself and do not take into account additional context that could be useful for deep learning models. In this paper, we …

Go to article

Self-Promoted Supervision For Few-Shot Transformer, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo Oct 2022

Self-Promoted Supervision For Few-Shot Transformer, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo

Research Collection School Of Computing and Information Systems

The few-shot learning ability of vision transformers (ViTs) is rarely investigated though heavily desired. In this work, we empirically find that with the same few-shot learning frameworks, e.g. MetaBaseline, replacing the widely used CNN feature extractor with a ViT model often severely impairs few-shot classification performance. Moreover, our empirical study shows that in the absence of inductive bias, ViTs often learn the low-qualified token dependencies under few-shot learning regime where only a few labeled training data are available, which largely contributes to the above performance degradation. To alleviate this issue, for the first time, we propose a simple yet effective …

Go to article

Video Graph Transformer For Video Question Answering, Junbin Xiao, Pan Zhou, Tat-Seng Chua, Shuicheng Yan Oct 2022

Video Graph Transformer For Video Question Answering, Junbin Xiao, Pan Zhou, Tat-Seng Chua, Shuicheng Yan

Research Collection School Of Computing and Information Systems

This paper proposes a Video Graph Transformer (VGT) model for Video Quetion Answering (VideoQA). VGT’s uniqueness are two-fold: 1) it designs a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations, and dynamics for complex spatio-temporal reasoning; and 2) it exploits disentangled video and text Transformers for relevance comparison between the video and text to perform QA, instead of entangled crossmodal Transformer for answer classification. Vision-text communication is done by additional cross-modal interaction modules. With more reasonable video encoding and QA solution, we show that VGT can achieve much better performances on VideoQA tasks …

Go to article

Mix-Dann And Dynamic-Modal-Distillation For Video Domain Adaptation, Yuehao Yin, Bin Zhu, Jingjing Chen, Lechao Cheng, Yu-Gang Jiang Oct 2022

Mix-Dann And Dynamic-Modal-Distillation For Video Domain Adaptation, Yuehao Yin, Bin Zhu, Jingjing Chen, Lechao Cheng, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Video domain adaptation is non-trivial due to video is inherently involved with multi-dimensional and multi-modal information. Existing works mainly adopt adversarial learning and self-supervised tasks to align features. Nevertheless, the explicit interaction between source and target in the temporal dimension, as well as the adaptation between modalities, are unexploited. In this paper, we propose Mix-Domain-Adversarial Neural Network and Dynamic-Modal-Distillation (MD-DMD), a novel multi-modal adversarial learning framework for unsupervised video domain adaptation. Our approach incorporates the temporal information between source and target domains, as well as the diversity of adaptability between modalities. On the one hand, for every single modality, we …

Go to article

First Prev Next Last