Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 4 of 4
Full-Text Articles in Physical Sciences and Mathematics
Adan: Adaptive Nesterov Momentum Algorithm For Faster Optimizing Deep Models, Xingyu Xie, Pan Zhou, Huan Li, Zhouchen Lin, Shuicheng Yan
Adan: Adaptive Nesterov Momentum Algorithm For Faster Optimizing Deep Models, Xingyu Xie, Pan Zhou, Huan Li, Zhouchen Lin, Shuicheng Yan
Research Collection School Of Computing and Information Systems
In deep learning, different kinds of deep networks typically need different optimizers, which have to be chosen after multiple trials, making the training process inefficient. To relieve this issue and consistently improve the model training speed across deep networks, we propose the ADAptive Nesterov momentum algorithm, Adan for short. Adan first reformulates the vanilla Nesterov acceleration to develop a new Nesterov momentum estimation (NME) method, which avoids the extra overhead of computing gradient at the extrapolation point. Then Adan adopts NME to estimate the gradient's first- and second-order moments in adaptive gradient algorithms for convergence acceleration. Besides, we prove that …
Adaptive Aggregation Networks For Class-Incremental Learning, Yaoyao Liu, Bernt Schiele, Qianru Sun
Adaptive Aggregation Networks For Class-Incremental Learning, Yaoyao Liu, Bernt Schiele, Qianru Sun
Research Collection School Of Computing and Information Systems
Class-Incremental Learning (CIL) aims to learn a classification model with the number of classes increasing phase-by-phase. An inherent problem in CIL is the stability-plasticity dilemma between the learning of old and new classes, i.e., high-plasticity models easily forget old classes, but high-stability models are weak to learn new classes. We alleviate this issue by proposing a novel network architecture called Adaptive Aggregation Networks (AANets) in which we explicitly build two types of residual blocks at each residual level (taking ResNet as the baseline architecture): a stable block and a plastic block. We aggregate the output feature maps from these two …
Breaking Neural Reasoning Architectures With Metamorphic Relation-Based Adversarial Examples, Alvin Chan, Lei Ma, Felix Juefei-Xu, Yew-Soon Ong, Xiaofei Xie, Minhui Xue, Yang Liu
Breaking Neural Reasoning Architectures With Metamorphic Relation-Based Adversarial Examples, Alvin Chan, Lei Ma, Felix Juefei-Xu, Yew-Soon Ong, Xiaofei Xie, Minhui Xue, Yang Liu
Research Collection School Of Computing and Information Systems
The ability to read, reason, and infer lies at the heart of neural reasoning architectures. After all, the ability to perform logical reasoning over language remains a coveted goal of Artificial Intelligence. To this end, models such as the Turing-complete differentiable neural computer (DNC) boast of real logical reasoning capabilities, along with the ability to reason beyond simple surface-level matching. In this brief, we propose the first probe into DNC's logical reasoning capabilities with a focus on text-based question answering (QA). More concretely, we propose a conceptually simple but effective adversarial attack based on metamorphic relations. Our proposed adversarial attack …
Virtualization In Wireless Sensor Networks: Fault Tolerant Embedding For Internet Of Things, Omprakash Kaiwartya, Abdul Hanan Abdullah, Yue Cao, Jaime Lloret, Sushil Kumar, Rajiv Ratn Shah, Mukesh Prasad, Shiv Prakash
Virtualization In Wireless Sensor Networks: Fault Tolerant Embedding For Internet Of Things, Omprakash Kaiwartya, Abdul Hanan Abdullah, Yue Cao, Jaime Lloret, Sushil Kumar, Rajiv Ratn Shah, Mukesh Prasad, Shiv Prakash
Research Collection School Of Computing and Information Systems
Recently, virtualization in wireless sensor networks (WSNs) has witnessed significant attention due to the growing service domain for IoT. Related literature on virtualization in WSNs explored resource optimization without considering communication failure in WSNs environments. The failure of a communication link in WSNs impacts many virtual networks running IoT services. In this context, this paper proposes a framework for optimizing fault tolerance in virtualization in WSNs, focusing on heterogeneous networks for service-oriented IoT applications. An optimization problem is formulated considering fault tolerance and communication delay as two conflicting objectives. An adapted non-dominated sorting based genetic algorithm (A-NSGA) is developed to …