Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 2 of 2
Full-Text Articles in Physical Sciences and Mathematics
Adaptive Loss-Aware Quantization For Multi-Bit Networks, Zhongnan Qu, Zimu Zhou, Yun Cheng, Lothar Thiele
Adaptive Loss-Aware Quantization For Multi-Bit Networks, Zhongnan Qu, Zimu Zhou, Yun Cheng, Lothar Thiele
Research Collection School Of Computing and Information Systems
We investigate the compression of deep neural networks by quantizing their weights and activations into multiple binary bases, known as multi-bit networks (MBNs), which accelerate the inference and reduce the storage for the deployment on low-resource mobile and embedded platforms. We propose Adaptive Loss-aware Quantization (ALQ), a new MBN quantization pipeline that is able to achieve an average bitwidth below one-bit without notable loss in inference accuracy. Unlike previous MBN quantization solutions that train a quantizer by minimizing the error to reconstruct full precision weights, ALQ directly minimizes the quantizationinduced error on the loss function involving neither gradient approximation nor …
Connectionist Expert System With Adaptive Learning Capability, B. T. Low, Hochung Lui, Ah-Hwee Tan, Hoonheng Teh
Connectionist Expert System With Adaptive Learning Capability, B. T. Low, Hochung Lui, Ah-Hwee Tan, Hoonheng Teh
Research Collection School Of Computing and Information Systems
A neural network expert system called adaptive connectionist expert system (ACES) which will learn adaptively from past experience is described. ACES is based on the neural logic network, which is capable of doing both pattern processing and logical inferencing. The authors discuss two strategies, pattern matching ACES and rule inferencing ACES. The pattern matching ACES makes use of past examples to construct its neural logic network and fine-tunes itself adaptively during its use by further examples supplied. The rule inferencing ACES conceptualizes new rules based on the frequencies of use on the rule-based neural logic network. A new rule could …