Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
- Discipline
- Publication
- Publication Type
Articles 1 - 7 of 7
Full-Text Articles in Physical Sciences and Mathematics
Information Bottleneck In Deep Learning - A Semiotic Approach, Bogdan Musat, Razvan Andonie
Information Bottleneck In Deep Learning - A Semiotic Approach, Bogdan Musat, Razvan Andonie
Computer Science Faculty Scholarship
The information bottleneck principle was recently proposed as a theory meant to explain some of the training dynamics of deep neural architectures. Via information plane analysis, patterns start to emerge in this framework, where two phases can be distinguished: fitting and compression. We take a step further and study the behaviour of the spatial entropy characterizing the layers of convolutional neural networks (CNNs), in relation to the information bottleneck theory. We observe pattern formations which resemble the information bottleneck fitting and compression phases. From the perspective of semiotics, also known as the study of signs and sign-using behavior, the saliency …
Learning In Convolutional Neural Networks Accelerated By Transfer Entropy, Adrian Moldovan, Angel Caţaron, Răzvan Andonie
Learning In Convolutional Neural Networks Accelerated By Transfer Entropy, Adrian Moldovan, Angel Caţaron, Răzvan Andonie
Computer Science Faculty Scholarship
Recently, there is a growing interest in applying Transfer Entropy (TE) in quantifying the effective connectivity between artificial neurons. In a feedforward network, the TE can be used to quantify the relationships between neuron output pairs located in different layers. Our focus is on how to include the TE in the learning mechanisms of a Convolutional Neural Network (CNN) architecture. We introduce a novel training mechanism for CNN architectures which integrates the TE feedback connections. Adding the TE feedback parameter accelerates the training process, as fewer epochs are needed. On the flip side, it adds computational overhead to each epoch. …
Semiotic Aggregation In Deep Learning, Bogdan Muşat, Răzvan Andonie
Semiotic Aggregation In Deep Learning, Bogdan Muşat, Răzvan Andonie
All Faculty Scholarship for the College of the Sciences
Convolutional neural networks utilize a hierarchy of neural network layers. The statistical aspects of information concentration in successive layers can bring an insight into the feature abstraction process. We analyze the saliency maps of these layers from the perspective of semiotics, also known as the study of signs and sign-using behavior. In computational semiotics, this aggregation operation (known as superization) is accompanied by a decrease of spatial entropy: signs are aggregated into supersign. Using spatial entropy, we compute the information content of the saliency maps and study the superization processes which take place between successive layers of the network. In …
Modeling Multi-Targets Sentiment Classification Via Graph Convolutional Networks And Auxiliary Relation, Ao Feng, Zhengjie Gao, Xinyu Song, Ke Ke, Tianhao Xu, Xuelei Zhang
Modeling Multi-Targets Sentiment Classification Via Graph Convolutional Networks And Auxiliary Relation, Ao Feng, Zhengjie Gao, Xinyu Song, Ke Ke, Tianhao Xu, Xuelei Zhang
All Faculty Scholarship for the College of the Sciences
Existing solutions do not work well when multi-targets coexist in a sentence. The reason is that the existing solution is usually to separate multiple targets and process them separately. If the original sentence has N target, the original sentence will be repeated for N times, and only one target will be processed each time. To some extent, this approach degenerates the fine-grained sentiment classification task into the sentencelevel sentiment classification task, and the research method of processing the target separately ignores the internal relation and interaction between the targets. Based on the above considerations, we proposes to use Graph Convolutional …
Learning In Feedforward Neural Networks Accelerated By Transfer Entropy, Adrian Moldovan, Angel Caţaron, Rǎzvan Andonie
Learning In Feedforward Neural Networks Accelerated By Transfer Entropy, Adrian Moldovan, Angel Caţaron, Rǎzvan Andonie
All Faculty Scholarship for the College of the Sciences
Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of …
Weighted Random Search For Hyperparameter Optimization, Adrian-Cǎtǎlin Florea, Rǎzvan Andonie
Weighted Random Search For Hyperparameter Optimization, Adrian-Cǎtǎlin Florea, Rǎzvan Andonie
All Faculty Scholarship for the College of the Sciences
We introduce an improved version of Random Search (RS), used here for hyperparameter optimization of machine learning algorithms. Unlike the standard RS, which generates for each trial new values for all hyperparameters, we generate new values for each hyperparameter with a probability of change. The intuition behind our approach is that a value that already triggered a good result is a good candidate for the next step, and should be tested in new combinations of hyperparameter values. Within the same computational budget, our method yields better results than the standard RS. Our theoretical results prove this statement. We test our …
Automatic Classification Of Perceived Gender From Face Images, Joseph Lemley, Sami Abdul-Wahid, Dipayan Banik
Automatic Classification Of Perceived Gender From Face Images, Joseph Lemley, Sami Abdul-Wahid, Dipayan Banik
Symposium Of University Research and Creative Expression (SOURCE)
Building software that can visually and accurately perceive gender from face images is an important step in making more intelligent machines. Several approaches to this problem have been suggested in the literature. We evaluate Histogram of Oriented Gradients, Dual Tree Complex Wavelet Transform (DTCWT) Principal Component Analysis (PCA) with Support Vector Machines (SVM) and compare them to Convolutional Neural Networks for this task. We train and test our classifiers with two benchmarks containing thousands of facial images. As expected, convolutional neural networks had the best performance while the performance of DTCWT varied most depending on the dataset used