Open Access. Powered by Scholars. Published by Universities.®

Physical Sciences and Mathematics Commons™

Open Access. Powered by Scholars. Published by Universities.®

Computer Sciences

PDF

2022

Deep Learning

Institution
Publication
Publication Type

Articles 1 - 30 of 57

Full-Text Articles in Physical Sciences and Mathematics

Algorithmic Improvements In Deep Reinforcement Learning, Norman L. Tasfi Dec 2022

Algorithmic Improvements In Deep Reinforcement Learning, Norman L. Tasfi

Electronic Thesis and Dissertation Repository

Reinforcement Learning (RL) has seen exponential performance improvements over the past decade, achieving super-human performance across many domains. Deep Reinforcement Learning (DRL), the combination of RL methods with deep neural networks (DNN) as function approximators, has unlocked much of this progress. The path to generalized artificial intelligence (GAI) will depend on deep learning (DL) and RL. However, much work is required before the technology reaches anything resembling GAI. Therefore, this thesis focuses on a subset of areas within RL that require additional research to advance the field, specifically: sample efficiency, planning, and task transfer. The first area, sample efficiency, refers …

Go to article

Glacier Segmentation From Remote Sensing Imagery Using Deep Learning, Bibek Aryal Dec 2022

Glacier Segmentation From Remote Sensing Imagery Using Deep Learning, Bibek Aryal

Open Access Theses & Dissertations

Large-scale study of glaciers improves our understanding of global glacier change and is imperative for monitoring the ecological environment, preventing disasters, and studying the effects of global climate change. In recent years, remote sensing imagery has been preferred over riskier and resource-intensive field visits for tracking landscape level changes like glaciers. However, periodic manual labeling of glaciers over a large area is not feasible due to the considerable amount of time it requires while automatic segmentation of glaciers has its own set of challenges. Our work aims to study the challenges associated with segmentation of glaciers from remote sensing imagery …

Go to article

Attention In The Faithful Self-Explanatory Nlp Models, Mostafa Rafaiejokandan Dec 2022

Attention In The Faithful Self-Explanatory Nlp Models, Mostafa Rafaiejokandan

Department of Computer Science and Engineering: Dissertations, Theses, and Student Research

Deep neural networks (DNNs) can perform impressively in many natural language processing (NLP) tasks, but their black-box nature makes them inherently challenging to explain or interpret. Self-Explanatory models are a new approach to overcoming this challenge, generating explanations in human-readable languages besides task objectives like answering questions. The main focus of this thesis is the explainability of NLP tasks, as well as how attention methods can help enhance performance. Three different attention modules are proposed, SimpleAttention, CrossSelfAttention, and CrossModality. It also includes a new dataset transformation method called Two-Documents that converts every dataset into two separate documents required by the …

Go to article

Bevers: A General, Simple, And Performant Framework For Automatic Fact Verification, Mitchell Dehaven Dec 2022

Bevers: A General, Simple, And Performant Framework For Automatic Fact Verification, Mitchell Dehaven

Department of Computer Science and Engineering: Dissertations, Theses, and Student Research

Fact verification has become an important process, primarily done manually by humans, to verify the authenticity of claims and statements made online. Increasingly, social media companies have utilized human effort to debunk false claims on their platforms, opting to either tag the content as misleading or false, or removing it entirely to combat misinformation on their sites. In tandem, the field of automatic fact verification has become a subject of focus among the natural language processing (NLP) community, spawning new datasets and research. The most popular dataset is the Fact Extraction and VERification (FEVER) dataset. In this thesis an end-to-end …

Go to article

Image-Based Cancer Diagnosis Using Novel Deep Neural Networks, Hosein Barzekar Dec 2022

Image-Based Cancer Diagnosis Using Novel Deep Neural Networks, Hosein Barzekar

Theses and Dissertations

Cancer is the major cause of death in many nations. This serious illness can only be effectivelytreated if it is diagnosed early. In contrast, biomedical imaging presents challenges to both clinical institutions and researchers. Physiological anomalies are often characterized by modest modifications in individual cells or tissues, making them difficult to detect visually. Physiological anomalies are often characterized by slight abnormalities in individual cells or tissues, making them difficult to detect visually. Traditionally, anomalies are diagnosed by radiologists and pathologists with extensive training. This procedure, however, demands the participation of professionals and incurs a substantial expense, making the classification of …

Go to article

Wildfire Spread Prediction Using Attention Mechanisms In U-Net, Kamen Haresh Shah, Kamen Haresh Shah Dec 2022

Wildfire Spread Prediction Using Attention Mechanisms In U-Net, Kamen Haresh Shah, Kamen Haresh Shah

Master's Theses

An investigation into using attention mechanisms for better feature extraction in wildfire spread prediction models. This research examines the U-net architecture to achieve image segmentation, a process that partitions images by classifying pixels into one of two classes. The deep learning models explored in this research integrate modern deep learning architectures, and techniques used to optimize them. The models are trained on 12 distinct observational variables derived from the Google Earth Engine catalog. Evaluation is conducted with accuracy, Dice coefficient score, ROC-AUC, and F1-score. This research concludes that when augmenting U-net with attention mechanisms, the attention component improves feature suppression …

Go to article

Generative Spatio-Temporal And Multimodal Analysis Of Neonatal Pain, Md Sirajus Salekin Nov 2022

Generative Spatio-Temporal And Multimodal Analysis Of Neonatal Pain, Md Sirajus Salekin

USF Tampa Graduate Theses and Dissertations

Neonates can not express their pain like an adult person. Due to the lacking of proper muscle growth and inability to express non-verbally, it is difficult to understand their emotional status. In addition, if the neonates are under any treatment or left monitored after any major surgeries (post-operative), it is more difficult to understand their pain due to the side effect of medications and the caring system (i.e. intubated, masked face, covered body with blanket, etc.). In a clinical environment, usually, bedside nurses routinely observe the neonate and measure the pain status following any standard clinical pain scale. But current …

Go to article

Computer Vision Based Analysis Of Crowd Behavior For Efficient Video Surveillance In Public Places, Abhilash K. Pai Nov 2022

Computer Vision Based Analysis Of Crowd Behavior For Efficient Video Surveillance In Public Places, Abhilash K. Pai

Technical Collection

Automated crowd behavior analysis and monitoring is a challenging task due to the unpredictable nature of the crowd, making it an open problem. Within this domain, we are focusing on two research problems which deal with low to mid-level analysis of crowd videos. The first problem focuses on how to effectively utilize the readily available compressed motion vector information in an input crowd video to model the motion of the crowd and perform crowd behavior analysis in varying densities of crowds. For this purpose, we consider a mid-level crowd scene analysis task called as motion pattern segmentation. The second research …

Go to article

Overview Of The Clpsych 2022 Shared Task: Capturing Moments Of Change In Longitudinal User Posts, Adam Tsakalidis, Jenny Chim, Iman Munire Bilal, Ayah Zirikly, Dana Atzil-Slonim, Federico Nanni, Philip Resnik, Manas Gaur, Kaushik Roy, Becky Inkster, Jeff Leintz, Maria Liakata Oct 2022

Overview Of The Clpsych 2022 Shared Task: Capturing Moments Of Change In Longitudinal User Posts, Adam Tsakalidis, Jenny Chim, Iman Munire Bilal, Ayah Zirikly, Dana Atzil-Slonim, Federico Nanni, Philip Resnik, Manas Gaur, Kaushik Roy, Becky Inkster, Jeff Leintz, Maria Liakata

Publications

We provide an overview of the CLPsych 2022 Shared Task, which focusses on the automatic identification of Moments of Change in longitudinal posts by individuals on social media and its connection with information regarding mental health . This year's task introduced the notion of longitudinal modelling of the text generated by an individual online over time, along with appropriate temporally sensitive evaluation metrics. The Shared Task consisted of two subtasks: (a) the main task of capturing changes in an individual's mood (drastic changes-`Switches'- and gradual changes -`Escalations'- on the basis of textual content shared online; and subsequently (b) the sub-task …

Go to article

Towards Understanding The Faults Of Javascript-Based Deep Learning Systems, Lili Quan, Qianyu Guo, Xiaofei Xie, Sen Chen, Xiaohong Li, Yang Liu Oct 2022

Towards Understanding The Faults Of Javascript-Based Deep Learning Systems, Lili Quan, Qianyu Guo, Xiaofei Xie, Sen Chen, Xiaohong Li, Yang Liu

Research Collection School Of Computing and Information Systems

Quality assurance is of great importance for deep learning (DL) systems, especially when they are applied in safety-critical applications. While quality issues of native DL applications have been extensively analyzed, the issues of JavaScript-based DL applications have never been systematically studied. Compared with native DL applications, JavaScript-based DL applications can run on major browsers, making the platform- and device-independent. Specifically, the quality of JavaScript-based DL applications depends on the 3 parts: the application, the third-party DL library used and the underlying DL framework (e.g., TensorFlow.js), called JavaScript-based DL system. In this paper, we conduct the first empirical study on the …

Go to article

Cov-Inception: Covid-19 Detection Tool Using Chest X-Ray, Aswini Thota, Ololade Awodipe, Rashmi Patel Sep 2022

Cov-Inception: Covid-19 Detection Tool Using Chest X-Ray, Aswini Thota, Ololade Awodipe, Rashmi Patel

SMU Data Science Review

Since the pandemic started, researchers have been trying to find a way to detect COVID-19 which is a cost-effective, fast, and reliable way to keep the economy viable and running. This research details how chest X-ray radiography can be utilized to detect the infection. This can be for implementation in Airports, Schools, and places of business. Currently, Chest imaging is not a first-line test for COVID-19 due to low diagnostic accuracy and confounding with other viral pneumonia. Different pre-trained algorithms were fine-tuned and applied to the images to train the model and the best model obtained was fine-tuned InceptionV3 model …

Go to article

Deep Learning For Coverage-Guided Fuzzing: How Far Are We?, Siqi Li, Xiaofei Xie, Yun Lin, Yuekang Li, Ruitao Feng, Xiaohong Li, Weimin Ge, Jin Song Dong Sep 2022

Deep Learning For Coverage-Guided Fuzzing: How Far Are We?, Siqi Li, Xiaofei Xie, Yun Lin, Yuekang Li, Ruitao Feng, Xiaohong Li, Weimin Ge, Jin Song Dong

Research Collection School Of Computing and Information Systems

Fuzzing is a widely-used software vulnerability discovery technology, many of which are optimized using coverage-feedback. Recently, some techniques propose to train deep learning (DL) models to predict the branch coverage of an arbitrary input owing to its always-available gradients etc. as a guide. Those techniques have proved their success in improving coverage and discovering bugs under different experimental settings. However, DL models, usually as a magic black-box, are notoriously lack of explanation. Moreover, their performance can be sensitive to the collected runtime coverage information for training, indicating potentially unstable performance. In this work, we conduct a systematic empirical study on …

Go to article

Sel-Covidnet: An Intelligent Application For The Diagnosis Of Covid-19 From Chest X-Rays And Ct-Scans, Ahmad Al Smadi, Ahed Abugabah, Ahmad Mohammad Al-Smadi, Sultan Almotairi Aug 2022

Sel-Covidnet: An Intelligent Application For The Diagnosis Of Covid-19 From Chest X-Rays And Ct-Scans, Ahmad Al Smadi, Ahed Abugabah, Ahmad Mohammad Al-Smadi, Sultan Almotairi

All Works

COVID-19 detection from medical imaging is a difficult challenge that has piqued the interest of experts worldwide. Chest X-rays and computed tomography (CT) scanning are the essential imaging modalities for diagnosing COVID-19. All researchers focus their efforts on developing viable methods and rapid treatment procedures for this pandemic. Fast and accurate automated detection approaches have been devised to alleviate the need for medical professionals. Deep Learning (DL) technologies have successfully recognized COVID-19 situations. This paper proposes a developed set of nine deep learning models for diagnosing COVID-19 based on transfer learning and implementation in a novel architecture (SEL-COVIDNET). In which …

Go to article

Improving Deep Entity Resolution By Constraints, Soudeh Nilforoushan Aug 2022

Improving Deep Entity Resolution By Constraints, Soudeh Nilforoushan

Electronic Thesis and Dissertation Repository

Entity resolutions the problem of finding duplicate data in a dataset and resolving possible differences and inconsistencies. ER is a long-standing data management and information retrieval problem and a core data integration and cleaning task. There are diverse solutions for ER that apply rule-based techniques, pairwise binary classification, clustering, and probabilistic inference, among other techniques. Deep learning (DL) has been extensively used for ER and has shown competitive performance compared to conventional ER solutions. The state-of-the-art (SOTA) ER solutions using DL are based on pairwise comparison and binary classification. They transform pairs of records into a latent space that can …

Go to article

Skin Lesion Segmentation Based On U-Shaped Network, Muna Khalaf, Ban N. Dhannoon Aug 2022

Skin Lesion Segmentation Based On U-Shaped Network, Muna Khalaf, Ban N. Dhannoon

Karbala International Journal of Modern Science

Skin lesion segmentation is an essential step toward accurate skin lesion diagnosis. The need to automate Skin lesion segmentation on the one hand, and the challenges it faces, on the other hand, have made it a growing area of research and focus. Automation of skin lesion segmentation helps reduce the effort and time needed for diagnosis and treatment and helps make better utilization of available data and shared experiences. The challenges faced by the automation of skin lesion segmentation can be broadly defined by (but not limited to); variations in texture, shape, and size for skin lesions and the low …

Go to article

Novel Deep Neural Network For Medical Image Classification, Dm Anisuzzaman Aug 2022

Novel Deep Neural Network For Medical Image Classification, Dm Anisuzzaman

Theses and Dissertations

Medical image classification is an essential part of diagnosis, which with automation may benefit both physicians and patients in terms of time and cost. For automation, different Artificial intelligence (AI) methods, including Machine Learning (ML) and Deep Learning (DL), are used widely. Specifically, DL algorithms have become popular in classifying medical images due to their propensity for good performance. This thesis studies medical image classification problems using deep learning models. Four specific medical applications are considered: (1) Osteosarcoma cancer classification in histological images, (2) Burn wound classification, (3) Wound severity classification from clinical images, and (4) Wound type classification using …

Go to article

Deep Active Genetic Learning With Evidential Uncertainty For Agriculture Crops And Lake Water Quality Assessment, Oguz M. Aranay Aug 2022

Deep Active Genetic Learning With Evidential Uncertainty For Agriculture Crops And Lake Water Quality Assessment, Oguz M. Aranay

Legacy Theses & Dissertations (2009 - 2024)

Despite significant advancements in the field of machine learning, there are two issues that still require further exploration. First, how to learn from a small dataset; and second, how to select appropriate features from the data. Although there exist many techniques to address these issues, choosing a combination of the techniques from these two groups is challenging, and worth investigating. To address these concerns, this thesis presents a learning framework that is based on a deep learning model utilizing active learning (with evidential uncertainty as a basis for acquisition function) for the first issue and a genetic algorithm for the …

Go to article

Holistic Combination Of Structural And Textual Code Information For Context Based Api Recommendation, Chi Chen, Xin Peng, Zhengchang Xing, Jun Sun, Xin Wang, Yifan Zhao, Wenyun Zhao Aug 2022

Holistic Combination Of Structural And Textual Code Information For Context Based Api Recommendation, Chi Chen, Xin Peng, Zhengchang Xing, Jun Sun, Xin Wang, Yifan Zhao, Wenyun Zhao

Research Collection School Of Computing and Information Systems

Context based API recommendation is an important way to help developers find the needed APIs effectively and efficiently. For effective API recommendation, we need not only a joint view of both structural and textual code information, but also a holistic view of correlated API usage in control and data flow graph as a whole. Unfortunately, existing API recommendation methods exploit structural or textual code information separately. In this work, we propose a novel API recommendation approach called APIRec-CST (API Recommendation by Combining Structural and Textual code information). APIRec-CST is a deep learning model that combines the API usage with the …

Go to article

Applied Deep Learning: Case Studies In Computer Vision And Natural Language Processing, Md Reshad Ul Hoque Aug 2022

Applied Deep Learning: Case Studies In Computer Vision And Natural Language Processing, Md Reshad Ul Hoque

Electrical & Computer Engineering Theses & Dissertations

Deep learning has proved to be successful for many computer vision and natural language processing applications. In this dissertation, three studies have been conducted to show the efficacy of deep learning models for computer vision and natural language processing. In the first study, an efficient deep learning model was proposed for seagrass scar detection in multispectral images which produced robust, accurate scars mappings. In the second study, an arithmetic deep learning model was developed to fuse multi-spectral images collected at different times with different resolutions to generate high-resolution images for downstream tasks including change detection, object detection, and land cover …

Go to article

Textual Emotion Detection Approaches: A Survey, Mahinda Mahmoud Samy Zidan, Ibrahim Elhenawy, Ahmed R. Abas, Mahmoud Othman Jul 2022

Textual Emotion Detection Approaches: A Survey, Mahinda Mahmoud Samy Zidan, Ibrahim Elhenawy, Ahmed R. Abas, Mahmoud Othman

Future Computing and Informatics Journal

Over the past decades, social media attracted individuals to express their feelings on any topic or item, resulting in an incremental growth in the size of created data. These feelings and unstructured data paved the path for business organizations to gather information and build statistical analysis. Various machine learning and natural language processing-based approaches are used for sentiment and emotion analysis. Moreover, deep learning-based approaches recently gained popularity due to their remarkable performance in text analysis. This paper provides a comprehensive overview of the prominent machine learning models applied in emotion analysis. It explores various emotion analysis taxonomies, in addition …

Go to article

Cross-Lingual Transfer Learning For Statistical Type Inference, Zhiming Li, Xiaofei Xie, Haoliang Li, Zhengzi Xu, Yi Li, Yang Liu Jul 2022

Cross-Lingual Transfer Learning For Statistical Type Inference, Zhiming Li, Xiaofei Xie, Haoliang Li, Zhengzi Xu, Yi Li, Yang Liu

Research Collection School Of Computing and Information Systems

Hitherto statistical type inference systems rely thoroughly on supervised learning approaches, which require laborious manual effort to collect and label large amounts of data. Most Turing-complete imperative languages share similar control- and data-flow structures, which make it possible to transfer knowledge learned from one language to another. In this paper, we propose a cross-lingual transfer learning framework, Plato, for statistical type inference, which allows us to leverage prior knowledge learned from the labeled dataset of one language and transfer it to the others, e.g., Python to JavaScript, Java to JavaScript, etc. Plato is powered by a novel kernelized attention mechanism …

Go to article

Machine Learning With Big Data For Electrical Load Forecasting, Alexandra L'Heureux Jun 2022

Machine Learning With Big Data For Electrical Load Forecasting, Alexandra L'Heureux

Electronic Thesis and Dissertation Repository

Today, the amount of data collected is exploding at an unprecedented rate due to developments in Web technologies, social media, mobile and sensing devices and the internet of things (IoT). Data is gathered in every aspect of our lives: from financial information to smart home devices and everything in between. The driving force behind these extensive data collections is the promise of increased knowledge. Therefore, the potential of Big Data relies on our ability to extract value from these massive data sets. Machine learning is central to this quest because of its ability to learn from data and provide data-driven …

Go to article

Anomaly Detection In Sequential Data: A Deep Learning-Based Approach, Jayesh Soni Jun 2022

Anomaly Detection In Sequential Data: A Deep Learning-Based Approach, Jayesh Soni

FIU Electronic Theses and Dissertations

Anomaly Detection has been researched in various domains with several applications in intrusion detection, fraud detection, system health management, and bio-informatics. Conventional anomaly detection methods analyze each data instance independently (univariate or multivariate) and ignore the sequential characteristics of the data. Anomalies in the data can be detected by grouping the individual data instances into sequential data and hence conventional way of analyzing independent data instances cannot detect anomalies. Currently: (1) Deep learning-based algorithms are widely used for anomaly detection purposes. However, significant computational overhead time is incurred during the training process due to static constant batch size and learning …

Go to article

Machine Learning With Kay, Lasith Niroshan, James Carswell Jun 2022

Machine Learning With Kay, Lasith Niroshan, James Carswell

Conference Papers

Computational power is very important when training Deep Learning (DL) models with large amounts of data (Wooldridge, 2021). Hence, High-Performance Computing (HPC) can be leveraged to reduce computational cost, and the Irish Centre for High-End Computing (ICHEC) provides significant infrastructure and services for research and development to both academia and industry. A portion of ICHEC's HPC system has been allocated for institutional access, and this paper presents a case study of how to use Kay (Ireland's national supercomputer) in the remote sensing domain. Specifically, this study uses clusters of Kay Graphics Processing Units (GPUs) for training DL models to extract …

Go to article

An Empirical Study On Sampling Approaches For 3d Image Classification Using Deep Learning, Nicholas Michelette Jun 2022

An Empirical Study On Sampling Approaches For 3d Image Classification Using Deep Learning, Nicholas Michelette

Theses and Dissertations

A 3D classification method requires more training data than a 2D image classification method to achieve good performance. These training data usually come in the form of multiple 2D images (e.g., slices in a CT scan) or point clouds (e.g., 3D CAD modeling) for volumetric object representation. The amount of data required to complete this higher dimension problem comes with the cost of requiring more processing time and space. This problem can be mitigated with data size reduction (i.e., sampling). In this thesis, we empirically study and compare the classification performance and deep learning training time of PointNet utilizing uniform …

Go to article

Video Anomaly Detection: Practical Challenges For Learning Algorithms, Keval Doshi Jun 2022

Video Anomaly Detection: Practical Challenges For Learning Algorithms, Keval Doshi

USF Tampa Graduate Theses and Dissertations

Anomaly detection in surveillance videos is attracting an increasing amount of attention. Despite the competitive performance of several existing methods, they lack theoretical performance analysis, particularly due to the complex deep neural network architectures used in decision making. Additionally, real-time decision making is an important but mostly neglected factor in this domain. Much of the existing methods that claim to be online, depend on batch or offline processing in practice. Furthermore, several critical tasks such as continual learning, model interpretability and cross-domain adaptability are completely neglected in existing works. Motivated by these research gaps, in this dissertation we discuss our …

Go to article

Towards A Computational Model Of Narrative On Social Media, Anne Bailey Jun 2022

Towards A Computational Model Of Narrative On Social Media, Anne Bailey

Dartmouth College Undergraduate Theses

This thesis describes a variety of approaches to developing a computational model of narrative on social media. Our goal is to use such a narrative model to identify efforts to manipulate public opinion on social media platforms like Twitter. We present a model in which narratives in a collection of tweets are represented as a graph. Elements from each tweet that are relevant to potential narratives are made into nodes in the graph; for this thesis, we populate graph nodes with tweets’ authors, hashtags, named entities (people, locations, organizations, etc.,), and moral foundations (central moral values framing the discussion). Two …

Go to article

Deep Learning For Person Re-Identification: A Survey And Outlook, Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, Steven C. H. Hoi Jun 2022

Deep Learning For Person Re-Identification: A Survey And Outlook, Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Person re-identification (Re-ID) aims at retrieving a person of interest across multiple non-overlapping cameras. With the advancement of deep neural networks and increasing demand of intelligent video surveillance, it has gained significantly increased interest in the computer vision community. By dissecting the involved components in developing a person Re-ID system, we categorize it into the closed-world and open-world settings. We first conduct a comprehensive overview with in-depth analysis for closed-world person Re-ID from three different perspectives, including deep feature representation learning, deep metric learning and ranking optimization. With the performance saturation under closed-world setting, the research focus for person Re-ID …

Go to article

Simultaneous Energy Harvesting And Gait Recognition Using Piezoelectric Energy Harvester, Dong Ma, Guohao Lan, Weitao Xu, Mahbub Hassan, Wen Hu Jun 2022

Simultaneous Energy Harvesting And Gait Recognition Using Piezoelectric Energy Harvester, Dong Ma, Guohao Lan, Weitao Xu, Mahbub Hassan, Wen Hu

Research Collection School Of Computing and Information Systems

Piezoelectric energy harvester, which generates electricity from stress or vibrations, is gaining increasing attention as a viable solution to extend battery life in wearables. Recent research further reveals that, besides generating energy, PEH can also serve as a passive sensor to detect human gait power-efficiently because its stress or vibration patterns are significantly influenced by the gait. However, as PEHs are not designed for precise measurement of motion, achievable gait recognition accuracy remains low with conventional classification algorithms. The accuracy deteriorates further when the generated electricity is stored simultaneously. To classify gait reliably while simultaneously storing generated energy, we make …

Go to article

Training Thinner And Deeper Neural Networks: Jumpstart Regularization, Carles Riera, Camilo Rey, Thiago Serra, Eloi Puertas, Oriol Pujol Jun 2022

Training Thinner And Deeper Neural Networks: Jumpstart Regularization, Carles Riera, Camilo Rey, Thiago Serra, Eloi Puertas, Oriol Pujol

Faculty Conference Papers and Presentations

Neural networks are more expressive when they have multiple layers. In turn, conventional training methods are only successful if the depth does not lead to numerical issues such as exploding or vanishing gradients, which occur less frequently when the layers are sufficiently wide. However, increasing width to attain greater depth entails the use of heavier computational resources and leads to overparameterized models. These subsequent issues have been partially addressed by model compression methods such as quantization and pruning, some of which relying on normalization-based regularization of the loss function to make the effect of most parameters negligible. In this work, …

Go to article

Next Last