Computer Sciences Commons^™

Parameterizing And Aggregating Activation Functions In Deep Neural Networks, Luke Benjamin Godfrey

Graduate Theses and Dissertations

The nonlinear activation functions applied by each neuron in a neural network are essential for making neural networks powerful representational models. If these are omitted, even deep neural networks reduce to simple linear regression due to the fact that a linear combination of linear combinations is still a linear combination. In much of the existing literature on neural networks, just one or two activation functions are selected for the entire network, even though the use of heterogenous activation functions has been shown to produce superior results in some cases. Even less often employed are activation functions that can adapt their …

Deep Code Comment Generation, Xing Hu, Ge Li, Xin Xia, David Lo, Zhi Jin

Research Collection School Of Computing and Information Systems

During software maintenance, code comments help developerscomprehend programs and reduce additional time spent on readingand navigating source code. Unfortunately, these comments areoften mismatched, missing or outdated in the software projects.Developers have to infer the functionality from the source code.This paper proposes a new approach named DeepCom to automatically generate code comments for Java methods. The generatedcomments aim to help developers understand the functionalityof Java methods. DeepCom applies Natural Language Processing(NLP) techniques to learn from a large code corpus and generatescomments from learned features. We use a deep neural networkthat analyzes structural information of Java methods for bettercomments generation. We conduct …

A Qualitative Research On Marketing And Sales In The Artificial Intelligence Age, Yin Yang, Keng Siau

Research Collection School Of Computing and Information Systems

The age of artificial intelligence is here! Artificial Intelligence, robotics, machine learning, and automation are impacting the field of marketing and sales in an unprecedented way. In this study, the qualitative research methodology will be used to better understand the revolution and evolution of marketing and sales field in the AI age. Multiple case studies will be performed in various marketing and sales units in different organizations. This research is of value to both academics and practitioners as it aims to provide a detailed analysis and documentation of the changes in marketing and sales functionalities and job markets as AI …

Comparative Study Of Deep Learning Models For Network Intrusion Detection, Brian Lee, Sandhya Amaresh, Clifford Green, Daniel Engels

SMU Data Science Review

In this paper, we present a comparative evaluation of deep learning approaches to network intrusion detection. A Network Intrusion Detection System (NIDS) is a critical component of every Internet connected system due to likely attacks from both external and internal sources. A NIDS is used to detect network born attacks such as Denial of Service (DoS) attacks, malware replication, and intruders that are operating within the system. Multiple deep learning approaches have been proposed for intrusion detection systems. We evaluate three models, a vanilla deep neural net (DNN), self-taught learning (STL) approach, and Recurrent Neural Network (RNN) based Long Short …

Speech Emotion Recognition Using Convolutional Neural Networks, Somayeh Shahsavarani

School of Computing: Dissertations, Theses, and Student Research

Automatic speech recognition is an active field of study in artificial intelligence and machine learning whose aim is to generate machines that communicate with people via speech. Speech is an information-rich signal that contains paralinguistic information as well as linguistic information. Emotion is one key instance of paralinguistic information that is, in part, conveyed by speech. Developing machines that understand paralinguistic information, such as emotion, facilitates the human-machine communication as it makes the communication more clear and natural. In the current study, the efficacy of convolutional neural networks in recognition of speech emotions has been investigated. Wide-band spectrograms of the …

Attributed Social Network Embedding, Lizi Liao, Xiangnan He, Hanwang Zhang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Embedding network data into a low-dimensional vector space has shown promising performance for many real-world applications, such as node classification and entity retrieval. However, most existing methods focused only on leveraging network structure. For social networks, besides the network structure, there also exists rich information about social actors, such as user profiles of friendship networks and textual content of citation networks. These rich attribute information of social actors reveal the homophily effect, exerting huge impacts on the formation of social networks. In this paper, we explore the rich evidence source of attributes in social networks to improve network embedding. We …

Patent Keyword Extraction Algorithm Based On Distributed Representation For Patent Classification, Jie Hu, Shaobo Li, Yong Yao, Liya Yu, Guanci Yang, Jianjun Hu

Faculty Publications

Many text mining tasks such as text retrieval, text summarization, and text comparisons depend on the extraction of representative keywords from the main text. Most existing keyword extraction algorithms are based on discrete bag-of-words type of word representation of the text. In this paper, we propose a patent keyword extraction algorithm (PKEA) based on the distributed Skip-gram model for patent classification. We also develop a set of quantitative performance measures for keyword extraction evaluation based on information gain and cross-validation, based on Support Vector Machine (SVM) classification, which are valuable when human-annotated keywords are not available. We used a standard …

Object Localization, Segmentation, And Classification In 3d Images, Allan Zelener

Dissertations, Theses, and Capstone Projects

We address the problem of identifying objects of interest in 3D images as a set of related tasks involving localization of objects within a scene, segmentation of observed object instances from other scene elements, classifying detected objects into semantic categories, and estimating the 3D pose of detected objects within the scene. The increasing availability of 3D sensors motivates us to leverage large amounts of 3D data to train machine learning models to address these tasks in 3D images. Leveraging recent advances in deep learning has allowed us to develop models capable of addressing these tasks and optimizing these tasks jointly …

Recurrent Neural Networks And Their Applications To Rna Secondary Structure Inference, Devin Willmott

Theses and Dissertations--Mathematics

Recurrent neural networks (RNNs) are state of the art sequential machine learning tools, but have difficulty learning sequences with long-range dependencies due to the exponential growth or decay of gradients backpropagated through the RNN. Some methods overcome this problem by modifying the standard RNN architecure to force the recurrent weight matrix W to remain orthogonal throughout training. The first half of this thesis presents a novel orthogonal RNN architecture that enforces orthogonality of W by parametrizing with a skew-symmetric matrix via the Cayley transform. We present rules for backpropagation through the Cayley transform, show how to deal with the Cayley …

Automated Tree-Level Forest Quantification Using Airborne Lidar, Hamid Hamraz

Theses and Dissertations--Computer Science

Traditional forest management relies on a small field sample and interpretation of aerial photography that not only are costly to execute but also yield inaccurate estimates of the entire forest in question. Airborne light detection and ranging (LiDAR) is a remote sensing technology that records point clouds representing the 3D structure of a forest canopy and the terrain underneath. We present a method for segmenting individual trees from the LiDAR point clouds without making prior assumptions about tree crown shapes and sizes. We then present a method that vertically stratifies the point cloud to an overstory and multiple understory tree …

Smart Augmented Reality Instructional System For Mechanical Assembly, Ze-Hao Lai

Masters Theses

"Quality and efficiency are pivotal indicators of a manufacturing company. Many companies are suffering from shortage of experienced workers across the production line to perform complex assembly tasks such as assembly of an aircraft engine. This could lead to a significant financial loss. In order to further reduce time and error in an assembly, a smart system consisting of multi-modal Augmented Reality (AR) instructions with the support of a deep learning network for tool detection is introduced. The multi-modal smart AR is designed to provide on-site information including various visual renderings with a fine-tuned Region-based Convolutional Neural Network, which is …

Deep Recurrent Learning For Efficient Image Recognition Using Small Data, Mahbubul Alam

Electrical & Computer Engineering Theses & Dissertations

Recognition is fundamental yet open and challenging problem in computer vision. Recognition involves the detection and interpretation of complex shapes of objects or persons from previous encounters or knowledge. Biological systems are considered as the most powerful, robust and generalized recognition models. The recent success of learning based mathematical models known as artificial neural networks, especially deep neural networks, have propelled researchers to utilize such architectures for developing bio-inspired computational recognition models. However, the computational complexity of these models increases proportionally to the challenges posed by the recognition problem, and more importantly, these models require a large amount of data …

Deep Learning Methods For Visual Object Recognition, Zeyad Hailat

Wayne State University Dissertations

Convolutional neural networks (CNNs) attain state-of-the-art performance on various classification tasks assuming a sufficiently large number of labeled training examples. Unfortunately, curating sufficiently large labeled training dataset requires human involvement, which is expensive, time-consuming, and susceptible to noisy labels. Semi-supervised learning methods can alleviate the aforementioned problems by employing one of two techniques. First, utilizing a limited number of labeled data in conjunction with sufficiently large unlabeled data to construct a classification model. Second, exploiting sufficiently large noisy label training data to learn a classification model. In this dissertation, we proposed a few new methods to mitigate the aforementioned problems. …

Representation Learning With Convolutional Neural Networks, Haotian Xu

Wayne State University Dissertations

Deep learning methods have achieved great success in the areas of Computer Vision and Natural Language Processing. Recently, the rapidly developing field of deep learning is concerned with questions surrounding how we can learn meaningful and effective representations of data. This is because the performance of machine learning approaches is heavily dependent on the choice and quality of data representation, and different kinds of representation entangle and hide the different explanatory factors of variation behind the data.

In this dissertation, we focus on representation learning with deep neural networks for different data formats including text, 3D polygon shapes, and brain …

A Comparison Of Classical Versus Deep Learning Techniques For Abusive Content Detection On Social Media Sites, Hao Che, Susan Mckeever, Sarah Jane Delany

Conference papers

The automated detection of abusive content on social media websites faces a variety of challenges including imbalanced training sets, the identification of an appropriate feature representation and the selection of optimal classifiers. Classifiers such as support vector machines (SVM), combined with bag of words or ngram feature representation, have traditionally dominated in text classification for decades. With the recent emergence of deep learning and word embeddings, an increasing number of researchers have started to focus on deep neural networks. In this paper, our aim is to explore cutting-edge techniques in automated abusive content detection. We use two deep learning approaches: …

Deep Convolutional Neural Networks Enable Discrimination Of Heterogeneous Digital Pathology Images, Pegah Khosravi, Ehsan Kazemi, Marcin Imielinski, Olivier Elemento, Iman Hajirasouliha

Publications and Research

Pathological evaluation of tumor tissue is pivotal for diagnosis in cancer patients and automated image analysis approaches have great potential to increase precision of diagnosis and help reduce human error.

In this study, we utilize several computational methods based on convolutional neural networks (CNN) and build a stand-alone pipeline to effectively classify different histopathology images across different types of cancer.

In particular, we demonstrate the utility of our pipeline to discriminate between two subtypes of lung cancer, four biomarkers of bladder cancer, and five biomarkers of breast cancer. In addition, we apply our pipeline to discriminate among four immunohistochemistry …

Review Of Deep Learning Methods In Robotic Grasp Detection, Shehan Caldera, Alexander Rassau, Douglas Chai

Research outputs 2014 to 2021

For robots to attain more general-purpose utility, grasping is a necessary skill to master. Such general-purpose robots may use their perception abilities to visually identify grasps for a given object. A grasp describes how a robotic end-effector can be arranged to securely grab an object and successfully lift it without slippage. Traditionally, grasp detection requires expert human knowledge to analytically form the task-specific algorithm, but this is an arduous and time-consuming approach. During the last five years, deep learning methods have enabled significant advancements in robotic vision, natural language processing, and automated driving applications. The successful results of these methods …

Estimating Meteorological Visibility Range Under Foggy Weather Conditions: A Deep Learning Approach, Hazar Chaabani, Naoufel Werghi, Faouzi Kamoun, Bilal Taha, Fatma Outay, Ansar Ul Haque Yasar

All Works

© 2018 The Authors. Published by Elsevier Ltd. Systems capable of estimating visibility distances under foggy weather conditions are extremely useful for next-generation cooperative situational awareness and collision avoidance systems. In this paper, we present a brief review of noticeable approaches for determining visibility distance under foggy weather conditions. We then propose a novel approach based on the combination of a deep learning method for feature extraction and an SVM classifier. We present a quantitative evaluation of the proposed solution and show that our approach provides better performance results compared to an earlier approach that was based on the combination …

Estimating Left Ventricular Volume With Roi-Based Convolutional Neural Network, Feng Zhu

Turkish Journal of Electrical Engineering and Computer Sciences

The volume of the human left ventricular (LV) chamber is an important indicator for diagnosing heart disease. Although LV volume can be measured manually with cardiac magnetic resonance imaging (MRI), the process is difficult and time-consuming for experienced cardiologists. This paper presents an end-to-end segmentation-free method that estimates LV volume from MRI images directly. The method initially uses Fourier transform and a regression filter to calculate the region of interest that contains the LV chambers. Then convolutional neural networks are trained to estimate the end-diastolic volume (EDV) and end-systolic volume (ESV). The resulting models accurately estimate the EDV and ESV …

Deep Learning Based Brain Tumor Classification And Detection System, Ali̇ Ari, Davut Hanbay

Turkish Journal of Electrical Engineering and Computer Sciences

The brain cancer treatment process depends on the physician's experience and knowledge. For this reason, using an automated tumor detection system is extremely important to aid radiologists and physicians to detect brain tumors. The proposed method has three stages, which are preprocessing, the extreme learning machine local receptive fields (ELM-LRF) based tumor classification, and image processing based tumor region extraction. At first, nonlocal means and local smoothing methods were used to remove possible noises. In the second stage, cranial magnetic resonance (MR) images were classified as benign or malignant by using ELM-LRF. In the third stage, the tumors were segmented. …

Visual Odometry Using Convolutional Neural Networks, Alec Graves, Steffen Lim, Thomas Fagan, Kevin Mcfall Phd.

The Kennesaw Journal of Undergraduate Research

Visual odometry is the process of tracking an agent's motion over time using a visual sensor. The visual odometry problem has only been recently solved using traditional, non-machine learning techniques. Despite the success of neural networks at many related problems such as object recognition, feature detection, and optical flow, visual odometry still has not been solved with a deep learning technique. This paper attempts to implement several Convolutional Neural Networks to solve the visual odometry problem and compare slight variations in data preprocessing. The work presented is a step toward reaching a legitimate neural network solution.

How To Best Apply Neural Networks In Geosciences: Towards Optimal "Averaging" In Dropout Training, Afshin Gholamy, Justin Parra, Vladik Kreinovich, Olac Fuentes, Elizabeth Y. Anthony

Departmental Technical Reports (CS)

The main objectives of geosciences is to find the current state of the Earth -- i.e., solve the corresponding inverse problems -- and to use this knowledge for predicting the future events, such as earthquakes and volcanic eruptions. In both inverse and prediction problems, often, machine learning techniques are very efficient, and at present, the most efficient machine learning technique is deep neural training. To speed up this training, the current learning algorithms use dropout techniques: they train several sub-networks on different portions of data, and then "average" the results. A natural idea is to use arithmetic mean for this …

A Deep Learníng-Based Data Minimization Algorithm For Big Genomics Data In Support Of Lot And Secure Smart Health Services, Mohammed Aledhari

Dissertations

In the age of Big Genomics Data, institutes such as the National Human Genome Research Institute (NHGRI),1000-Genomes project, and the international cancer sequencing consortium are faced with the challenge of sharing large volumes of data between internationallydispersed sample collectors, data analyzers, and researchers, a process that up until now has been plagued by unreliable transfers and slow connection speeds. These occur due to the inherent throughput bottlenecks of traditional transfer technologies. One suggested solution is using the cloud as an infrastructure to solve the store and analysis challenges. However, the transfer and share of the genomics datasets between biological laboratories …

Online Learning With Nonlinear Models, Doyen Sahoo

Dissertations and Theses Collection (Open Access)

Recent years have witnessed the success of two broad categories of machine learning algorithms: (i) Online Learning; and (ii) Learning with nonlinear models. Typical machine learning algorithms assume that the entire data is available prior to the training task. This is often not the case in the real world, where data often arrives sequentially in a stream, or is too large to be stored in memory. To address these challenges, Online Learning techniques evolved as a promising solution to having highly scalable and efficient learning methodologies which could learn from data arriving sequentially. Next, as the real world data exhibited …

Intent Recognition In Smart Living Through Deep Recurrent Neural Networks, Xiang Zhang, Lina Yao, Chaoran Huang, Quan Z. Sheng, Xianzhi Wang

Research Collection School Of Computing and Information Systems

Electroencephalography (EEG) signal based intent recognition has recently attracted much attention in both academia and industries, due to helping the elderly or motor-disabled people controlling smart devices to communicate with outer world. However, the utilization of EEG signals is challenged by low accuracy, arduous and time-consuming feature extraction. This paper proposes a 7-layer deep learning model to classify raw EEG signals with the aim of recognizing subjects’ intents, to avoid the time consumed in pre-processing and feature extraction. The hyper-parameters are selected by an Orthogonal Array experiment method for efficiency. Our model is applied to an open EEG dataset provided …

Hierarchical Fusion Based Deep Learning Framework For Lung Nodule Classification, Kazim Sekeroglu

LSU Doctoral Dissertations

Lung cancer is the leading cancer type that causes the mortality in both men and women. Computer aided detection (CAD) and diagnosis systems can play a very important role for helping the physicians in cancer treatments. This dissertation proposes a CAD framework that utilizes a hierarchical fusion based deep learning model for detection of nodules from the stacks of 2D images. In the proposed hierarchical approach, a decision is made at each level individually employing the decisions from the previous level. Further, individual decisions are computed for several perspectives of a volume of interest (VOI). This study explores three different …

Exploring The Internal Statistics: Single Image Super-Resolution, Completion And Captioning, Yang Xian

Dissertations, Theses, and Capstone Projects

Image enhancement has drawn increasingly attention in improving image quality or interpretability. It aims to modify images to achieve a better perception for human visual system or a more suitable representation for further analysis in a variety of applications such as medical imaging, remote sensing, and video surveillance. Based on different attributes of the given input images, enhancement tasks vary, e.g., noise removal, deblurring, resolution enhancement, prediction of missing pixels, etc. The latter two are usually referred to as image super-resolution and image inpainting (or completion).

Image super-resolution and completion are numerically ill-posed problems. Multi-frame-based approaches make use of the …

Automated Breast Cancer Diagnosis Using Deep Learning And Region Of Interest Detection (Bc-Droid), Richard Platania, Jian Zhang, Shayan Shams, Kisung Lee, Seungwon Yang, Seung Jong Park

Computer Science Faculty Research & Creative Works

Detection of suspicious regions in mammogram images and the subsequent diagnosis of these regions remains a challenging problem in the medical world. There still exists an alarming rate of misdiagnosis of breast cancer. This results in both over treatment through incorrect positive diagnosis of cancer and under treatment through overlooked cancerous masses. Convolutional neural networks have shown strong applicability to various image datasets, enabling detailed features to be learned from the data and, as a result, the ability to classify these images at extremely low error rates. In order to overcome the difficulty in diagnosing breast cancer from mammogram images, …

Hybrid Recommender Systems With Deep Learning, Fei Li

LSU Master's Theses

As one of the most popular recommendation algorithms, collaborative filtering (CF) suggests items favored by like-minded based on user ratings. However, CF performs worse for users and items with fewer ratings, which is known as the cold-start problem. On the other hand, the auxiliary information of items such as images and reviews can be helpful for relieving the cold-start issue and improving recommendation accuracy. How to effectively extract features from heterogeneous auxiliary information and integrate them with collaborative filtering remains a big challenge. In this thesis, we propose a tightly-coupled hybrid recommender system named Fusion-MF-Mix via a deep fusion framework, …

Deep Learning Based Multi-Label Classification For Surgical Tool Presence Detection In Laparoscopic Videos, Ashwin Raju

Computer Science and Engineering Theses - Archive

Laparoscopic surgery, Modern surgery, where the surgery is performed far away from the patient by inserting small incisions on the patient's body and the surgery is performed with a help of a video recorder and through which the doctor performs the surgery. The computer assisted intervention are increasing exponentially and the need for accurate and reliable intervention is very important because of the domain which is very critical. Efforts have made to develop a system that is both fast and accurate approach but it is still an active area of research due its importance. Some applications which involve identifying the …