Digital Commons Network^™

Human Activity Recognition Using Wearable Sensors: A Deep Learning Approach, Jialun Xue

Theses

In the past decades, Human Activity Recognition (HAR) grabbed considerable research attentions from a wide range of pattern recognition and human–computer interaction researchers due to its prominent applications such as smart home health care. The wealth of information requires efficient classification and analysis methods. Deep learning represents a promising technique for large-scale data analytics. There are various ways of using different sensors for human activity recognition in a smartly controlled environment. Among them, physical human activity recognition through wearable sensors provides valuable information about an individual’s degree of functional ability and lifestyle. There is abundant research that works upon real …

Metarec: Meta-Learning Meets Recommendation Systems, James Le

Theses

Artificial neural networks (ANNs) have recently received increasing attention as powerful modeling tools to improve the performance of recommendation systems. Meta-learning, on the other hand, is a paradigm that has re-surged in popularity within the broader machine learning community over the past several years. In this thesis, we will explore the intersection of these two domains and work on developing methods for integrating meta-learning to design more accurate and flexible recommendation systems.

In the present work, we propose a meta-learning framework for the design of collaborative filtering methods in recommendation systems, drawing from ideas, models, and solutions from modern approaches …

Nature-Inspired Topology Optimization Of Recurrent Neural Networks, Abdelrahman A. Elsaid

Theses

Hand-crafting effective and efficient structures for recurrent neural networks (RNNs) is a difficult, expensive, and time-consuming process. To address this challenge, this work presents three nature-inspired (NI) algorithms for neural architecture search (NAS), introducing the subfield of nature-inspired neural architecture search (NI-NAS). These algorithms, based on ant colony optimization (ACO), progress from memory cell structure optimization, to bounded discrete-space architecture optimization, and finally to unbounded continuous-space architecture optimization. These methods were applied to real-world data sets representing challenging engineering problems, such as data from a coal-fired power plant, wind-turbine power generators, and aircraft flight data recorder (FDR) data.

Initial work …

New Methods For Deep Learning Based Real-Valued Inter-Residue Distance Prediction, Jacob Barger

Theses

Background: Much of the recent success in protein structure prediction has been a result of accurate protein contact prediction--a binary classification problem. Dozens of methods, built from various types of machine learning and deep learning algorithms, have been published over the last two decades for predicting contacts. Recently, many groups, including Google DeepMind, have demonstrated that reformulating the problem as a multi-class classification problem is a more promising direction to pursue. As an alternative approach, we recently proposed real-valued distance predictions, formulating the problem as a regression problem. The nuances of protein 3D structures make this formulation appropriate, allowing predictions …

Point Completion Networks And Segmentation Of 3d Mesh, Naga Durga Harish Kanamarlapudi

Theses

Deep learning has made many advancements in fields such as computer vision, natural language processing and speech processing. In autonomous driving, deep learning has made great improvements pertaining to the tasks of lane detection, steering estimation, throttle control, depth estimation, 2D and 3D object detection, object segmentation and object tracking. Understanding the 3D world is necessary for safe end-to-end self-driving. 3D point clouds provide rich 3D information, but processing point clouds is difficult since point clouds are irregular and unordered. Neural point processing methods like GraphCNN and PointNet operate on individual points for accurate classification and segmentation results. Occlusion of …

Query-Driven Global Graph Attention Model For Visual Parsing: Recognizing Handwritten And Typeset Math Formulas, Mahshad Mahdavi

Theses

We present a new visual parsing method based on standard Convolutional Neural Networks (CNNs) for handwritten and typeset mathematical formulas. The Query-Driven Global Graph Attention (QD-GGA) parser employs multi-task learning, using a single feature representation for locating, classifying, and relating symbols. QD-GGA parses formulas by first constructing a Line-Of-Sight (LOS) graph over the input primitives (e.g handwritten strokes or connected components in images). Second, class distributions for LOS nodes and edges are obtained using query-specific feature filters (i.e., attention) in a single feed-forward pass. This allows end-to-end structure learning using a joint loss over primitive node and edge class distributions. …

Antipodal Robotic Grasping Using Deep Learning, Shirin Joshi

Theses

In this work, we discuss two implementations that predict antipodal grasps for novel objects: A deep Q-learning approach and a Generative Residual Convolutional Neural Network approach. We present a deep reinforcement learning based method to solve the problem of robotic grasping using visio-motor feedback. The use of a deep learning based approach reduces the complexity caused by the use of hand-designed features. Our method uses an off-policy reinforcement learning framework to learn the grasping policy. We use the double deep Q-learning framework along with a novel Grasp-Q-Network to output grasp probabilities used to learn grasps that maximize the pick success. …

Anomaly Detection In Videos Through Deep Unsupervised Techniques, Parikshit Prashant Shembekar

Theses

Identifying abnormality in videos is an area of active research. Most of the work makes extensive use of supervised approaches, even though these methods often give superior performances the major drawback being abnormalities cannot be conformed to select classes, thus the need for unsupervised models to approach this task. We introduce Dirichlet Process Mixture Models (DPMM) along with Autoencoders to learn the normality in the data. Autoencoders have been extensively used in the literature for feature extraction and enable us to capture rich features into a small dimensional space. We use the Stick Breaking formulation of the DPMM which is …

A Data-Parallel Approach For Efficient Resource Utilization In Distributed Serverless Deep Learning, Kevin Tunder Elom Assogba

Theses

Serverless computing is an integral part of the recent success of cloud computing, offering cost and performance efficiency for small and large scale distributed systems. Owing to the increasing interest of developers in integrating distributed computing techniques into deep learning frameworks for better performance, serverless infrastructures have been the choice of many to host their applications. However, this computing architecture bears resource limitations which challenge the successful completion of many deep learning jobs.

In our research, an approach is presented to address timeout and memory resource limitations which are two key issues to deep learning on serverless infrastructures. Focusing on …

Gaze Estimation Based On Multi-View Geometric Neural Networks, Devarth Parikh

Theses

Gaze and head pose estimation can play essential roles in various applications, such as human attention recognition and behavior analysis. Most of the deep neural network-based gaze estimation techniques use supervised regression techniques where features are extracted from eye images by neural networks and regress 3D gaze vectors. I plan to apply the geometric features of the eyes to determine the gaze vectors of observers relying on the concepts of 3D multiple view geometry. We develop an end to-end CNN framework for gaze estimation using 3D geometric constraints under semi-supervised and unsupervised settings and compare the results. We explore the …

Self-Supervision Initialization For Semantic Segmentation Networks, Kenneth Alexopoulos

Theses

Convolutional neural networks excel at extracting features from signals. These features are able to be utilized for many downstream tasks. These tasks include object recognition, object detection, depth estimation, pixel level semantic segmentation, and more. These tasks can be used for applications such as autonomous driving where images captured by a camera can be used to give a detailed understanding of the scene. While these models are impressive, they can fail to generalize to new environments. This forces the cumbersome process of collecting images from multifarious environments and annotating them by hand. Annotating thousands or millions of images is both …

Self-Supervised Video Representation Learning By Recurrent Networks And Frame Order Prediction, Sai Shashidhar Nagabandi

Theses

The success of deep learning models in challenging tasks of computer vision and natural language processing depend on good vector representations of data. For example, learning efficient and salient video representations is one of the fundamental steps for many tasks like action recognition and next frame prediction. Most methods in deep learning rely on large datasets like ImageNet or MSCOCO for training, which is expensive and time consuming to collect. Some of the earlier works in video representation learning relied on encoder-decoder style networks in an unsupervised fashion, which would take in a few frames at a time. Research in …

Graph Networks For Multi-Label Image Recognition, Sidharth Makhija

Theses

Providing machines with a robust visualization of multiple objects in a scene has a myriad of applications in the physical world. This research solves the task of multi-label image recognition using a deep learning approach. For most multi-label image recognition datasets, there are multiple objects within a single image and a single label can be seen many times throughout the dataset. Therefore, it is not efficient to classify each object in isolation, rather it is important to infer the inter-dependencies between the labels. To extract a latent representation of the pixels from an image, this work uses a convolutional network …

Deep Learning For Quantitative Motion Tracking Based On Optical Coherence Tomography, Peter Abdelmalak

Theses

Optical coherence tomography (OCT) is a cross-sectional imaging modality based on low coherence light interferometry. OCT has been widely used in diagnostic ophthalmology and has found applications in other biomedical fields such as cancer detection and surgical guidance.

In the Laboratory of Biophotonics Imaging and Sensing at New Jersey Institute of Technology, we developed a unique needle OCT imager based on a single fiber probe for breast cancer imaging. The needle OCT imager with sub-millimeter diameter can be inserted into tissue for minimally invasive in situ breast imaging. OCT imaging provides spatial resolution similar to histology and has the potential …

High-Capacity Directional Graph Networks, Miguel Dominguez

Theses

Deep Neural Networks (DNN) have proven themselves to be a useful tool in many computer vision problems. One of the most popular forms of the DNN is the Convolutional Neural Network (CNN). The CNN effectively learns features on images by learning a weighted sum of local neighborhoods of pixels, creating filtered versions of the image. Point cloud analysis seems like it would benefit from this useful model. However, point clouds are much less structured than images. Many analogues to CNNs for point clouds have been proposed in the literature, but they are often much more constrained networks than the typical …

Improving Omnidirectional Camera-Based Robot Localization Through Self-Supervised Learning, Robert Relyea

Theses

Autonomous agents in any environment require accurate and reliable position and motion estimation to complete their required tasks. Many different sensor modalities have been utilized for this task such as GPS, ultra-wide band, visual simultaneous localization and mapping (SLAM), and light detection and ranging (LiDAR) SLAM. Many of the traditional positioning systems do not take advantage of the recent advances in the machine learning field. In this work, an omnidirectional camera position estimation system relying primarily on a learned model is presented. The positioning system benefits from the wide field of view provided by an omnidirectional camera. Recent developments in …

Design And Simulation Analysis Of Deep Learning Based Approaches And Multi-Attribute Algorithms For Warehouse Task Selection, Prashant Sankaran

Theses

With the growth and adoption of global supply chains and internet technologies, warehouse operations have become more demanding. Particularly, the number of orders being processed over a given time frame is drastically increasing, leading to more work content. This makes operational tasks, such as material retrieval and storage, done manually more inefficient. To improve system-level warehouse efficiency, collaborating Autonomous Vehicles (AVs) are needed. Several design challenges encompass an AV, some critical aspects are navigation, path planning, obstacle avoidance, task selection decisions, communication, and control systems. The current study addresses the warehouse task selection problem given a dynamic pending task list …

Advancing Multi-Modal Deep Learning: Towards Language-Grounded Visual Understanding, Kushal Kafle

Theses

Using deep learning, computer vision now rivals people at object recognition and detection, opening doors to tackle new challenges in image understanding. Among these challenges, understanding and reasoning about language grounded visual content is of fundamental importance to advancing artificial intelligence. Recently, multiple datasets and algorithms have been created as proxy tasks towards this goal, with visual question answering (VQA) being the most widely studied. In VQA, an algorithm needs to produce an answer to a natural language question about an image. However, our survey of datasets and algorithms for VQA uncovered several sources of dataset bias and sub-optimal evaluation …

Feature Extraction For Classification Of Auroral Images, Shwetha Herga

Theses

Auroras are a dynamically evolving phenomenon. Different auroral forms are correlated with various physical processes in the magnetosphere and ionosphere system. Millions of auroral images are captured every year by the modern ground-based All-Sky Imager(ASI). In dealing with data from ASI, machine learning techniques play a critical scientific role, facilitating both efficient searches and statistical studies. In this work, we manually label night-side auroral images from various Time History of Events and Macroscale Interactions during Substorms (THEMIS) all-sky imager based on the sky conditions; the labels are clear sky with auroras, cloudy with the moon, cloudy, clear-sky with the moon, …