Engineering Commons^™

Amodal Instance Segmentation And Multi-Object Tracking With Deep Pixel Embedding, Yanfeng Liu

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

This thesis extends upon the representational output of semantic instance segmentation by explicitly including both visible and occluded parts. A fully convolutional network is trained to produce consistent pixel-level embedding across two layers such that, when clustered, the results convey the full spatial extent and depth ordering of each instance. Results demonstrate that the network can accurately estimate complete masks in the presence of occlusion and outperform leading top-down bounding-box approaches.

The model is further extended to produce consistent pixel-level embeddings across two consecutive image frames from a video to simultaneously perform amodal instance segmentation and multi-object tracking. No post-processing …

Self-Driving Toy Car Using Deep Learning, Fahim Ahmed, Suleyman Turac, Mubtasem Ali

Publications and Research

Our research focuses on building a student affordable platform for scale model self-driving cars. The goal of this project is to explore current developments of Open Source hardware and software to build a low-cost platform consisting of the car chassis/framework, sensors, and software for the autopilot. Our research will allow other students with low budget to enter into the world of Deep Learning, self-driving cars, and autonomous cars racing competitions.

Seer: An Explainable Deep Learning Midi-Based Hybrid Song Recommender System, Khalil Damak, Olfa Nasraoui

Faculty Scholarship

State of the art music recommender systems mainly rely on either matrix factorization-based collaborative filtering approaches or deep learning architectures. Deep learning models usually use metadata for content-based filtering or predict the next user interaction by learning from temporal sequences of user actions. Despite advances in deep learning for song recommendation, none has taken advantage of the sequential nature of songs by learning sequence models that are based on content. Aside from the importance of prediction accuracy, other significant aspects are important, such as explainability and solving the cold start problem. In this work, we propose a hybrid deep learning …

Identifying Regional Trends In Avatar Customization, Peter Mawhorter, Sercan Sengun, Haewoon Kwak, D. Fox Harrell

Research Collection School Of Computing and Information Systems

Since virtual identities such as social media profiles and avatars have become a common venue for self-expression, it has become important to consider the ways in which existing systems embed the values of their designers. In order to design virtual identity systems that reflect the needs and preferences of diverse users, understanding how the virtual identity construction differs between groups is important. This paper presents a new methodology that leverages deep learning and differential clustering for comparative analysis of profile images, with a case study of almost 100 000 avatars from a large online community using a popular avatar creation …

Deep Learning (Partly) Demystified, Vladik Kreinovich, Olga Kosheleva

Departmental Technical Reports (CS)

Successes of deep learning are partly due to appropriate selection of activation function, pooling functions, etc. Most of these choices have been made based on empirical comparison and heuristic ideas. In this paper, we show that many of these choices -- and the surprising success of deep learning in the first place -- can be explained by reasonably simple and natural mathematics.

Why Deep Learning Is More Efficient Than Support Vector Machines, And How It Is Related To Sparsity Techniques In Signal Processing, Laxman Bokati, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

Several decades ago, traditional neural networks were the most efficient machine learning technique. Then it turned out that, in general, a different technique called support vector machines is more efficient. Reasonably recently, a new technique called deep learning has been shown to be the most efficient one. These are empirical observations, but how we explain them -- thus making the corresponding conclusions more reliable? In this paper, we provide a possible theoretical explanation for the above-described empirical comparisons. This explanation enables us to explain yet another empirical fact -- that sparsity techniques turned out to be very efficient in signal …

Flood Management Deep Learning Model Inputs: A Review Of Necessary Data And Predictive Tools, Jacob Hale, Suzanna Long, Steven Corns, Tom Shoberg

Engineering Management and Systems Engineering Faculty Research & Creative Works

Current flood management models are often hampered by the lack of robust predictive analytics, as well as incomplete datasets for river basins prone to heavy flooding. This research uses a State-of-the-Art matrix (SAM) analysis and integrative literature review to categorize existing models by method and scope, then determines opportunities for integrating deep learning techniques to expand predictive capability. Trends in the SAM analysis are then used to determine geospatial characteristics of the region that can contribute to flash flood scenarios, as well as develop inputs for future modeling efforts. Preliminary progress on the selection of one urban and one rural …

Limited Data Rolling Bearing Fault Diagnosis With Few-Shot Learning, Ansi Zhang, Shaobo Li, Yuxin Cui, Wanli Yang, Rongzhi Dong, Jianjun Hu

Faculty Publications

This paper focuses on bearing fault diagnosis with limited training data. A major challenge in fault diagnosis is the infeasibility of obtaining sufficient training samples for every fault type under all working conditions. Recently deep learning based fault diagnosis methods have achieved promising results. However, most of these methods require large amount of training data. In this study, we propose a deep neural network based few-shot learning approach for rolling bearing fault diagnosis with limited data. Our model is based on the siamese neural network, which learns by exploiting sample pairs of the same or different categories. Experimental results over …

A Review Of Text Corpus-Based Tourism Big Data Mining, Qin Lin, Shaobo Li, Sen Zhang, Jie Hu, Jianjun Hu

Faculty Publications

With the massive growth of the Internet, text data has become one of the main formats of tourism big data. As an effective expression means of tourists’ opinions, text mining of such data has big potential to inspire innovations for tourism practitioners. In the past decade, a variety of text mining techniques have been proposed and applied to tourism analysis to develop tourism value analysis models, build tourism recommendation systems, create tourist profiles, and make policies for supervising tourism markets. The successes of these techniques have been further boosted by the progress of natural language processing (NLP), machine learning, and …

A Review Of Text Corpus-Based Tourism Big Data Mining, Qin Li, Shaobo Li, Sen Zhang, Jie Hu, Jianhun Hu

Faculty Publications

With the massive growth of the Internet, text data has become one of the main formats of tourism big data. As an effective expression means of tourists’ opinions, text mining of such data has big potential to inspire innovations for tourism practitioners. In the past decade, a variety of text mining techniques have been proposed and applied to tourism analysis to develop tourism value analysis models, build tourism recommendation systems, create tourist profiles, and make policies for supervising tourism markets. The successes of these techniques have been further boosted by the progress of natural language processing (NLP), machine learning, and …

Machine Learning Methodology Review For Computational Electromagnetics, He Ming Yao, Lijun Jiang, Huan Huan Zhang, Wei E.I. Sha

Electrical and Computer Engineering Faculty Research & Creative Works

While machine learning is revolutionizing every corner of modern technologies, we have been attempting to explore whether machine learning methods could be used in computational electromagnetic (CEM). In this paper, five efforts in line with this direction are reviewed. They include forward methods such as the method of moments (MoM) solved by the artificial neural network training process, FDTD PML (perfectly matched layer) using the hyperbolic tangent basis function (HTBF), etc. There are also inverse problems that use the deep ConvNets for the effective source reconstruction and subwavelength imaging in the far-field. Benchmarks are provided to demonstrate the feasibility of …

Mid To Late Season Weed Detection In Soybean Production Fields Using Unmanned Aerial Vehicle And Machine Learning, Arun Narenthiran Veeranampalayam Sivakumar

Department of Agricultural and Biological Systems Engineering: Dissertations, Theses, and Student Research

Mid-late season weeds are those that escape the early season herbicide applications and those that emerge late in the season. They might not affect the crop yield, but if uncontrolled, will produce a large number of seeds causing problems in the subsequent years. In this study, high-resolution aerial imagery of mid-season weeds in soybean fields was captured using an unmanned aerial vehicle (UAV) and the performance of two different automated weed detection approaches – patch-based classification and object detection was studied for site-specific weed management. For the patch-based classification approach, several conventional machine learning models on Haralick texture features were …

In Vivo Human-Like Robotic Phenotyping Of Leaf And Stem Traits In Maize And Sorghum In Greenhouse, Abbas Atefi

Department of Agricultural and Biological Systems Engineering: Dissertations, Theses, and Student Research

In plant phenotyping, the measurement of morphological, physiological and chemical traits of leaves and stems is needed to investigate and monitor the condition of plants. The manual measurement of these properties is time consuming, tedious, error prone, and laborious. The use of robots is a new approach to accomplish such endeavors, which enables automatic monitoring with minimal human intervention. In this study, two plant phenotyping robotic systems were developed to realize automated measurement of plant leaf properties and stem diameter which could reduce the tediousness of data collection compare to manual measurements. The robotic systems comprised of a four degree …

Deep Autoencoder Neural Networks For Short-Term Traffic Congestion Prediction Of Transportation Networks, Sen Zhang, Yong Yao, Jie Hu, Yong Zhao, Shaobo Li, Jianjun Hu

Faculty Publications

Traffic congestion prediction is critical for implementing intelligent transportation systems for improving the efficiency and capacity of transportation networks. However, despite its importance, traffic congestion prediction is severely less investigated compared to traffic flow prediction, which is partially due to the severe lack of large-scale high-quality traffic congestion data and advanced algorithms. This paper proposes an accessible and general workflow to acquire large-scale traffic congestion data and to create traffic congestion datasets based on image analysis. With this workflow we create a dataset named Seattle Area Traffic Congestion Status (SATCS) based on traffic congestion map snapshots from a publicly available …

Deep Autoencoder Neural Networks For Short-Term Traffic Congestion Prediction Of Transportation Networks, Sen Zhang, Yong Yao, Jie Hu, Yong Zhao, Shaobo Li, Jianjun Hu

Faculty Publications

Traffic congestion prediction is critical for implementing intelligent transportation systems for improving the efficiency and capacity of transportation networks. However, despite its importance, traffic congestion prediction is severely less investigated compared to traffic flow prediction, which is partially due to the severe lack of large-scale high-quality traffic congestion data and advanced algorithms. This paper proposes an accessible and general workflow to acquire large-scale traffic congestion data and to create traffic congestion datasets based on image analysis. With this workflow we create a dataset named Seattle Area Traffic Congestion Status (SATCS) based on traffic congestion map snapshots from a publicly available …

Deep Autoencoder Neural Networks For Short-Term Traffic Congestion Prediction Of Transportation Networks, Sen Zhang, Yong Yao, Jie Hu, Yong Zhao, Shaobo Li, Jianjun Hu

Faculty Publications

Traffic congestion prediction is critical for implementing intelligent transportation systems for improving the efficiency and capacity of transportation networks. However, despite its importance, traffic congestion prediction is severely less investigated compared to traffic flow prediction, which is partially due to the severe lack of large-scale high-quality traffic congestion data and advanced algorithms. This paper proposes an accessible and general workflow to acquire large-scale traffic congestion data and to create traffic congestion datasets based on image analysis. With this workflow we create a dataset named Seattle Area Traffic Congestion Status (SATCS) based on traffic congestion map snapshots from a publicly available …

Deep Learning Segmentation Of Coronary Calcified Plaque From Intravascular Optical Coherence Tomography (Ivoct) Images With Application To Finite Element Modeling Of Stent Deployment, Yazan Gharaibeh, Pengfei Dong, David Prabhu, Chaitanya Kolluru, Juhwan Lee, Vlad Zimin, Hozhabr Mozafari, Hiram Bizzera, Linxia Gu, David Wilson

Department of Mechanical and Materials Engineering: Faculty Publications

Because coronary artery calcified plaques can hinder or eliminate stent deployment, interventional cardiologists need a better way to plan interventions, which might include one of the many methods for calcification modification (e.g., atherectomy). We are imaging calcifications with intravascular optical coherence tomography (IVOCT), which is the lone intravascular imaging technique with the ability to image the extent of a calcification, and using results to build vessel-specific finite element models for stent deployment. We applied methods to a large set of image data (>45 lesions and > 2,600 image frames) of calcified plaques, manually segmented by experts into calcified, lumen and …

Multi-Pig Part Detection And Association With A Fully-Convolutional Network, Eric T. Psota, Mateusz Mittek, Lance C. Pérez, Ty Schmidt, Benny Mote

Department of Electrical and Computer Engineering: Faculty Publications

Computer vision systems have the potential to provide automated, non-invasive monitoring of livestock animals, however, the lack of public datasets with well-defined targets and evaluation metrics presents a significant challenge for researchers. Consequently, existing solutions often focus on achieving task-specific objectives using relatively small, private datasets. This work introduces a new dataset and method for instance-level detection of multiple pigs in group-housed environments. The method uses a single fully-convolutional neural network to detect the location and orientation of each animal, where both body part locations and pairwise associations are represented in the image space. Accompanying this method is a new …

Mouldingnet: Deep-Learning For 3d Object Reconstruction, Tobias Burns, Barak Pearlmutter, John B. Mcdonald

Session 2: Deep Learning for Computer Vision

th the rise of deep neural networks a number of approaches for learning over 3D data have gained popularity. In this paper, we take advantage of one of these approaches, bilateral convolutional layers to propose a novel end-to-end deep auto-encoder architecture to efficiently encode and reconstruct 3D point clouds. Bilateral convolutional layers project the input point cloud onto an even tessellation of a hyperplane in the (d Å1)-dimensional space known as the permutohedral lattice and perform convolutions over this representation. In contrast to existing point cloud based learning approaches, this allows us to learn over the underlying geometry of the …

Deep Cnn Frameworks For Comparison For Malaria Diagnosis, Priyadarshini Adyasha Pattanaik, Zelong Wang, Patrick Horain

Session 2: Deep Learning for Computer Vision

Abstract We compare Deep Convolutional Neural Networks (DCNN) frameworks, namely AlexNet and VGGNet, for the classification of healthy and malaria-infected cells in large, grayscale, low quality and low resolution microscopic images, in the case only a small training set is available. Experimental results deliver promising results on the path to quick, automatic and precise classification in unstrained images.

Place Recognition In Challenging Conditions, Saravanabalagi Ramachandran, John Mcdonald

Session 2: Deep Learning for Computer Vision

Place recognition in a visual SLAM system helps build and maintain a map from multiple traversals of the same environment while closing loops to correct drift accumulated over time. Despite the marked success in visual place recognition research over the past decade, it remains a challenging problem in the context of variations caused due to different times of the day, weather, lighting and seasons. In this paper, we address this problem by progressively training convolutional neural networks in a siamese fashion to generate embeddings that encode semantic and visual features for sequence-aligned image pairs taken at different timescales and viewpoints. …

Deep Convolutional Neural Networks For Estimating Lens Distortion Parameters, Sebastian Lutz, Mark Davey, Aljosa Smolic

Session 2: Deep Learning for Computer Vision

In this paper we present a convolutional neural network (CNN) to predict multiple lens distortion parameters from a single input image. Unlike other methods, our network is suitable to create high resolution output as it directly estimates the parameters from the image which then can be used to rectify even very high resolution input images. As our method it is fully automatic, it is suitable for both casual creatives and professional artists. Our results show that our network accurately predicts the lens distortion parameters of high resolution images and corrects the distortions satisfactory.

Synthetic Positron Emission Tomography Using Conditional-Generative Adversarial Networks For Healthy Bone Marrow Baseline Image Generation, Patrick Leydon, Martin O'Connell, Derek Greene, Kathleen Curran

Session 6: Applications, Architecture and Systems Integration

A Conditional-Generative Adversarial Network has been used for a supervised image-to-image transla- tion task which outputs a synthetic PET scan based on real patient CT data. The network is trained using only data of patients with healthy bone marrow metabolism. This allows for a patient specific synthetic healthy baseline scan to be produced. This can be used by a clinician for comparison to real PET data in the absence of a baseline scan or to aid in the diagnosis of conditions such as Multiple Myeloma which manifest as changes in bone marrow metabolism.

An Explainable Autoencoder For Collaborative Filtering Recommendation, Pegah Sagheb Haghighi, Olurotimi Seton, Olfa Nasraoui

Faculty Scholarship

Autoencoders are a common building block of Deep Learning architectures, where they are mainly used for representation learning. They have also been successfully used in Collaborative Filtering (CF) recommender systems to predict missing ratings. Unfortunately, like all black box machine learning models, they are unable to explain their outputs. Hence, while predictions from an Autoencoderbased recommender system might be accurate, it might not be clear to the user why a recommendation was generated. In this work, we design an explainable recommendation system using an Autoencoder model whose predictions can be explained using the neighborhood based explanation style. Our preliminary work …

A Statistical Approach To Provide Explainable Convolutional Neural Network Parameter Optimization, Saman Akbarzadeh, Selam Ahderom, Kamal Alameh

Research outputs 2014 to 2021

Algorithms based on convolutional neural networks (CNNs) have been great attention in image processing due to their ability to find patterns and recognize objects in a wide range of scientific and industrial applications. Finding the best network and optimizing its hyperparameters for a specific application are central challenges for CNNs. Most state-of-the-art CNNs are manually designed, while techniques for automatically finding the best architecture and hyperparameters are computationally intensive, and hence, there is a need to severely limit their search space. This paper proposes a fast statistical method for CNN parameter optimization, which can be applied in many CNN applications …

Multi-Sensory Deep Learning Architectures For Slam Dunk Scene Classification, Paul Minogue

Dissertations

Basketball teams at all levels of the game invest a considerable amount of time and effort into collecting, segmenting, and analysing footage from their upcoming opponents previous games. This analysis helps teams identify and exploit the potential weaknesses of their opponents and is commonly cited as one of the key elements required to achieve success in the modern game. The growing importance of this type of analysis has prompted research into the application of computer vision and audio classification techniques to help teams classify scoring sequences and key events using game footage. However, this research tends to focus on classifying …

Fisheyemultinet: Real-Time Multi-Task Learning Architecture For Surround-View Automated Parking System., Pullaro Maddu, Wayne Doherty, Ganesh Sistu, Isabelle Leang, Michal Uricar, Sumanth Chennupati, Hazem Rashed, Jonathan Horgan, Ciaran Hughes, Senthil Yogamani

Session 6: Applications, Architecture and Systems Integration

Automated Parking is a low speed manoeuvring scenario which is quite unstructured and complex, requiring full 360° near-field sensing around the vehicle. In this paper, we discuss the design and implementation of an automated parking system from the perspective of camera based deep learning algorithms. We provide a holistic overview of an industrial system covering the embedded system, use cases and the deep learning architecture. We demonstrate a real-time multi-task deep learning network called FisheyeMultiNet, which detects all the necessary objects for parking on a low-power embedded system. FisheyeMultiNet runs at 15 fps for 4 cameras and it has three …