Artificial Intelligence and Robotics Commons^™

Monitoring Plants Growth In Indoor Vertical Farms Using Computer Vision And Ai Techniques, Bhama Krishna Pillutla

Graduate Research Theses & Dissertations

Climatic conditions like temperature, drought, and heavy metals disturb plant cell structures and, ultimately, plant growth that significantly affects crop production. Due to increasing climate change, maize crop yields are projected to decline by 24% by the end of century. With the increase in food demands and decrease in agricultural land and water resources, the space for effective farming is left much desired. Though limited to a few crops at this moment, Indoor Vertical Farming is one technique that requires much less land space, water, soil, and sunlight when compared to traditional farming. Vertical farming allows artificial control of temperature, …

Customer Gaze Estimation In Retail Using Deep Learning, Shashimal Senarath, Primesh Pathirana, Dulani Meedeniya, Sampath Jayarathna

Computer Science Faculty Publications

At present, intelligent computing applications are widely used in different domains, including retail stores. The analysis of customer behaviour has become crucial for the benefit of both customers and retailers. In this regard, the concept of remote gaze estimation using deep learning has shown promising results in analyzing customer behaviour in retail due to its scalability, robustness, low cost, and uninterrupted nature. This study presents a three-stage, three-attention-based deep convolutional neural network for remote gaze estimation in retail using image data. In the first stage, we design a mechanism to estimate the 3D gaze of the subject using image data …

Machine Learning And Computer Vision In Solar Physics, Haodi Jiang

Dissertations

In the recent decades, the difficult task of understanding and predicting violent solar eruptions and their terrestrial impacts has become a strategic national priority, as it affects the life of human beings, including communication, transportation, the power grid, national defense, space travel, and more. This dissertation explores new machine learning and computer vision techniques to tackle this difficult task. Specifically, the dissertation addresses four interrelated problems in solar physics: magnetic flux tracking, fibril tracing, Stokes inversion and vector magnetogram generation.

First, the dissertation presents a new deep learning method, named SolarUnet, to identify and track solar magnetic flux elements in …

Ow-Detr: Open-World Detection Transformer, Akshita Gupta, Sanath Narayan, K.J. Joseph, Salman Khan, Fahad Shahbaz Khan, Mubarak Shah

Computer Vision Faculty Publications

Open-world object detection (OWOD) is a challenging computer vision problem, where the task is to detect a known set of object categories while simultaneously identifying unknown objects. Additionally, the model must incrementally learn new classes that become known in the next training episodes. Distinct from standard object detection, the OWOD setting poses significant challenges for generating quality candidate proposals on potentially unknown objects, separating the unknown objects from the background and detecting diverse unknown objects. Here, we introduce a novel end-to-end transformer-based framework, OW-DETR, for open-world object detection. The proposed OW-DETR comprises three dedicated components namely, attention-driven pseudo-labeling, novelty classification …

Situate: An Agent-Based System For Situation Recognition, Max Henry Quinn

Dissertations and Theses

Computer vision and machine learning systems have improved significantly in recent years, largely based on the development of deep learning systems, leading to impressive performance on object detection tasks. Understanding the content of images is considerably more difficult. Even simple situations, such as "a handshake", "walking the dog", "a game of ping-pong", or "people waiting for a bus", present significant challenges. Each consists of common objects, but are not reliably detectable as a single entity nor through the simple co-occurrence of their parts.

In this dissertation, toward the goal of developing machine learning systems that demonstrate properties associated with understanding, …

Advances In Deep Learning With Applications To Computer Vision And Astronomy, Zhihang Hu

Dissertations

Deep Learning has spanned a variety of applications in computer vision as well as computational astronomy. These two aspects obtained similar data structure, therefore, their solutions can be transferable between each other. This dissertation look into two video-related tasks in computer vision and propose a novel problem in computational astronomy.

Specifically, acquiring an in-depth understanding of videos has been a cornerstone problem in computer vision. This problem has been studied by various researchers from different perspectives, among which video prediction has attracted much attention. Video prediction aims to generate the pixels of future frames given a sequence of context frames. …

Novel Statistical Modeling Methods For Traffic Video Analysis, Hang Shi

Dissertations

Video analysis is an active and rapidly expanding research area in computer vision and artificial intelligence due to its broad applications in modern society. Many methods have been proposed to analyze the videos, but many challenging factors remain untackled. In this dissertation, four statistical modeling methods are proposed to address some challenging traffic video analysis problems under adverse illumination and weather conditions.

First, a new foreground detection method is presented to detect the foreground objects in videos. A novel Global Foreground Modeling (GFM) method, which estimates a global probability density function for the foreground and applies the Bayes decision rule …

Computer Vision Applications For Autonomous Aerial Vehicles, Burak Kakillioglu

Dissertations - ALL

Undoubtedly, unmanned aerial vehicles (UAVs) have experienced a great leap forward over the last decade. It is not surprising anymore to see a UAV being used to accomplish a certain task, which was previously carried out by humans or a former technology. The proliferation of special vision sensors, such as depth cameras, lidar sensors and thermal cameras, and major breakthroughs in computer vision and machine learning fields accelerated the advance of UAV research and technology. However, due to certain unique challenges imposed by UAVs, such as limited payload capacity, unreliable communication link with the ground stations and data safety, UAVs …

Signal Processing And Data Analysis For Real-Time Intermodal Freight Classification Through A Multimodal Sensor System., Enrique J. Sanchez Headley

Graduate Theses and Dissertations

Identifying freight patterns in transit is a common need among commercial and municipal entities. For example, the allocation of resources among Departments of Transportation is often predicated on an understanding of freight patterns along major highways. There exist multiple sensor systems to detect and count vehicles at areas of interest. Many of these sensors are limited in their ability to detect more specific features of vehicles in traffic or are unable to perform well in adverse weather conditions. Despite this limitation, to date there is little comparative analysis among Laser Imaging and Detection and Ranging (LIDAR) sensors for freight detection …

Methods For Detecting Floodwater On Roadways From Ground Level Images, Cem Sazara

Computational Modeling & Simulation Engineering Theses & Dissertations

Recent research and statistics show that the frequency of flooding in the world has been increasing and impacting flood-prone communities severely. This natural disaster causes significant damages to human life and properties, inundates roads, overwhelms drainage systems, and disrupts essential services and economic activities. The focus of this dissertation is to use machine learning methods to automatically detect floodwater in images from ground level in support of the frequently impacted communities. The ground level images can be retrieved from multiple sources, including the ones that are taken by mobile phone cameras as communities record the state of their flooded streets. …

Adaptive Aggregation Networks For Class-Incremental Learning, Yaoyao Liu, Bernt Schiele, Qianru Sun

Research Collection School Of Computing and Information Systems

Class-Incremental Learning (CIL) aims to learn a classification model with the number of classes increasing phase-by-phase. An inherent problem in CIL is the stability-plasticity dilemma between the learning of old and new classes, i.e., high-plasticity models easily forget old classes, but high-stability models are weak to learn new classes. We alleviate this issue by proposing a novel network architecture called Adaptive Aggregation Networks (AANets) in which we explicitly build two types of residual blocks at each residual level (taking ResNet as the baseline architecture): a stable block and a plastic block. We aggregate the output feature maps from these two …

Counterfactual Zero-Shot And Open-Set Visual Recognition, Zhongqi Yue, Tan Wang, Qianru Sun, Xian-Sheng Hua, Hanwang Zhang

Research Collection School Of Computing and Information Systems

We present a novel counterfactual framework for both Zero-Shot Learning (ZSL) and Open-Set Recognition (OSR), whose common challenge is generalizing to the unseen-classes by only training on the seen-classes. Our idea stems from the observation that the generated samples for unseen-classes are often out of the true distribution, which causes severe recognition rate imbalance between the seen-class (high) and unseen-class (low). We show that the key reason is that the generation is not Counterfactual Faithful, and thus we propose a faithful one, whose generation is from the sample-specific counterfactual question: What would the sample look like, if we set its …

Projecting Your View Attentively: Monocular Road Scene Layout Estimation Via Cross-View Transformation, Weixiang Yang, Qi Li, Wenxi Liu, Yuanlong Yu, Yuexin Ma, Shengfeng He, Jia Pan

Research Collection School Of Computing and Information Systems

HD map reconstruction is crucial for autonomous driving. LiDAR-based methods are limited due to the deployed expensive sensors and time-consuming computation. Camera-based methods usually need to separately perform road segmentation and view transformation, which often causes distortion and the absence of content. To push the limits of the technology, we present a novel framework that enables reconstructing a local map formed by road layout and vehicle occupancy in the bird's-eye view given a front-view monocular image only. In particular, we propose a cross-view transformation module, which takes the constraint of cycle consistency between views into account and makes full use …

Rm-Net: Rasterizing Markov Signals To Images For Deep Learning, Kajal Gupta

Theses

Statistical machine learning approaches are quite famous for processing Markov signal data. They can model unobserved states and learn certain characteristics particular to a signal with good accuracy. However, with the advent of Deep learning the novice ways of solving a problem has shifted towards this more sophisticated algorithm, which is much better, powerful and more accurate. Specifically, Convolutional Neural Nets (CNN) have shown many promising results on images and videos. Here we illustrate how CNN can be applied to a 1D numeric signal using signal rasterization technique. We start by rasterizing a 1D numeric Markov signal into an image …

Towards Open World Object Detection, K. J. Joseph, Salman Khan, Fahad Shahbaz Khan, Vineeth N. Balasubramanian

Computer Vision Faculty Publications

Humans have a natural instinct to identify unknown object instances in their environments. The intrinsic curiosity about these unknown instances aids in learning about them, when the corresponding knowledge is eventually available. This motivates us to propose a novel computer vision problem called: 'Open World Object Detection', where a model is tasked to: 1) identify objects that have not been introduced to it as 'unknown', without explicit supervision to do so, and 2) incrementally learn these identified unknown categories without forgetting previously learned classes, when the corresponding labels are progressively received. We formulate the problem, introduce a strong evaluation protocol …

Using Deep Learning To Analyze Materials In Medical Images, Carson Molder

Computer Science and Computer Engineering Undergraduate Honors Theses

Modern deep learning architectures have become increasingly popular in medicine, especially for analyzing medical images. In some medical applications, deep learning image analysis models have been more accurate at predicting medical conditions than experts. Deep learning has also been effective for material analysis on photographs. We aim to leverage deep learning to perform material analysis on medical images. Because material datasets for medicine are scarce, we first introduce a texture dataset generation algorithm that automatically samples desired textures from annotated or unannotated medical images. Second, we use a novel Siamese neural network called D-CNN to predict patch similarity and build …

Multi-Modal Classification Using Images And Text, Stuart J. Miller, Justin Howard, Paul Adams, Mel Schwan, Robert Slater

SMU Data Science Review

This paper proposes a method for the integration of natural language understanding in image classification to improve classification accuracy by making use of associated metadata. Traditionally, only image features have been used in the classification process; however, metadata accompanies images from many sources. This study implemented a multi-modal image classification model that combines convolutional methods with natural language understanding of descriptions, titles, and tags to improve image classification. The novelty of this approach was to learn from additional external features associated with the images using natural language understanding with transfer learning. It was found that the combination of ResNet-50 image …

Learning Graphs For Object Tracking And Counting, Shengkun Li

Legacy Theses & Dissertations (2009 - 2024)

As important problems in computer vision, object tracking and counting attract increasing amounts of attention in recent years due to its wide range of applications, such as video surveillance, human- computer interaction, smart city. Despite much progress has been made in object tracking and counting with the arriving of deep neural networks (DNN), there still remains much room for improvement to satisfy the real-world applications.

Ship Deck Segmentation In Engineering Document Using Generative Adversarial Networks, Mohammad Shahab Uddin, Raphael Pamie-George, Daron Wilkins, Andres Sousa Poza, Mustafa Canan, Samuel Kovacic, Jiang Li

Engineering Management & Systems Engineering Faculty Publications

Generative adversarial networks (GANs) have become very popular in recent years. GANs have proved to be successful in different computer vision tasks including image-translation, image super-resolution etc. In this paper, we have used GAN models for ship deck segmentation. We have used 2D scanned raster images of ship decks provided by US Navy Military Sealift Command (MSC) to extract necessary information including ship walls, objects etc. Our segmentation results will be helpful to get vector and 3D image of a ship that can be later used for maintenance of the ship. We applied the trained models to engineering documents provided …

Inference Of Surface Velocities From Oblique Time Lapse Photos And Terrestrial Based Lidar At The Helheim Glacier, Franklyn T. Dunbar Ii

Graduate Student Theses, Dissertations, & Professional Papers

Using time dependent observations derived from terrestrial LiDAR and oblique
time-lapse imagery, we demonstrate that a Bayesian approach to glacial motion es-
timation provides a concise way to incorporate multiple data products into a single
motion estimation procedure effectively producing surface velocity estimates with
an associated uncertainty. This approach brings both improved computational effi-
ciency, and greater scalability across observational time-frames when compared to
existing methods. To gauge efficacy, we apply these methods to a set of observa-
tions from the Helheim Glacier, a critical actor in contemporary mass loss trends
observed in the Greenland Ice Sheet. We find that …

Survey On Deep Neural Networks In Speech And Vision Systems, M. Alam, Manar D. Samad, Lasitha Vidyaratne, ‪Alexander Glandon, Khan M. Iftekharuddin

Computer Science Faculty Research

This survey presents a review of state-of-the-art deep neural network architectures, algorithms, and systems in speech and vision applications. Recent advances in deep artificial neural network algorithms and architectures have spurred rapid innovation and development of intelligent speech and vision systems. With availability of vast amounts of sensor data and cloud computing for processing and training of deep neural networks, and with increased sophistication in mobile and embedded technology, the next-generation intelligent systems are poised to revolutionize personal and commercial computing. This survey begins by providing background and evolution of some of the most successful deep learning models for intelligent …

Language-Driven Region Pointer Advancement For Controllable Image Captioning, Annika Lindh, Robert J. Ross, John D. Kelleher

Conference papers

Controllable Image Captioning is a recent sub-field in the multi-modal task of Image Captioning wherein constraints are placed on which regions in an image should be described in the generated natural language caption. This puts a stronger focus on producing more detailed descriptions, and opens the door for more end-user control over results. A vital component of the Controllable Image Captioning architecture is the mechanism that decides the timing of attending to each region through the advancement of a region pointer. In this paper, we propose a novel method for predicting the timing of region pointer advancement by treating the …

Modular Neural Networks For Low-Power Image Classification On Embedded Devices, Abhinav Goel, Sara Aghajanzadeh, Caleb Tung, Shuo-Han Chen, George K. Thiruvathukal, Yung-Hisang Lu

Computer Science: Faculty Publications and Other Works

Embedded devices are generally small, battery-powered computers with limited hardware resources. It is difficult to run deep neural networks (DNNs) on these devices, because DNNs perform millions of operations and consume significant amounts of energy. Prior research has shown that a considerable number of a DNN’s memory accesses and computation are redundant when performing tasks like image classification. To reduce this redundancy and thereby reduce the energy consumption of DNNs, we introduce the Modular Neural Network Tree architecture. Instead of using one large DNN for the classifier, this architecture uses multiple smaller DNNs (called modules) to progressively classify images …

Study On Irregular Projection Surface Depth Perception Geometric Correction, Baoxing Bai, Yang Fan, Han Cheng, Zhang Chao

Journal of System Simulation

Abstract: In order to improve the universality of the projector projection scene, a method of depth perception geometric correction on the irregular projection surface was proposed. With the theoretical analysis of the coupling, the weaken coupling solution was proposed. The depth of the irregular projection surface could be acquired by the encoded color structured light. After that, the feedback was used to solve the maximum visual projection area, the dynamic slicing algorithm for the irregular surfaces depth information was used to pull away the stratification planes, so a corresponding homography matrix for each stratification plane was calculated and the homography …

Using Color Thresholding And Contouring To Understand Coral Reef Biodiversity, Scott Vuong Tran

Master's Projects

This paper presents research outcomes of understanding coral reef biodiversity through the usage of various computer vision applications and techniques. It aims to help further analyze and understand the coral reef biodiversity through the usage of color thresholding and contouring onto images of the ARMS plates to extract groups of microorganisms based on color. The results are comparable to the manual markup tool developed to do the same tasks and shows that the manual process can be sped up using computer vision. The paper presents an automated way to extract groups of microorganisms based on color without the use of …

Object Detection With Deep Learning To Accelerate Pose Estimation For Automated Aerial Refueling, Andrew T. Lee

Theses and Dissertations

Remotely piloted aircraft (RPAs) cannot currently refuel during flight because the latency between the pilot and the aircraft is too great to safely perform aerial refueling maneuvers. However, an AAR system removes this limitation by allowing the tanker to directly control the RP A. The tanker quickly finding the relative position and orientation (pose) of the approaching aircraft is the first step to create an AAR system. Previous work at AFIT demonstrates that stereo camera systems provide robust pose estimation capability. This thesis first extends that work by examining the effects of the cameras' resolution on the quality of pose …

Monocular Depth Image Mark-Less Pose Estimation Based On Feature Regression, Chen Ying, Shen Li

Journal of System Simulation

Abstract: Monocular camera mark-less pose estimation system suffers low accuracy, robustness and efficiency due to variety of action, self-occlusion of human body. A method of feature exaction from point clouds was proposed, in which a single-to-multiple (S2M) feature regressor and a joint position regressor were designed to quickly and accurately predict the 3D positions of body joints from a single depth image without any temporal information. Experiment result shows that the estimation accuracy is superior to that of state-of-the-arts and multi-camera based methods.

Automated Recognition Of Facial Affect Using Deep Neural Networks, Behzad Hasani

Electronic Theses and Dissertations

Automated Facial Expression Recognition (FER) has been a topic of study in the field of computer vision and machine learning for decades. In spite of efforts made to improve the accuracy of FER systems, existing methods still are not generalizable and accurate enough for use in real-world applications. Many of the traditional methods use hand-crafted (a.k.a. engineered) features for representation of facial images. However, these methods often require rigorous hyper-parameter tuning to achieve favorable results.

Recently, Deep Neural Networks (DNNs) have shown to outperform traditional methods in visual object recognition. DNNs require huge data as well as powerful computing units …

An Automated Method For Detecting Water Levels Using Computer Vision And Artificial Intelligence, Priyanjani Chowdary Chandra

Graduate Research Theses & Dissertations

Flooding is one of the most dangerous weather events today. Between 2015-2019, on average, it has caused more than 130 deaths every year in the USA alone. World Health Organization has reported that, between 1998-2017, floods have affected more than 2 billion people worldwide. The devastating nature of flood necessitates the continuous monitoring of water level in the rivers and streams in flood-prone areas to detect the incoming flood. In this thesis, we have designed and implemented a computer vision and AI-based system that continuously detect the water level in the creek. Our solution employs an effective template matching algorithm …

Computer Vision Gesture Recognition For Rock Paper Scissors, Nicholas Hunter

Senior Independent Study Theses

This project implements a human versus computer game of rock-paper-scissors using machine learning and computer vision. Player’s hand gestures are detected using single images with the YOLOv3 object detection system. This provides a generalized detection method which can recognize player moves without the need for a special background or lighting setup. Additionally, past moves are examined in context to predict the most probable next move of the system’s opponent. In this way, the system achieves higher win rates against human opponents than by using a purely random strategy.