Other Computer Engineering Commons^™

Early Detection Of Oak Wilt Using Unmanned Aerial Vehicles (Uav) & Computer Vision, Muttaki I. Bismoy

Masters Theses

Forests are critical ecosystems, delivering services such as biodiversity conservation, climate regulation, timber production, and recreation. However, they face increasing threats from pathogens like Bretziella fagacearum, which causes Oak Wilt, a lethal disease that disrupts water transport in oak trees, leading to canopy dieback and eventual death. Traditional detection methods rely on manual ground surveys, which are labor-intensive, time-consuming, and prone to error, particularly in early disease stages.

This research presents an automated, scalable, high-precision Oak Wilt detection system using Unmanned Aerial Vehicles (UAVs) combined with deep learning-based computer vision. Expanding on earlier work with a lightweight CNN achieving an …

Simulated Live Studio Audience, Theodore David Shellenberger

Computer Engineering

The Simulated Live Studio Audience is a Python based application that utilizes Vosk, Roboflow, and Llama 3.2 to provide a user with auditory feedback based upon both visual and audible input from their device's microphone and camera. This system functions with a custom trained computer vision model to detect a specified object and when individuals walk in and out of the camera frame, outputting sitcom style simulated crowd reaction sounds accordingly. The simulated studio audience program also takes in vocal input from users, converts it to text, and, using a large language model, analyzes it for content that can be …

Deeppanorf: Deep Prior For Neural 3d Reconstruction From Sparse Panoramas, Edward Du

Master's Theses

Advances in neural field representations have led to a significant improvement in view synthesis quality. However, many current novel view synthesis methods rely on a dense set of input views, which can be impractical and inefficient in real-world applications. We propose DeepPanoRF, a novel method for 360◦ scene reconstruction from a sparse set of input equirectangular panoramas. Built upon K-Planes, a radiance field representation that encodes explicit features on orthogonal feature planes, our method does not directly learn feature grids. Instead, we parameterize the feature grids to enable sparse view reconstruction without pretraining or additional regularization. We implement a custom …

Modality Distillation Using A Sam-Guided Multimodal Teacher For Unimodal Wildfire Segmentation And Temperature Prediction, Michael N. Marinaccio

All Theses

Wildfires are one of the world’s most devastating natural disasters that affect the environment, communities, and more critically, humans that live in and around those communities. Due to the threat of large-scale destruction in landscapes and human inhabited areas, it has become increasingly more important to develop wildfire detection, management, and suppression strategies to mitigate and prevent these negative outcomes. Wildfire research encompasses many different areas. Most notably, the development of communication, navigation, remote sensing, and monitoring systems. In wildfire monitoring, limitations discovered in-ground and satellite observation have shifted the focus toward Unmanned Aerial Vehicle (UAV) based wildfire research, which …

Augmenting Machine Learning Technique Through Natural Language, Tasmia Tasrin

Theses and Dissertations--Computer Science

While artificial intelligence (AI) and machine learning (ML) have proven effective at addressing many of the challenges that we face in our everyday lives, there are many situations in which these methods struggle. Examples include environments where AI or ML systems must perform complex behaviors or those where rewards are difficult to calculate. To address this limitation, interactive machine learning (IML) techniques have been introduced, which incorporate machine-understandable human feedback into traditional ML approaches. This feedback is often given as a discrete, positive or negative numeric value. This feedback is typically provided as often as possible to convey a dense …

Framework For Identity Privacy Through Gender Based Skeletonization, Harrison Hwang

Master's Projects

The protection of one’s privacy and sensitive information is becoming increasingly difficult in the modern age full of surveillance and data collection. Through the use of image based object detection machine learning models trained for human and facial recognition, people can be identified and tracked to a terrifyingly accurate degree. On the other hand, the information present in surveillance media can play a key role in security and law enforcement. This presents a problem of how to preserve key information without compromising the privacy of any individuals present in the video. In this research project, Computer Vision techniques and a …

Deep-Learning Based Microstructure Reconstruction And Generation, Cameron J. Maloney, Lucas Taliaferro

College of Engineering Summer Undergraduate Research Program

Characterizing the microstructural behavior of materials is crucial for understanding their properties and performance. Traditional imaging methods, such as optical microscopy and electron microscopy, are effective but costly and time-consuming. Computational approaches can reduce costs and time while expanding the accessibility of microstructural analysis through the generation of new microstructure images. Traditional computational approaches, namely descriptor-based approaches, are slow but effective in low-data scenarios. Modern approaches use machine learning (ML), which is faster but often requires a lot of data to approach the performance of descriptor-based methods. This research leverages a special data-efficient Generative Adversarial Network (GAN) architecture to artificially …

Seal Counting On Our Plages (S.C.O.O.P.), Kaanan Kharwa

Master's Theses

The Vertebrate Integrative Physiology (VIP) lab monitors the population of northern elephant seals at the largest mainland breeding colony, located at Piedras Blancas (San Simeon, CA). As the population expands, more human-seal interactions and conflicts over land use occur. The VIP lab's work informs California State Parks and helps with the management of the rookery. Currently, members of the VIP lab fly a drone over the beaches, capture multiple images, and manually count the seals, which takes around 14 to 21 hours of analysis per survey. Machine learning methods such as Convolutional Neural Networks (CNN) and Region-based Convolutional Neural Networks …

Real-Time Gun Detection In Video Streams Using Yolo V8, Harish Kumar Reddy Kunchala

Electronic Theses, Projects, and Dissertations

In this research, we advance the domain of public safety by developing a machine learning model that utilizes the YOLO v8 architecture for real-time detection of firearms in video streams. A diverse and extensive dataset, capturing a range of firearms in varying lighting and backgrounds, was meticulously assembled and preprocessed to enhance the model's adaptability to real-world scenarios. Leveraging the YOLO v8 framework, known for its real-time object detection accuracy, the model was fine-tuned to accurately identify firearms across different shapes and orientations.

The training phase capitalized on GPU computing and transfer learning to expedite the learning process while preserving …

Insights Into Cellular Evolution: Temporal Deep Learning Models And Analysis For Cell Image Classification, Xinran Zhao

Master's Theses

Understanding the temporal evolution of cells poses a significant challenge in developmental biology. This study embarks on a comparative analysis of various machine-learning techniques to classify cell colony images across different timestamps, thereby aiming to capture dynamic transitions of cellular states. By performing Transfer Learning with state-of-the-art classification networks, we achieve high accuracy in categorizing single-timestamp images. Furthermore, this research introduces the integration of temporal models, notably LSTM (Long Short Term Memory Network), R-Transformer (Recurrent Neural Network enhanced Transformer) and ViViT (Video Vision Transformer), to undertake this classification task to verify the effectiveness of incorporating temporal features into the classification …

Gender Classification Through Postural Analysis: A Comparative Study Of 2d Images And 3d Reconstructions., Prathamesh Dixit

Master's Projects

In computer vision, gender classification has become a vital task having applications in human-computer interaction, healthcare, and surveillance. In this study, we look at a two-step approach based on human joint information for gender classification. In this research, we use convolutional neural networks (CNNs).

With Leeds Sports Pose (LSP) dataset, we use a C5 pre-trained model to map and extract joint information from 2D RGB images and after pre-processing and background removal, we use PiFUHD to transform these 2D images into 3D representations. Next, we train our models on RGB images and joint images for both 2D and 3D representations. …

Hard-Hearted Scrolls: A Noninvasive Method For Reading The Herculaneum Papyri, Stephen Parsons

Theses and Dissertations--Computer Science

The Herculaneum scrolls were buried and carbonized by the eruption of Mount Vesuvius in A.D. 79 and represent the only classical library discovered in situ. Charred by the heat of the eruption, the scrolls are extremely fragile. Since their discovery two centuries ago, some scrolls have been physically opened, leading to some textual recovery but also widespread damage. Many other scrolls remain in rolled form, with unknown contents. More recently, various noninvasive methods have been attempted to reveal the hidden contents of these scrolls using advanced imaging. Unfortunately, their complex internal structure and lack of clear ink contrast has prevented …

Gesture Recognition With Deep Learning, Chaz Chang

Master's Projects

Gesture recognition is a machine learning and computer vision application where gestures are detected from videos. This project uses pose estimation to find the coordinates of important joints as a preprocessing step before trying to classify the gesture. Machine learning layers such as Convolutional Neural Network and Long Short-Term Memory are used. Various types of machine learning models are trained. The accuracy and f1 score of each model are compared. Feature selection is done by testing with different subsets of features. The results show that pose estimation as a preprocessing step provides good accuracy for gesture recognition. The results also …

Gesture Recognition Of Sign Language Alphabet Using Machine Learning Techniques, Gursimran Singh

Master's Projects

With the rising incidence of hearing loss, effective sign language recognition has become crucial for enhancing communication for individuals with hearing impairments. Traditional sensor-based recognition systems have been challenged by the complexities of realworld settings, prompting a shift toward more adaptable vision-based recognition systems. Distinct from previous studies, this work pioneers the use of ensemble methods with advanced filtering techniques on the Sign Language MNIST dataset, offering a novel perspective on sign language recognition. This research delves into the intersection of machine learning and image processing to develop a robust framework for sign language recognition. A range of filters, including …

Lung Cancer Type Classification, Mohit Ramajibhai Ankoliya

Electronic Theses, Projects, and Dissertations

Lung cancer is the third most common cancer in the U.S. This research focuses on classifying lung cancer cells based on their tumor cell, shape, and biological traits in images automatically obtained by passing through the

convolutional layers. Additionally, I classify whether the lung cell is adenocarcinoma, large cell carcinoma, squamous cell carcinoma, or normal cell carcinoma. The benefit of this classification is an accurate prognosis, leading to patients receiving proper therapy. The Lung Cancer CT(Computed Tomography) image dataset from Kaggle has been drawn with 1000 CT images of various types of lung cancer. Two state-of-the-art convolutional neural networks (CNNs) …

Camera And Lidar Fusion For Point Cloud Semantic Segmentation, Ali Abdelkader

Theses and Dissertations

Perception is a fundamental component of any autonomous driving system. Semantic segmentation is the perception task of assigning semantic class labels to sensor inputs. While autonomous driving systems are currently equipped with a suite of sensors, much focus in the literature has been on semantic segmentation of camera images only. Research in the fusion of different sensor modalities for semantic segmentation has not been investigated as much. Deep learning models based on transformer architectures have proven successful in many tasks in computer vision and natural language processing. This work explores the use of deep learning transformers to fuse information from …

An Analysis Of Camera Configurations And Depth Estimation Algorithms For Triple-Camera Computer Vision Systems, Jared Peter-Contesse

Master's Theses

The ability to accurately map and localize relevant objects surrounding a vehicle is an important task for autonomous vehicle systems. Currently, many of the environmental mapping approaches rely on the expensive LiDAR sensor. Researchers have been attempting to transition to cheaper sensors like the camera, but so far, the mapping accuracy of single-camera and dual-camera systems has not matched the accuracy of LiDAR systems. This thesis examines depth estimation algorithms and camera configurations of a triple-camera system to determine if sensor data from an additional perspective will improve the accuracy of camera-based systems. Using a synthetic dataset, the performance of …

Localization Using Convolutional Neural Networks, Shannon D. Fong

Computer Engineering

With the increased accessibility to powerful GPUs, ability to develop machine learning algorithms has increased significantly. Coupled with open source deep learning frameworks, average users are now able to experiment with convolutional neural networks (CNNs) to solve novel problems. This project sought to train a CNN capable of classifying between various locations within a building. A single continuous video was taken while standing at each desired location so that every class in the neural network was represented by a single video. Each location was given a number to be used for classification and the video was subsequently titled locX. These …

Automatic Identification Of Animals In The Wild: A Comparative Study Between C-Capsule Networks And Deep Convolutional Neural Networks., Joel Kamdem Teto, Ying Xie

Master of Science in Computer Science Theses

The evolution of machine learning and computer vision in technology has driven a lot of

improvements and innovation into several domains. We see it being applied for credit decisions, insurance quotes, malware detection, fraud detection, email composition, and any other area having enough information to allow the machine to learn patterns. Over the years the number of sensors, cameras, and cognitive pieces of equipment placed in the wilderness has been growing exponentially. However, the resources (human) to leverage these data into something meaningful are not improving at the same rate. For instance, a team of scientist volunteers took 8.4 years, …

Investigating Dataset Distinctiveness, Andrew Ulmer, Kent W. Gauen, Yung-Hsiang Lu, Zohar R. Kapach, Daniel P. Merrick

The Summer Undergraduate Research Fellowship (SURF) Symposium

Just as a human might struggle to interpret another human’s handwriting, a computer vision program might fail when asked to perform one task in two different domains. To be more specific, visualize a self-driving car as a human driver who had only ever driven on clear, sunny days, during daylight hours. This driver – the self-driving car – would inevitably face a significant challenge when asked to drive when it is violently raining or foggy during the night, putting the safety of its passengers in danger. An extensive understanding of the data we use to teach computer vision models – …

Underwater Computer Vision - Fish Recognition, Spencer Chang, Austin Otto

Computer Engineering

The Underwater Computer Vision – Fish Recognition project includes the design and implementation of a device that can withstand staying underwater for a duration of time, take pictures of underwater creatures, such as fish, and be able to identify certain fish. The system is meant to be cheap to create, yet still able to process the images it takes and identify the objects in the pictures with some accuracy. The device can output its results to another device or an end user.

Multispectral Identification Array, Zachary D. Eagan

Computer Engineering

The Multispectral Identification Array is a device for taking full image spectroscopy data via the illumination of a subject with sixty-four unique spectra. The array combines images under the illumination spectra to produce an approximate reflectance graph for every pixel in a scene. Acquisition of an entire spectrum allows the array to differentiate objects based on surface material. Spectral graphs produced are highly approximate and should not be used to determine material properties, however the output is sufficiently consistent to allow differentiation and identification of previously sampled subjects. While not sufficiently advanced for use as a replacement to spectroscopy the …

Software Updates To A Multiple Autonomous Quadcopter Search System (Maqss), Jared Speck, Toby Chan

Computer Engineering

A series of performance-based and feature implementation software updates to an existing multiple vehicle autonomous target search system is outlined in this paper. The search system, MAQSS, is designed to address a computational power constraint found on modern autonomous aerial platforms by separating real-time and computationally expensive tasks through delegation to multiple multirotor vehicles. A Ground Control Station (GCS) is also described as part of the MAQSS system to perform the delegation and provide a low workload user interface. Ultimately, the changes to MAQSS noted in this paper helped to improve the performance of the autonomous search mission, the accuracy …

Hybrid Single And Dual Pattern Structured Light Illumination, Minghao Wang

Theses and Dissertations--Electrical and Computer Engineering

Structured Light Illumination is a widely used 3D shape measurement technique in non-contact surface scanning. Multi-pattern based Structured Light Illumination methods reconstruct 3-D surface with high accuracy, but are sensitive to object motion during the pattern projection and the speed of scanning process is relatively long. To reduce this sensitivity, single pattern techniques are developed to achieve a high speed scanning process, such as Composite Pattern (CP) and Modified Composite Pattern (MCP) technique. However, most of single patter techniques have a significant banding artifact and sacrifice the accuracy. We focus on developing SLI techniques can achieve both high speed, high …

Model-Free Method Of Reinforcement Learning For Visual Tasks, Jeff S. Soldate, Jonghoon Jin, Eugenio Culurciello

The Summer Undergraduate Research Fellowship (SURF) Symposium

There has been success in recent years for neural networks in applications requiring high level intelligence such as categorization and assessment. In this work, we present a neural network model to learn control policies using reinforcement learning. It takes a raw pixel representation of the current state and outputs an approximation of a Q value function made with a neural network that represents the expected reward for each possible state-action pair. The action is chosen an \epsilon-greedy policy, choosing the highest expected reward with a small chance of random action. We used gradient descent to update the weights and biases …

Automatic Performance Level Assessment In Minimally Invasive Surgery Using Coordinated Sensors And Composite Metrics, Sami Taha Abu Snaineh

Theses and Dissertations--Computer Science

Skills assessment in Minimally Invasive Surgery (MIS) has been a challenge for training centers for a long time. The emerging maturity of camera-based systems has the potential to transform problems into solutions in many different areas, including MIS. The current evaluation techniques for assessing the performance of surgeons and trainees are direct observation, global assessments, and checklists. These techniques are mostly subjective and can, therefore, involve a margin of bias.

The current automated approaches are all implemented using mechanical or electromagnetic sensors, which suffer limitations and influence the surgeon’s motion. Thus, evaluating the skills of the MIS surgeons and trainees …

A Computer Vision Application To Accurately Estimate Object Distance, Kayton B. Parekh

Mathematics, Statistics, and Computer Science Honors Projects

Scientists have been working to create robots that perform manual work for years. However, creating machines that can navigate themselves and respond to their environment has proven to be difficult. One integral task to such research is to estimate the position of objects in the robot's visual field.

In this project we examine an implementation of computer vision depth perception. Our application uses color-based object tracking combined with model-based pose estimation to estimate the depth of specific objects in the view of our Pioneer 2 and Power Wheels robots. We use the Camshift algorithm for color-based object tracking, which uses …