Engineering Commons^™

Seal Counting On Our Plages (S.C.O.O.P.), Kaanan Kharwa

Master's Theses

The Vertebrate Integrative Physiology (VIP) lab monitors the population of northern elephant seals at the largest mainland breeding colony, located at Piedras Blancas (San Simeon, CA). As the population expands, more human-seal interactions and conflicts over land use occur. The VIP lab's work informs California State Parks and helps with the management of the rookery. Currently, members of the VIP lab fly a drone over the beaches, capture multiple images, and manually count the seals, which takes around 14 to 21 hours of analysis per survey. Machine learning methods such as Convolutional Neural Networks (CNN) and Region-based Convolutional Neural Networks …

Uav-Based Tracking And Following Of Railroad Lines, Keith Michael Lewandowski

Theses and Dissertations

Given the pivotal role of the railroad industry in modern transportation and the potential risks associated with track malfunctions, the inspection and maintenance of railroad tracks emerges as a critical concern. While existing solutions excel in performing accurate measurements and detection, they often rely on large, expensive, and time-consuming platforms for inspections. This project, however, seeks to solve the same problem with the use of an unmanned aerial vehicle (UAV), significantly reducing time and cost while maintaining detection capabilities. In particular, this solution is ideal for large-scale, high-level inspections following major events such as floods [6], hurricanes [7] or earthquakes …

Real-Time Gun Detection In Video Streams Using Yolo V8, Harish Kumar Reddy Kunchala

Electronic Theses, Projects, and Dissertations

In this research, we advance the domain of public safety by developing a machine learning model that utilizes the YOLO v8 architecture for real-time detection of firearms in video streams. A diverse and extensive dataset, capturing a range of firearms in varying lighting and backgrounds, was meticulously assembled and preprocessed to enhance the model's adaptability to real-world scenarios. Leveraging the YOLO v8 framework, known for its real-time object detection accuracy, the model was fine-tuned to accurately identify firearms across different shapes and orientations.

The training phase capitalized on GPU computing and transfer learning to expedite the learning process while preserving …

Vysion Software, Isaias Hernandez-Dominguez Jr, Chander Luderman Miller

2024 Symposium

Vision loss presents significant challenges in daily life. Existing solutions for blind and visually impaired individuals are often limited in functionality, expensive, or complex to use. Vysion Software addresses this gap by developing a user-friendly, all-in-one AI companion app that provides features including text summarization, real-time audio descriptions, and AI-enhanced navigation. This project details the development plan, initial functionalities, and future vision for Vysion Software.

Can Neural Networks Reach Human Vision Levels On Object Recognition Tasks?, Luke D. Baumel, Mikayla Cutler, Matt Hyatt, Joseph Tocco, William Friebel, Nicholas Baker Dr., George K. Thiruvathukal Dr.

Psychology: Faculty Publications and Other Works

Object recognition is a crucial function of biological vision; it allows us to draw conclusions about a visual scene that transcends the image formed by the retina. However, the task of object recognition quickly becomes a challenge when hindrances such as viewing angle, object distance from observer, illuminant qualities, and potential occlusions become active variables. Additionally, the diversity of visual features within the same category of object, coupled with the numerous contexts in which an object may be observed is demonstrative of the formidable task that is object recognition. Previous research showed a significant texture bias in Convolutional Neural Networks’ …

Insights Into Cellular Evolution: Temporal Deep Learning Models And Analysis For Cell Image Classification, Xinran Zhao

Master's Theses

Understanding the temporal evolution of cells poses a significant challenge in developmental biology. This study embarks on a comparative analysis of various machine-learning techniques to classify cell colony images across different timestamps, thereby aiming to capture dynamic transitions of cellular states. By performing Transfer Learning with state-of-the-art classification networks, we achieve high accuracy in categorizing single-timestamp images. Furthermore, this research introduces the integration of temporal models, notably LSTM (Long Short Term Memory Network), R-Transformer (Recurrent Neural Network enhanced Transformer) and ViViT (Video Vision Transformer), to undertake this classification task to verify the effectiveness of incorporating temporal features into the classification …

Towards Multi-Modal Interpretable Video Understanding, Quang Sang Truong

Graduate Theses and Dissertations

This thesis introduces an innovative approach to video comprehension, which simulates human perceptual mechanisms and establishes a comprehensible and coherent narrative representation of video content. At the core of this approach lies the creation of a Visual-Linguistic (VL) feature for an interpretable video portrayal and an adaptive attention mechanism (AAM) aimed at concentrating solely on principal actors or pertinent objects while modeling their interconnections. Taking cues from the way humans disassemble scenes into visual and non-visual constituents, the proposed VL feature characterizes a scene via three distinct modalities: (i) a global visual environment, providing a broad contextual comprehension of the …

Advanced Traffic Video Analytics For Robust Traffic Accident Detection, Hadi Ghahremannezhad

Dissertations

Automatic traffic accident detection is an important task in traffic video analysis due to its key applications in developing intelligent transportation systems. Reducing the time delay between the occurrence of an accident and the dispatch of the first responders to the scene may help lower the mortality rate and save lives. Since 1980, many approaches have been presented for the automatic detection of incidents in traffic videos. In this dissertation, some challenging problems for accident detection in traffic videos are discussed and a new framework is presented in order to automatically detect single-vehicle and intersection traffic accidents in real-time.

First, …

Insect Classification And Explainability From Image Data Via Deep Learning Techniques, Tanvir Hossain Bhuiyan

USF Tampa Graduate Theses and Dissertations

Since the dawn of the Industrial Revolution, humanity has always tried to make labor more efficient and automated, and this trend is only continuing in the modern digital age. With the advent of artificial intelligence (AI) techniques in the latter part of the 20th century, the speed and scale with which AI has been leveraged to automate tasks defy human imagination. Many people deeply entrenched in the technology field are genuinely intrigued and concerned about how AI may change many of the ways in which humans have been living for millennia. Only time will provide the answers. This dissertation is …

Biologically Inspired Multi-Robot System Based On Wolf Hunting Behavior, Zachary Hinnen, Chance Hamilton, Alfredo Weitzenfeld

36th Florida Conference on Recent Advances in Robotics

Studies involving the group predator behavior of wolves have inspired multiple robotic architectures to mimic these biological behaviors in their designs and research. In this work, we aim to use robotic systems to mimic wolf packs' single and group behavior. This work aims to extend the original research by Weitzenfeld et al [7] and evaluate under a new multi-robot robot system architecture. The multiple robot architecture includes a 'Prey' pursued by a wolf pack consisting of an 'Alpha' and 'Beta' robotic group. The Alpha Wolf' will be the group leader, searching and tracking the 'Prey.' At the same time, the …

A Human-In-The-Loop Robot Grasping System With Grasp Quality Refinement, Tian Tan

USF Tampa Graduate Theses and Dissertations

The goal of this dissertation is to develop a grasping system for assistive robots that can help people with disabilities and the elderly to perform tasks of daily living. In developing this robot grasping system, we maximize its reliability, accuracy, and autonomy. High reliability and accuracy are required for robots to perform tasks around human users and to safely interact with objects that might be fragile or have contents that could spill. High autonomy is desired as users with disabilities are usually not dexterous enough to directly operate the robot. In this dissertation, a human-in-the-loop (HitL) robot grasping system is …

Ai Applications On Planetary Rovers, Alexis David Pascual

Electronic Thesis and Dissertation Repository

The rise in the number of robotic missions to space is paving the way for the use of artificial intelligence and machine learning in the autonomy and augmentation of rover operations. For one, more rovers mean more images, and more images mean more data bandwidth required for downlinking as well as more mental bandwidth for analyzing the images. On the other hand, light-weight, low-powered microrover platforms are being developed to accommodate the drive for planetary exploration. As a result of the mass and power constraints, these microrover platforms will not carry typical navigational instruments like a stereocamera or a laser …

Hard-Hearted Scrolls: A Noninvasive Method For Reading The Herculaneum Papyri, Stephen Parsons

Theses and Dissertations--Computer Science

The Herculaneum scrolls were buried and carbonized by the eruption of Mount Vesuvius in A.D. 79 and represent the only classical library discovered in situ. Charred by the heat of the eruption, the scrolls are extremely fragile. Since their discovery two centuries ago, some scrolls have been physically opened, leading to some textual recovery but also widespread damage. Many other scrolls remain in rolled form, with unknown contents. More recently, various noninvasive methods have been attempted to reveal the hidden contents of these scrolls using advanced imaging. Unfortunately, their complex internal structure and lack of clear ink contrast has prevented …

Lung Cancer Type Classification, Mohit Ramajibhai Ankoliya

Electronic Theses, Projects, and Dissertations

Lung cancer is the third most common cancer in the U.S. This research focuses on classifying lung cancer cells based on their tumor cell, shape, and biological traits in images automatically obtained by passing through the

convolutional layers. Additionally, I classify whether the lung cell is adenocarcinoma, large cell carcinoma, squamous cell carcinoma, or normal cell carcinoma. The benefit of this classification is an accurate prognosis, leading to patients receiving proper therapy. The Lung Cancer CT(Computed Tomography) image dataset from Kaggle has been drawn with 1000 CT images of various types of lung cancer. Two state-of-the-art convolutional neural networks (CNNs) …

Generative Spatio-Temporal And Multimodal Analysis Of Neonatal Pain, Md Sirajus Salekin

USF Tampa Graduate Theses and Dissertations

Neonates can not express their pain like an adult person. Due to the lacking of proper muscle growth and inability to express non-verbally, it is difficult to understand their emotional status. In addition, if the neonates are under any treatment or left monitored after any major surgeries (post-operative), it is more difficult to understand their pain due to the side effect of medications and the caring system (i.e. intubated, masked face, covered body with blanket, etc.). In a clinical environment, usually, bedside nurses routinely observe the neonate and measure the pain status following any standard clinical pain scale. But current …

Softskip: Empowering Multi-Modal Dynamic Pruning For Single-Stage Referring Comprehension, Dulanga Weerakoon, Vigneshwaran Subbaraju, Tuan Tran, Archan Misra

Research Collection School Of Computing and Information Systems

Supporting real-time referring expression comprehension (REC) on pervasive devices is an important capability for human-AI collaborative tasks. Model pruning techniques, applied to DNN models, can enable real-time execution even on resource-constrained devices. However, existing pruning strategies are designed principally for uni-modal applications, and suffer a significant loss of accuracy when applied to REC tasks that require fusion of textual and visual inputs. We thus present a multi-modal pruning model, LGMDP, which uses language as a pivot to dynamically and judiciously select the relevant computational blocks that need to be executed. LGMDP also introduces a new SoftSkip mechanism, whereby 'skipped' visual …

Learning Domain Invariant Information To Enhance Presentation Attack Detection In Visible Face Recognition Systems, Jennifer Hamblin

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

Face signatures, including size, shape, texture, skin tone, eye color, appearance, and scars/marks, are widely used as discriminative, biometric information for access control. Despite recent advancements in facial recognition systems, presentation attacks on facial recognition systems have become increasingly sophisticated. The ability to detect presentation attacks or spoofing attempts is a pressing concern for the integrity, security, and trust of facial recognition systems. Multi-spectral imaging has been previously introduced as a way to improve presentation attack detection by utilizing sensors that are sensitive to different regions of the electromagnetic spectrum (e.g., visible, near infrared, long-wave infrared). Although multi-spectral presentation attack …

Conditional Variational Autoencoder (Cvae) For The Augmentation Of Ecl Biosensor Data, Matthew Dulcich

Honors Theses

Machine Learning (ML) is vastly improving the world, from computer vision to fully self-driving cars, we are now able accomplish objectives that were thought to only be dreams. In order to train ML models accurately, they require mountains of information to work with, but sometimes it becomes impossible to collect the data needed, so we turn to data augmentation. In this project we use a conditional variational auto encoder to supplement the original video electrochemiluminescence biosensor dataset, in order to increase the accuracy of a future classification model. In other words, using a cVAE we will create unique realistic videos …

Humanoid Robot Motion Control For Ramps And Stairs, Tommy Truong

USF Tampa Graduate Theses and Dissertations

Humanoid robot research and development have been an ongoing effort since the 1900sand can be broken down to two problems. A mechanical problem, getting a humanoid robot to move human-like or a software problem, getting a humanoid robot to behave human-like. These problems of moving and behaving human-like can be often solved using control theory as research advances. For the premise of this research, we explore how to balance and walk on non-flat terrain for the humanoid robot Darwin-Op. Since the focus was on the control theory, the vision control to detect the non-flat terrain was a side objective. The …

Camera And Lidar Fusion For Point Cloud Semantic Segmentation, Ali Abdelkader

Theses and Dissertations

Perception is a fundamental component of any autonomous driving system. Semantic segmentation is the perception task of assigning semantic class labels to sensor inputs. While autonomous driving systems are currently equipped with a suite of sensors, much focus in the literature has been on semantic segmentation of camera images only. Research in the fusion of different sensor modalities for semantic segmentation has not been investigated as much. Deep learning models based on transformer architectures have proven successful in many tasks in computer vision and natural language processing. This work explores the use of deep learning transformers to fuse information from …

Computer Vision Based Classification Of Fruits And Vegetables For Self-Checkout At Supermarkets, Khurram Hameed

Theses: Doctorates and Masters

The field of machine learning, and, in particular, methods to improve the capability of machines to perform a wider variety of generalised tasks are among the most rapidly growing research areas in today’s world. The current applications of machine learning and artificial intelligence can be divided into many significant fields namely computer vision, data sciences, real time analytics and Natural Language Processing (NLP). All these applications are being used to help computer based systems to operate more usefully in everyday contexts. Computer vision research is currently active in a wide range of areas such as the development of autonomous vehicles, …

An Analysis Of Camera Configurations And Depth Estimation Algorithms For Triple-Camera Computer Vision Systems, Jared Peter-Contesse

Master's Theses

The ability to accurately map and localize relevant objects surrounding a vehicle is an important task for autonomous vehicle systems. Currently, many of the environmental mapping approaches rely on the expensive LiDAR sensor. Researchers have been attempting to transition to cheaper sensors like the camera, but so far, the mapping accuracy of single-camera and dual-camera systems has not matched the accuracy of LiDAR systems. This thesis examines depth estimation algorithms and camera configurations of a triple-camera system to determine if sensor data from an additional perspective will improve the accuracy of camera-based systems. Using a synthetic dataset, the performance of …

Forecasting Pedestrian Trajectory Using Deep Learning, Arsal Syed

UNLV Theses, Dissertations, Professional Papers, and Capstones

In this dissertation we develop different methods for forecasting pedestrian trajectories. Complete understanding of pedestrian motion is essential for autonomous agents and social robots to make realistic and safe decisions. Current trajectory prediction methods rely on incorporating historic motion, scene features and social interaction to model pedestrian behaviors. Our focus is to accurately understand scene semantics to better forecast trajectories. In order to do so, we leverage semantic segmentation to encode static scene features such as walkable paths, entry/exits, static obstacles etc. We further evaluate the effectiveness of using semantic maps on different datasets and compare its performance with already …

Analysis Of Microscopic Objects Using Computer Vision Methods, Yuan Dao

UNLV Theses, Dissertations, Professional Papers, and Capstones

As an essential and powerful tool to observe living organisms, three-dimensional fluorescence microscopy is widely used in biological research and diagnosis. The 4D fluorescence microscopy data can be obtained using time-lapsed videos of 3D images. To analyze and extract useful information from the increasingly large and complex biological image dataset, efficient and effective computational tools are in need but still lagging behind. In analyzing biological data, two major challenges are faced. First, time-lapsed fluorescence microscopic images typically have a low SNR. Second, biological objects often change their morphology and internal structure frequently. As such, conventional image processing methods may not …

Robot Object Detection And Locomotion Demonstration For Eecs Department Tours, Bryson Howell, Ethan Haworth, Chris Mobley, Ian Mulet

Chancellor’s Honors Program Projects

No abstract provided.

Analysis Of Hardware Accelerated Deep Learning And The Effects Of Degradation On Performance, Samuel C. Leach

Masters Theses

As convolutional neural networks become more prevalent in research and real world applications, the need for them to be faster and more robust will be a constant battle. This thesis investigates the effect of degradation being introduced to an image prior to object recognition with a convolutional neural network. As well as experimenting with methods to reduce the degradation and improve performance. Gaussian smoothing and additive Gaussian noise are both analyzed degradation models within this thesis and are reduced with Gaussian and Butterworth masks using unsharp masking and smoothing, respectively. The results show that each degradation is disruptive to the …

Ping Pong Ball Collecting Robot, Gina Lanese, Jason Colonna, Sarah Kuchcinski, Johndavid Rogers

Williams Honors College, Honors Research Projects

The ping pong ball collecting robot will collect ping pong balls in a ping pong arena, such as after a game or practice session. Ball collection is now typically done by a human, collecting the balls either by hand or with a net. A robot, autonomously collecting the ping pong balls from the floor, would allow for the time spent playing or practicing to be maximized. The team will design the software and hardware for the robot, including the mechanical system, the software, and circuit boards. This robot will serve as a capstone project for the senior design team consisting …

Self && Self, Shuang Cai

Senior Projects Spring 2021

Seldom before the COVID-19 pandemic have so many people simultaneously had their lifestyle drastically changed in the same way. The forced physical isolation is, ironically, a communal experience. The sickening quarantine left everyone nothing but time to confront and reconnect with themselves. Another inevitable result of corporal isolation is the predominant awakening awareness of digital existences and connections. Evoking the shared sensitivity and delicacy, studying the tectonic activity of the digital world, the project documents the endured contemplation in the upcoming resurgence.

Ppmexplorer: Using Information Retrieval, Computer Vision And Transfer Learning Methods To Index And Explore Images Of Pompeii, Cindy Roullet

Graduate Theses and Dissertations

In this dissertation, we present and analyze the technology used in the making of PPMExplorer: Search, Find, and Explore Pompeii. PPMExplorer is a software tool made with data extracted from the Pompei: Pitture e Mosaic (PPM) volumes. PPM is a valuable set of volumes containing 20,000 historical annotated images of the archaeological site of Pompeii, Italy accompanied by extensive captions. We transformed the volumes from paper, to digital, to searchable. PPMExplorer enables archaeologist researchers to conduct and check hypotheses on historical findings. We present a theory that such a concept is possible by leveraging computer generated correlations between artifacts using …

Camera Placement Meeting Restrictions Of Computer Vision, Sara Aghajanzadeh, Roopasree Naidu, Shuo-Han Chen, Caleb Tung, Abhinav Goel, Yung-Hsiang Lu, George K. Thiruvathukal

Computer Science: Faculty Publications and Other Works

In the blooming era of smart edge devices, surveillance cam- eras have been deployed in many locations. Surveillance cam- eras are most useful when they are spaced out to maximize coverage of an area. However, deciding where to place cam- eras is an NP-hard problem and researchers have proposed heuristic solutions. Existing work does not consider a signifi- cant restriction of computer vision: in order to track a moving object, the object must occupy enough pixels. The number of pixels depends on many factors (how far away is the object? What is the camera resolution? What is the focal length?). …