Engineering Commons^™

Towards Multi-Modal Interpretable Video Understanding, Quang Sang Truong

Graduate Theses and Dissertations

This thesis introduces an innovative approach to video comprehension, which simulates human perceptual mechanisms and establishes a comprehensible and coherent narrative representation of video content. At the core of this approach lies the creation of a Visual-Linguistic (VL) feature for an interpretable video portrayal and an adaptive attention mechanism (AAM) aimed at concentrating solely on principal actors or pertinent objects while modeling their interconnections. Taking cues from the way humans disassemble scenes into visual and non-visual constituents, the proposed VL feature characterizes a scene via three distinct modalities: (i) a global visual environment, providing a broad contextual comprehension of the …

Real-Time Object Tracking With Yolov5 And Recurrent Network On A Hardware Platform, Mohammed Abdulhakeem Alameri

Theses

The advancement of computer vision, particularly in the domain of object tracking, involves the integration of conventional feature-based approaches with contemporary deep learning methodologies. The initial phase of object detection plays a fundamental role in generating prospective objects for further tracking, and its level of success has a direct influence on the overall effectiveness of the tracking process. Many of the methods used to generate candidate objects start with object detection, and then object tracking algorithms are developed to link the object instances together to create the trajectories.
The primary difficulties in object tracking pertain to the establishment of reliable …

Smartphone Based Object Detection For Shark Spotting, Darrick W. Oliver

Master's Theses

Given concern over shark attacks in coastal regions, the recent use of unmanned aerial vehicles (UAVs), or drones, has increased to ensure the safety of beachgoers. However, much of city officials' process remains manual, with drone operation and review of footage still playing a significant role. In pursuit of a more automated solution, researchers have turned to the usage of neural networks to perform detection of sharks and other marine life. For on-device solutions, this has historically required assembling individual hardware components to form an embedded system to utilize the machine learning model. This means that the camera, neural processing …

Hardware-In-The-Loop Reaction Wheel Testbed With Camera Vision, Abigail Romero, Harvey Perkins, Stephen Kwok-Choon

College of Engineering Summer Undergraduate Research Program

Reaction wheels are widely used in aerospace systems as a method of attitude control. This research was focused on the design, development, and testing of a hardware-in-the-loop reaction wheel testbed that can be used for research and teaching applications related to satellite navigation and control. This project successfully utilized commercial off-the-shelf components to develop a reaction wheel capable of controlling the orientation of a freely rotating platform, as well as tracking objects using computer vision.

Advanced Traffic Video Analytics For Robust Traffic Accident Detection, Hadi Ghahremannezhad

Dissertations

Automatic traffic accident detection is an important task in traffic video analysis due to its key applications in developing intelligent transportation systems. Reducing the time delay between the occurrence of an accident and the dispatch of the first responders to the scene may help lower the mortality rate and save lives. Since 1980, many approaches have been presented for the automatic detection of incidents in traffic videos. In this dissertation, some challenging problems for accident detection in traffic videos are discussed and a new framework is presented in order to automatically detect single-vehicle and intersection traffic accidents in real-time.

First, …

Enhancing Human Key Point Identification: A Comparative Study Of High-Resolution Vicon Dataset And Coco Dataset Using Bpnet, Bibash Lama

Masters Theses

Accurately identifying human key points is crucial for various applications, including activity recognition, pose estimation, and gait analysis. This study presents a high-resolution dataset created using the VICON motion capture system and three differently oriented 2D cameras, that can be used to train different neural networks for estimating the 2D key joint positions of the person from the 2D images or videos. The participants in the study included 25 healthy adults (17 males and 8 females) performing normal gait movements for about 2 to 3 seconds. The VICON system captured 3D ground truth data, while the three 2D cameras collected …

Insect Classification And Explainability From Image Data Via Deep Learning Techniques, Tanvir Hossain Bhuiyan

USF Tampa Graduate Theses and Dissertations

Since the dawn of the Industrial Revolution, humanity has always tried to make labor more efficient and automated, and this trend is only continuing in the modern digital age. With the advent of artificial intelligence (AI) techniques in the latter part of the 20th century, the speed and scale with which AI has been leveraged to automate tasks defy human imagination. Many people deeply entrenched in the technology field are genuinely intrigued and concerned about how AI may change many of the ways in which humans have been living for millennia. Only time will provide the answers. This dissertation is …

Experimental Characterization And Computer Vision-Assisted Detection Of Pitting Corrosion On Stainless Steel Structural Members, Riley J. Muehler

Master's Theses

Pitting corrosion is a prevalent form of corrosive damage that can weaken, damage, and initiate failure in corrosion-resistant metallic materials. For instance, 304 stainless steel is commonly utilized in various structures (e.g., miter gates, heat exchangers, and storage tanks), but is prone to failure through pitting corrosion and stress corrosion cracking under mechanical loading, regardless of its high corrosion resistance. In this study, to better understand the pitting corrosion damage development, controlled corrosion experiments were conducted to generate pits on 304 stainless steel specimens with and without mechanical loading. The pit development over time was characterized using a high-resolution laser …

Biologically Inspired Multi-Robot System Based On Wolf Hunting Behavior, Zachary Hinnen, Chance Hamilton, Alfredo Weitzenfeld

36th Florida Conference on Recent Advances in Robotics

Studies involving the group predator behavior of wolves have inspired multiple robotic architectures to mimic these biological behaviors in their designs and research. In this work, we aim to use robotic systems to mimic wolf packs' single and group behavior. This work aims to extend the original research by Weitzenfeld et al [7] and evaluate under a new multi-robot robot system architecture. The multiple robot architecture includes a 'Prey' pursued by a wolf pack consisting of an 'Alpha' and 'Beta' robotic group. The Alpha Wolf' will be the group leader, searching and tracking the 'Prey.' At the same time, the …

Compliant Robot Manipulator For Transurethral Diagnosis, Samson A. Adejokun

Mechanical and Aerospace Engineering Dissertations

ABSTRACT: Minimally invasive procedures employ continuum manipulators, however, the internal human anatomy presents challenges relating to size, dexterity, and workspace for these manipulators. This research presents modeling, kinematic analysis, limited prototyping, and characterization of a micro-robotic manipulator to assess viscoelastic tissue properties of the bladder through transurethral palpation towards the diagnosis of bladder dysfunction, urinary incontinence, and early-stage bladder cancer. The proposed micro-robot consists of two subsystems; a unique 4mm outer diameter tendon-driven continuum segment from assembled “vertebrae” components with male and female snap-in features for joint assembly, an elastic tube encompassing each joint for controlled compliance and structural integrity …

A Human-In-The-Loop Robot Grasping System With Grasp Quality Refinement, Tian Tan

USF Tampa Graduate Theses and Dissertations

The goal of this dissertation is to develop a grasping system for assistive robots that can help people with disabilities and the elderly to perform tasks of daily living. In developing this robot grasping system, we maximize its reliability, accuracy, and autonomy. High reliability and accuracy are required for robots to perform tasks around human users and to safely interact with objects that might be fragile or have contents that could spill. High autonomy is desired as users with disabilities are usually not dexterous enough to directly operate the robot. In this dissertation, a human-in-the-loop (HitL) robot grasping system is …

Ai Applications On Planetary Rovers, Alexis David Pascual

Electronic Thesis and Dissertation Repository

The rise in the number of robotic missions to space is paving the way for the use of artificial intelligence and machine learning in the autonomy and augmentation of rover operations. For one, more rovers mean more images, and more images mean more data bandwidth required for downlinking as well as more mental bandwidth for analyzing the images. On the other hand, light-weight, low-powered microrover platforms are being developed to accommodate the drive for planetary exploration. As a result of the mass and power constraints, these microrover platforms will not carry typical navigational instruments like a stereocamera or a laser …

Hard-Hearted Scrolls: A Noninvasive Method For Reading The Herculaneum Papyri, Stephen Parsons

Theses and Dissertations--Computer Science

The Herculaneum scrolls were buried and carbonized by the eruption of Mount Vesuvius in A.D. 79 and represent the only classical library discovered in situ. Charred by the heat of the eruption, the scrolls are extremely fragile. Since their discovery two centuries ago, some scrolls have been physically opened, leading to some textual recovery but also widespread damage. Many other scrolls remain in rolled form, with unknown contents. More recently, various noninvasive methods have been attempted to reveal the hidden contents of these scrolls using advanced imaging. Unfortunately, their complex internal structure and lack of clear ink contrast has prevented …

An Adaptive Multiple-Object Tracking Architecture For Long-Duration Videos With Variable Target Density, Joachim Lohn-Jaramillo

Dartmouth College Ph.D Dissertations

Multiple-Object Tracking (MOT) methods are used to detect targets in individual video frames, e.g., vehicles, people, and other objects, and then record each unique target’s path over time. Current state-of-the-art approaches are extremely complex because most rely on extracting and comparing visual features at every frame to track each object. These approaches are geared toward high-difficulty-tracking scenarios, e.g., crowded airports, and require expensive dedicated hardware, e.g., Graphics Processing Units. In hardware-constrained applications, researchers are turning to older, less complex MOT methods, which reveals a serious scalability issue within the state-of-the-art. Crowded environments are a niche application for MOT, i.e., there …