Physical Sciences and Mathematics Commons^™

Context In Computer Vision: A Taxonomy, Multi-Stage Integration, And A General Framework, Xuan Wang

Dissertations, Theses, and Capstone Projects

Contextual information has been widely used in many computer vision tasks, such as object detection, video action detection, image classification, etc. Recognizing a single object or action out of context could be sometimes very challenging, and context information may help improve the understanding of a scene or an event greatly. However, existing approaches design specific contextual information mechanisms for different detection tasks.

In this research, we first present a comprehensive survey of context understanding in computer vision, with a taxonomy to describe context in different types and levels. Then we proposed MultiCLU, a new multi-stage context learning and utilization framework, …

Crash Detecting System Using Deep Learning, Yogesh Reddy Muddam

Electronic Theses, Projects, and Dissertations

Accidents pose a significant risk to both individual and property safety, requiring effective detection and response systems. This work introduces an accident detection system using a convolutional neural network (CNN), which provides an impressive accuracy of 86.40%. Trained on diverse data sets of images and videos from various online sources, the model exhibits complex accident detection and classification and is known for its prowess in image classification and visualization.

CNN ensures better accident detection in various scenarios and road conditions. This example shows its adaptability to a real-world accident scenario and enhances its effectiveness in detecting early events. A key …

Pedestrian Pathing Prediction Using Complex Contextual Behavioral Data In High Foot Traffic Settings, Laurel Bingham

All Graduate Theses and Dissertations, Fall 2023 to Present

Ensuring the safe integration of autonomous vehicles into real-world environments requires a comprehensive understanding of pedestrian behavior. This study addresses the challenge of predicting the movement and crossing intentions of pedestrians, a crucial aspect in the development of fully autonomous vehicles.

The research focuses on leveraging Honda's TITAN dataset, comprising 700 unique clips captured by moving vehicles in high-foot-traffic areas of Tokyo, Japan. Each clip provides detailed contextual information, including human-labeled tags for individuals and vehicles, encompassing attributes such as age, motion status, and communicative actions. Long Short-Term Memory (LSTM) networks were employed and trained on various combinations of contextual …

Evaluation Of An End-To-End Radiotherapy Treatment Planning Pipeline For Prostate Cancer, Mohammad Daniel El Basha, Court Laurence, Carlos Eduardo Cardenas, Julianne Pollard-Larkin, Steven Frank, David T. Fuentes, Falk Poenisch, Zhiqian H. Yu

Dissertations & Theses (Open Access)

Radiation treatment planning is a crucial and time-intensive process in radiation therapy. This planning involves carefully designing a treatment regimen tailored to a patient’s specific condition, including the type, location, and size of the tumor with reference to surrounding healthy tissues. For prostate cancer, this tumor may be either local, locally advanced with extracapsular involvement, or extend into the pelvic lymph node chain. Automating essential parts of this process would allow for the rapid development of effective treatment plans and better plan optimization to enhance tumor control for better outcomes.

The first objective of this work, to automate the treatment …

Detection And Classification Of Diabetic Retinopathy Using Deep Learning Models, Aishat Olatunji

Electronic Theses and Dissertations

Healthcare analytics leverages extensive patient data for data-driven decision-making, enhancing patient care and results. Diabetic Retinopathy (DR), a complication of diabetes, stems from damage to the retina’s blood vessels. It can affect both type 1 and type 2 diabetes patients. Ophthalmologists employ retinal images for accurate DR diagnosis and severity assessment. Early detection is crucial for preserving vision and minimizing risks. In this context, we utilized a Kaggle dataset containing patient retinal images, employing Python’s versatile tools. Our research focuses on DR detection using deep learning techniques. We used a publicly available dataset to apply our proposed neural network and …

Deep Learning In Indus Valley Script Digitization, Deva Munikanta Reddy Atturu

Theses and Dissertations

This research introduces ASR-net(Ancient Script Recognition), a groundbreaking system that automatically digitizes ancient Indus seals by converting them into coded text, similar to Optical Character Recognition for modern languages. ASR-net, with an 95% success rate in identifying individual symbols, aims to address the crucial need for automated techniques in deciphering the enigmatic Indus script. Initially Yolov3 is utilized to create the bounding boxes around each graphemes present in the Indus Valley Seal. In addition to that we created M-net(Mahadevan) model to encode the graphemes. Beyond digitization, the paper proposes a new research challenge called the Motif Identification Problem (MIP) related …

Blueberry Drone Ai: Estimating Crop Yield Using Deep Learning & Smart Drones, Luke Tonon, Brandon Mchenry, Anthony Thompson, Harper Zappone, Jacob Green, Hieu Nguyen, Thanh Nguyen

STEM Student Research Symposium Posters

This project seeks to assist blueberry growers in New Jersey estimate crop yield by developing software that allows autonomous drones to capture aerial images of blueberry bushes in the field, perform berry count, and identify blueberry conditions using deep learning models & computer vision.

Extracting Dnn Architectures Via Runtime Profiling On Mobile Gpus, Dong Hyub Kim

Masters Theses

Due to significant investment, research, and development efforts over the past decade, deep neural networks (DNNs) have achieved notable advancements in classification and regression domains. As a result, DNNs are considered valuable intellectual property for artificial intelligence providers. Prior work has demonstrated highly effective model extraction attacks which steal a DNN, dismantling the provider’s business model and paving the way for unethical or malicious activities, such as misuse of personal data, safety risks in critical systems, or spreading misinformation. This thesis explores the feasibility of model extraction attacks on mobile devices using aggregated runtime profiles as a side-channel to leak …

Functional Data Learning Using Convolutional Neural Networks, Jose Galarza, Tamer Oraby

School of Mathematical and Statistical Sciences Faculty Publications and Presentations

In this paper, we show how convolutional neural networks (CNNs) can be used in regression and classification learning problems for noisy and non-noisy functional data (FD). The main idea is to transform the FD into a 28 by 28 image. We use a specific but typical architecture of a CNN to perform all the regression exercises of parameter estimation and functional form classification. First, we use some functional case studies of FD with and without random noise to showcase the strength of the new method. In particular, we use it to estimate exponential growth and decay rates, the bandwidths of …

Single-Class Target-Specific Attack Against Interpretable Deep Learning Systems, Eldor Abdukhamidov, Mohammed Abuhamad, George K. Thiruvathukal, Hyoungshick Kim, Tamer Abuhmed

Computer Science: Faculty Publications and Other Works

In this paper, we present a novel Single-class target-specific Adversarial attack called SingleADV. The goal of SingleADV is to generate a universal perturbation that deceives the target model into confusing a specific category of objects with a target category while ensuring highly relevant and accurate interpretations. The universal perturbation is stochastically and iteratively optimized by minimizing the adversarial loss that is designed to consider both the classifier and interpreter costs in targeted and non-targeted categories. In this optimization framework, ruled by the first- and second-moment estimations, the desired loss surface promotes high confidence and interpretation score of adversarial samples. By …

Deep Learning Applications On Ionospheric Studies, Yang Pan, Mingwu Jin, Yue Deng

Physics Dissertations

Machine learning techniques, particularly deep learning techniques, have been vigorously pursued to tackle space physics problems and achieved some impressive results recently. The growth of deep learning technologies in different domains enables innovative solutions to those problems compared to conventional methods. Filling data gaps in instrumental observations is among the demanding issues, which benefits space physicists to study ionospheric phenomena with complete data coverage. Global total electron content (TEC) and regional ionospheric electron density (Ne) are among important physical parameters in ionospheric studies. Due to the limited coverage of global navigation satellite system (GNSS) ground receivers and sporadic …

Smart Applications And Resource Management In Internet Of Things, Zeinab Akhavan

Computer Science ETDs

Internet of Things (IoT) technologies are currently the principal solutions driving smart cities. These new technologies such as Cyber Physical Systems, 5G and data analytic have emerged to address various cities' infrastructure issues ranging from transportation and energy management to healthcare systems. An IoT setting primarily consists of a wide range of users and devices as a massive network interacting with different layers of the city infrastructure resulting in generating sheer volume of data to enable smart city services. The goal of smart city services is to create value for the entire ecosystem, whether this is health, education, transportation, energy, …

Utilizing Multitask Transfer Learning For Sonographic Rheumatoid Arthritis Synovitis Grading, Jordan Marie Claire Sanders

Doctoral Dissertations and Master's Theses

Classifying the four sonographic Rheumatoid Arthritis (RA) synovitis grades (Grade 0, Grade 1, Grade 2, and Grade 3) is a difficult problem due to the complexity of the relevant markers. Therefore, the current research proposes a Multitask Transfer Learning (MTL) framework for sonographic RA synovitis grading of Ultrasound (US) images in Brightness mode (B-Mode) and Power Doppler mode.

In the medical community, the lack of reliability of scoring these images has been an issue and reason for concern for doctors and other medical practitioners. The human/machine variability across the acquisition procedure of these US images creates an additional challenge that …

Deep Learning Approaches For Chaotic Dynamics And High-Resolution Weather Simulations In The Us Midwest, Vlada Volyanskaya, Kabir Batra, Shubham Shrivastava

Discovery Undergraduate Interdisciplinary Research Internship

Weather prediction is indispensable across various sectors, from agriculture to disaster forecasting, deeply influencing daily life and work. Recent advancement of AI foundation models for weather and climate predictions makes it possible to perform a large number of predictions in reasonable time to support timesensitive policy- and decision-making. However, the uncertainty quantification, validation, and attribution of these models have not been well explored, and the lack of knowledge can eventually hinder the improvement of their prediction accuracy and precision. Our project is embarking on a two-fold approach leveraging deep learning techniques (LSTM and Transformer) architectures. Firstly, we model the Lorenz …

Generalized Differentiable Neural Architecture Search With Performance And Stability Improvements, Emily J. Herron

Doctoral Dissertations

This work introduces improvements to the stability and generalizability of Cyclic DARTS (CDARTS). CDARTS is a Differentiable Architecture Search (DARTS)-based approach to neural architecture search (NAS) that uses a cyclic feedback mechanism to train search and evaluation networks concurrently, thereby optimizing the search process by enforcing that the networks produce similar outputs. However, the dissimilarity between the loss functions used by the evaluation networks during the search and retraining phases results in a search-phase evaluation network, a sub-optimal proxy for the final evaluation network utilized during retraining. ICDARTS, a revised algorithm that reformulates the search phase loss functions to ensure …

Context-Aware Temporal Embeddings For Text And Video Data, Ahnaf Farhan

Open Access Theses & Dissertations

Recent years have seen an exponential increase in unstructured data, primarily in the form of text, images, and videos. Extracting useful features and trends from large-scale unstructured datasets -- such as news outlets, scientific papers, and videos like security cameras or body cam recordings -- is faced with substantial challenges of volume, scalability, complexity, and semantic understanding. In analyzing trends, comprehending the temporal context is vital for uncovering patterns and narratives that are not apparent from a single video frame or text document. Despite its importance, many existing data mining and machine learning approaches overlook extracting evolutionary contextual features in …

Deep Learning For Photovoltaic Characterization, Adrian Manuel De Luis Garcia

Graduate Theses and Dissertations

This thesis introduces a novel approach to Photovoltaic (PV) installation segmentation by proposing a new architecture to understand and identify PV modules from overhead imagery. Pivotal to this concept is the creation of a new Transformer-based network, S3Former, which focuses on small object characterization and modelling intra- and inter- object differentiation inside an image. Accurate mapping of PV installations is pivotal for understanding their adoption and guiding energy policy decisions. Drawing insights from current Deep Learning methodologies for image segmentation and building upon State-of-the-Art (SOTA) techniques in solar cell mapping, this work puts forth S3Former with the following enhancements: 1. …

Domain Specific Feature Representation Learning For Diverse Temporal Data, Farhan Asif Chowdhury

Computer Science ETDs

Humans can leverage domain context to recognize novel patterns and categories based on limited known examples. In contrast, computational learning methods are not adept at exploiting context and require sufficient labeled examples to achieve similar accuracy. Many temporal data domain, for example, seismic signals and oil mining sensor data, requires domain expert annotation, which is both costly and time-consuming. The dependency on training data limits the applicability of machine learning algorithms for domains with limited labeled data. This dissertation aims to address this gap by developing temporal mining algorithms that exploit domain context to learn discriminative feature representation from limited …

Machine Learning Modeling Of Polymer Coating Formulations: Benchmark Of Feature Representation Schemes, Nelson I. Evbarunegbe

Masters Theses

Polymer coatings offer a wide range of benefits across various industries, playing a crucial role in product protection and extension of shelf life. However, formulating them can be a non-trivial task given the multitude of variables and factors involved in the production process, rendering it a complex, high-dimensional problem. To tackle this problem, machine learning (ML) has emerged as a promising tool, showing considerable potential in enhancing various polymer and chemistry-based applications, particularly those dealing with high dimensional complexities.

Our research aims to develop a physics-guided ML approach to facilitate the formulations of polymer coatings. As the first step, this …

Deciphering Trends And Tactics: Data-Driven Techniques For Forecasting Information Spread And Detecting Coordinated Campaigns In Social Media, Kin Wai Ng Lugo

USF Tampa Graduate Theses and Dissertations

The main objective of this dissertation is to develop models that predict and investigate the spread of information in social media over time. In this context, we consider topics of discussions as the information that spreads. Thus, we are interested in forecasting the number of messages per day in a future interval of time. We take a data-driven approach, in which we compare our results with real datasets from a multitude of socio-political contexts and from multiple social media platforms, specifically, Twitter and YouTube.

We identified a number of challenges related to forecasting social media time series per topic. First, …

Optimizing Uncertainty Quantification Of Vision Transformers In Deep Learning On Novel Ai Architectures, Erik Pautsch, John Li, Silvio Rizzi, George K. Thiruvathukal, Maria Pantoja

Computer Science: Faculty Publications and Other Works

Deep Learning (DL) methods have shown substantial efficacy in computer vision (CV) and natural language processing (NLP). Despite their proficiency, the inconsistency in input data distributions can compromise prediction reliability. This study mitigates this issue by introducing uncertainty evaluations in DL models, thereby enhancing dependability through a distribution of predictions. Our focus lies on the Vision Transformer (ViT), a DL model that harmonizes both local and global behavior. We conduct extensive experiments on the ImageNet-1K dataset, a vast resource with over a million images across 1,000 categories. ViTs, while competitive, are vulnerable to adversarial attacks, making uncertainty estimation crucial for …

Evaluating Methods For Improving Dnn Robustness Against Adversarial Attacks, Laureano Griffin

USF Tampa Graduate Theses and Dissertations

Deep learning has become more widespread as advances in the field continue. As aresult, making sure deep learning is safe has become a priority. A seemingly normal image with intentional pixel changes can cause a well-trained model to misclassify the image with high confidence. Those kinds of images are called adversarial attacks. Adversarial training has been developed to defend against adversarial attacks. This thesis evaluates different adversarial training methods against a variety of adversarial attacks. The key metrics for evaluation are classification accuracy and training time. This thesis also experiments with an improvement on an existing adversarial training method, the …

Semantic Lung Segmentation From Chest X-Ray Images Using Seg-Net Deep Cnn Model, Dathar Abas Hasan, Umed Hayder Jader

Polytechnic Journal

Implementing an accurate image segmentation to extract the lung shape from X-ray images is a vital step in designing a CAD system that diagnoses various types of chest diseases. Lung segmentation is a complex process due to the blurred regions that separate the lung area and the rest of the image. The conventional image segmentation techniques do not meet the ambitions to achieve precise lung segmentation. In this paper, we utilized the Seg-Net semantic segmentation model as a practical approach to distinguish the lung region pixels in X-ray images. The model involves an encoder network that extracts the data from …

Flacgec: A Chinese Grammatical Error Correction Dataset With Fine-Grained Linguistic Annotation, Hanyue Du, Yike Zhao, Qingyuan Tian, Jiani Wang, Lei Wang, Yunshi Lan, Xuesong Lu

Research Collection School Of Computing and Information Systems

Chinese Grammatical Error Correction (CGEC) has been attracting growing attention from researchers recently. In spite of the fact that multiple CGEC datasets have been developed to support the research, these datasets lack the ability to provide a deep linguistic topology of grammar errors, which is critical for interpreting and diagnosing CGEC approaches. To address this limitation, we introduce FlaCGEC, which is a new CGEC dataset featured with fine-grained linguistic annotation. Specifically, we collect raw corpus from the linguistic schema defined by Chinese language experts, conduct edits on sentences via rules, and refine generated samples manually, which results in 10k sentences …

Emotion-Aware Music Recommendation, Hieu Tran, Tuan Le, Anh Do, Tram Vu, Steven Bogaerts, Brian T. Howard

Computer Science Faculty publications

It is common to listen to songs that match one's mood. Thus, an AI music recommendation system that is aware of the user's emotions is likely to provide a superior user experience to one that is unaware. In this paper, we present an emotion-aware music recommendation system. Multiple models are discussed and evaluated for affect identification from a live image of the user. We propose two models: DRViT, which applies dynamic routing to vision transformers, and InvNet50, which uses involution. All considered models are trained and evaluated on the AffectNet dataset. Each model outputs the user's estimated valence and arousal …

Gpachov At Checkthat! 2023: A Diverse Multi-Approach Ensemble For Subjectivity Detection In News Articles, Georgi Pachov, Dimitar Dimitrov, Ivan Koychev, Preslav Nakov

Natural Language Processing Faculty Publications

The wide-spread use of social networks has given rise to subjective, misleading, and even false information on the Internet. Thus, subjectivity detection can play an important role in ensuring the objectiveness and the quality of a piece of information. This paper presents the solution built by the Gpachov team for the CLEF-2023 CheckThat! lab Task 2 on subjectivity detection. Three different research directions are explored. The first one is based on fine-tuning a sentence embeddings encoder model and dimensionality reduction. The second one explores a sample-efficient few-shot learning model. The third one evaluates fine-tuning a multilingual transformer on an altered …

Out-Of-Distribution Generalization Of Deep Learning To Illuminate Dark Protein Functional Space, Tian Cai

Dissertations, Theses, and Capstone Projects

Dark protein illumination is a fundamental challenge in drug discovery where majority human proteins are understudied, i.e. with only known protein sequence but no known small molecule binder. It's a major road block to enable drug discovery paradigm shift from single-targeted which looks to identify a single target and design drug to regulate the single target to multi-targeted in a Systems Pharmacology perspective. Diseases such as Alzheimer's and Opioid-Use-Disorder plaguing millions of patients call for effective multi-targeted approach involving dark proteins. Using limited protein data to predict dark protein property requires deep learning systems with OOD generalization capacity. Out-of-Distribution (OOD) …

Optimization And Application Of Graph Neural Networks, Shuo Zhang

Dissertations, Theses, and Capstone Projects

Graph Neural Networks (GNNs) are widely recognized for their potential in learning from graph-structured data and solving complex problems. However, optimal performance and applicability of GNNs have been an open-ended challenge. This dissertation presents a series of substantial advances addressing this problem. First, we investigate attention-based GNNs, revealing a critical shortcoming: their ignorance of cardinality information that impacts their discriminative power. To rectify this, we propose Cardinality Preserved Attention (CPA) models that can be applied to any attention-based GNNs, which exhibit a marked improvement in performance. Next, we introduce the Directional Node Pair (DNP) descriptor and the Robust Molecular Graph …

Learning Representations For Effective And Explainable Software Bug Detection And Fixing, Yi Li

Dissertations

Software has an integral role in modern life; hence software bugs, which undermine software quality and reliability, have substantial societal and economic implications. The advent of machine learning and deep learning in software engineering has led to major advances in bug detection and fixing approaches, yet they fall short of desired precision and recall. This shortfall arises from the absence of a 'bridge,' known as learning code representations, that can transform information from source code into a suitable representation for effective processing via machine and deep learning.

This dissertation builds such a bridge. Specifically, it presents solutions for effectively learning …

Countnet3d: A 3d Computer Vision Approach To Infer Counts Of Occluded Objects With Quantified Uncertainty, Stephen W. Nelson

Theses and Dissertations

3D scene understanding is an important problem that has experienced great progress in recent years, in large part due to the development of state-of-the-art methods for 3D object detection. However, the performance of 3D object detectors can suffer in scenarios where extreme occlusion of objects is present, or the number of object classes is large. In this paper, we study the problem of inferring 3D counts from densely packed scenes with heterogeneous objects. This problem has applications to important tasks such as inventory management or automatic crop yield estimation. We propose a novel regression-based method, CountNet3D, that uses mature 2D …