Digital Commons Network^™

Context In Computer Vision: A Taxonomy, Multi-Stage Integration, And A General Framework, Xuan Wang

Dissertations, Theses, and Capstone Projects

Contextual information has been widely used in many computer vision tasks, such as object detection, video action detection, image classification, etc. Recognizing a single object or action out of context could be sometimes very challenging, and context information may help improve the understanding of a scene or an event greatly. However, existing approaches design specific contextual information mechanisms for different detection tasks.

In this research, we first present a comprehensive survey of context understanding in computer vision, with a taxonomy to describe context in different types and levels. Then we proposed MultiCLU, a new multi-stage context learning and utilization framework, …

Singleadv: Single-Class Target-Specific Attack Against Interpretable Deep Learning Systems, Eldor Abdukhamidov, Mohammed Abuhamad, George K. Thiruvathukal, Hyoungshick Kim, Tamer Abuhmed

Computer Science: Faculty Publications and Other Works

In this paper, we present a novel Single-class target-specific Adversarial attack called SingleADV. The goal of SingleADV is to generate a universal perturbation that deceives the target model into confusing a specific category of objects with a target category while ensuring highly relevant and accurate interpretations. The universal perturbation is stochastically and iteratively optimized by minimizing the adversarial loss that is designed to consider both the classifier and interpreter costs in targeted and non-targeted categories. In this optimization framework, ruled by the first- and second-moment estimations, the desired loss surface promotes high confidence and interpretation score of adversarial samples. By …

Context Aware Music Recommendation And Playlist Generation, Elias Mann

SMU Journal of Undergraduate Research

There are many reasons people listen to music, and the type of music is largely determined by what the listener may be doing while they listen. For example, one may listen to one type of music while commuting, another while exercising, and yet another while relaxing. Without access to the physiological state of the user, current music recommendation methods rely on collaborative filtering - recommending music based on what other similar users listen to - and content based filtering - recommending songs based on their similarities to songs the user already prefers. With the rise in popularity of smart devices …

Crash Detecting System Using Deep Learning, Yogesh Reddy Muddam

Electronic Theses, Projects, and Dissertations

Accidents pose a significant risk to both individual and property safety, requiring effective detection and response systems. This work introduces an accident detection system using a convolutional neural network (CNN), which provides an impressive accuracy of 86.40%. Trained on diverse data sets of images and videos from various online sources, the model exhibits complex accident detection and classification and is known for its prowess in image classification and visualization.

CNN ensures better accident detection in various scenarios and road conditions. This example shows its adaptability to a real-world accident scenario and enhances its effectiveness in detecting early events. A key …

Evaluation Of An End-To-End Radiotherapy Treatment Planning Pipeline For Prostate Cancer, Mohammad Daniel El Basha, Court Laurence, Carlos Eduardo Cardenas, Julianne Pollard-Larkin, Steven Frank, David T. Fuentes, Falk Poenisch, Zhiqian H. Yu

Dissertations & Theses (Open Access)

Radiation treatment planning is a crucial and time-intensive process in radiation therapy. This planning involves carefully designing a treatment regimen tailored to a patient’s specific condition, including the type, location, and size of the tumor with reference to surrounding healthy tissues. For prostate cancer, this tumor may be either local, locally advanced with extracapsular involvement, or extend into the pelvic lymph node chain. Automating essential parts of this process would allow for the rapid development of effective treatment plans and better plan optimization to enhance tumor control for better outcomes.

The first objective of this work, to automate the treatment …

Pedestrian Pathing Prediction Using Complex Contextual Behavioral Data In High Foot Traffic Settings, Laurel Bingham

All Graduate Theses and Dissertations, Fall 2023 to Present

Ensuring the safe integration of autonomous vehicles into real-world environments requires a comprehensive understanding of pedestrian behavior. This study addresses the challenge of predicting the movement and crossing intentions of pedestrians, a crucial aspect in the development of fully autonomous vehicles.

The research focuses on leveraging Honda's TITAN dataset, comprising 700 unique clips captured by moving vehicles in high-foot-traffic areas of Tokyo, Japan. Each clip provides detailed contextual information, including human-labeled tags for individuals and vehicles, encompassing attributes such as age, motion status, and communicative actions. Long Short-Term Memory (LSTM) networks were employed and trained on various combinations of contextual …

Deep Learning In Indus Valley Script Digitization, Deva Munikanta Reddy Atturu

Theses and Dissertations

This research introduces ASR-net(Ancient Script Recognition), a groundbreaking system that automatically digitizes ancient Indus seals by converting them into coded text, similar to Optical Character Recognition for modern languages. ASR-net, with an 95% success rate in identifying individual symbols, aims to address the crucial need for automated techniques in deciphering the enigmatic Indus script. Initially Yolov3 is utilized to create the bounding boxes around each graphemes present in the Indus Valley Seal. In addition to that we created M-net(Mahadevan) model to encode the graphemes. Beyond digitization, the paper proposes a new research challenge called the Motif Identification Problem (MIP) related …

Robust And Trustworthy Deep Learning: Attacks, Defenses And Designs, Bingyin Zhao

All Dissertations

Deep neural networks (DNNs) have achieved unprecedented success in many fields. However, robustness and trustworthiness have become emerging concerns since DNNs are vulnerable to various attacks and susceptible to data distributional shifts. Attacks such as data poisoning and out-of-distribution scenarios such as natural corruption significantly undermine the performance and robustness of DNNs in model training and inference and impose uncertainty and insecurity on the deployment in real-world applications. Thus, it is crucial to investigate threats and challenges against deep neural networks, develop corresponding countermeasures, and dig into design tactics to secure their safety and reliability. The works investigated in this …

Modeling The Spatiotemporal Variations Of The Magnetic Field In Active Regions On The Sun Using Deep Neural Networks, Godwill Asare Mensah Mensah

Open Access Theses & Dissertations

Solar active regions are areas on the Sun's surface that have especially strong magnetic fields. Active regions are usually linked to a number of phenomena that can have serious detrimental consequences on technology and, in turn, human life. Examples of these phenomena include solar flares and coronal mass ejections, or CMEs. The precise predictionof solar flares and coronal mass ejections is still an open problem since the fundamental processes underpinning the formation and development of active regions are still not well understood. One key area of research at the intersection of solar physics and artificial intelligence is deriving insights from …

Detection And Classification Of Diabetic Retinopathy Using Deep Learning Models, Aishat Olatunji

Electronic Theses and Dissertations

Healthcare analytics leverages extensive patient data for data-driven decision-making, enhancing patient care and results. Diabetic Retinopathy (DR), a complication of diabetes, stems from damage to the retina’s blood vessels. It can affect both type 1 and type 2 diabetes patients. Ophthalmologists employ retinal images for accurate DR diagnosis and severity assessment. Early detection is crucial for preserving vision and minimizing risks. In this context, we utilized a Kaggle dataset containing patient retinal images, employing Python’s versatile tools. Our research focuses on DR detection using deep learning techniques. We used a publicly available dataset to apply our proposed neural network and …

Ghost Connect-Net: A Connectivity-Based Companion Network To Enhance Pruning Methods, Mary Isabelle Wisell

Honors College

Deep neural network (DNN) approaches excel in various real-world applications like robotics and computer vision, yet their computational demands and memory requirements hinder usability on advanced devices. Also, larger models heighten overparameterization risks, making networks more vulnerable to input disturbances. Recent studies aim to boost DNN efficiency by trimming redundant neurons or filters based on task relevance. Instead of introducing a new pruning method, this project aims to enhance existing techniques by introducing a companion network, Ghost Connect-Net (GC-Net), to monitor the connections in the original network. The initial weights of GC- Net are equal to the connectivity measurements of …

Blueberry Drone Ai: Estimating Crop Yield Using Deep Learning & Smart Drones, Luke Tonon, Brandon Mchenry, Anthony Thompson, Harper Zappone, Jacob Green, Hieu Nguyen, Thanh Nguyen

STEM Student Research Symposium Posters

This project seeks to assist blueberry growers in New Jersey estimate crop yield by developing software that allows autonomous drones to capture aerial images of blueberry bushes in the field, perform berry count, and identify blueberry conditions using deep learning models & computer vision.

Extracting Dnn Architectures Via Runtime Profiling On Mobile Gpus, Dong Hyub Kim

Masters Theses

Due to significant investment, research, and development efforts over the past decade, deep neural networks (DNNs) have achieved notable advancements in classification and regression domains. As a result, DNNs are considered valuable intellectual property for artificial intelligence providers. Prior work has demonstrated highly effective model extraction attacks which steal a DNN, dismantling the provider’s business model and paving the way for unethical or malicious activities, such as misuse of personal data, safety risks in critical systems, or spreading misinformation. This thesis explores the feasibility of model extraction attacks on mobile devices using aggregated runtime profiles as a side-channel to leak …

Functional Data Learning Using Convolutional Neural Networks, Jose Galarza, Tamer Oraby

School of Mathematical and Statistical Sciences Faculty Publications and Presentations

In this paper, we show how convolutional neural networks (CNNs) can be used in regression and classification learning problems for noisy and non-noisy functional data (FD). The main idea is to transform the FD into a 28 by 28 image. We use a specific but typical architecture of a CNN to perform all the regression exercises of parameter estimation and functional form classification. First, we use some functional case studies of FD with and without random noise to showcase the strength of the new method. In particular, we use it to estimate exponential growth and decay rates, the bandwidths of …

Statistically Principled Deep Learning For Sar Image Segmentation, Cassandra Goldberg

Honors Projects

This project explores novel approaches for Synthetic Aperture Radar (SAR) image segmentation that integrate established statistical properties of SAR into deep learning models. First, Perlin Noise and Generalized Gamma distribution sampling methods were utilized to generate a synthetic dataset that effectively captures the statistical attributes of SAR data. Subsequently, deep learning segmentation architectures were developed that utilize average pooling and 1x1 convolutions to perform statistical moment computations. Finally, supervised and unsupervised disparity-based losses were incorporated into model training. The experimental outcomes yielded promising results: the synthetic dataset effectively trained deep learning models for real SAR data segmentation, the statistically-informed architectures …

Deep Learning Applications On Ionospheric Studies, Yang Pan

Physics Dissertations

Machine learning techniques, particularly deep learning techniques, have been vigorously pursued to tackle space physics problems and achieved some impressive results recently. The growth of deep learning technologies in different domains enables innovative solutions to those problems compared to conventional methods. Filling data gaps in instrumental observations is among the demanding issues, which benefits space physicists to study ionospheric phenomena with complete data coverage. Global total electron content (TEC) and regional ionospheric electron density (Ne) are among important physical parameters in ionospheric studies. Due to the limited coverage of global navigation satellite system (GNSS) ground receivers and sporadic …

Smart Applications And Resource Management In Internet Of Things, Zeinab Akhavan

Computer Science ETDs

Internet of Things (IoT) technologies are currently the principal solutions driving smart cities. These new technologies such as Cyber Physical Systems, 5G and data analytic have emerged to address various cities' infrastructure issues ranging from transportation and energy management to healthcare systems. An IoT setting primarily consists of a wide range of users and devices as a massive network interacting with different layers of the city infrastructure resulting in generating sheer volume of data to enable smart city services. The goal of smart city services is to create value for the entire ecosystem, whether this is health, education, transportation, energy, …

Utilizing Multitask Transfer Learning For Sonographic Rheumatoid Arthritis Synovitis Grading, Jordan Marie Claire Sanders

Doctoral Dissertations and Master's Theses

Classifying the four sonographic Rheumatoid Arthritis (RA) synovitis grades (Grade 0, Grade 1, Grade 2, and Grade 3) is a difficult problem due to the complexity of the relevant markers. Therefore, the current research proposes a Multitask Transfer Learning (MTL) framework for sonographic RA synovitis grading of Ultrasound (US) images in Brightness mode (B-Mode) and Power Doppler mode.

In the medical community, the lack of reliability of scoring these images has been an issue and reason for concern for doctors and other medical practitioners. The human/machine variability across the acquisition procedure of these US images creates an additional challenge that …

Deep Learning Approaches For Chaotic Dynamics And High-Resolution Weather Simulations In The Us Midwest, Vlada Volyanskaya, Kabir Batra, Shubham Shrivastava

Discovery Undergraduate Interdisciplinary Research Internship

Weather prediction is indispensable across various sectors, from agriculture to disaster forecasting, deeply influencing daily life and work. Recent advancement of AI foundation models for weather and climate predictions makes it possible to perform a large number of predictions in reasonable time to support timesensitive policy- and decision-making. However, the uncertainty quantification, validation, and attribution of these models have not been well explored, and the lack of knowledge can eventually hinder the improvement of their prediction accuracy and precision. Our project is embarking on a two-fold approach leveraging deep learning techniques (LSTM and Transformer) architectures. Firstly, we model the Lorenz …

Generalized Differentiable Neural Architecture Search With Performance And Stability Improvements, Emily J. Herron

Doctoral Dissertations

This work introduces improvements to the stability and generalizability of Cyclic DARTS (CDARTS). CDARTS is a Differentiable Architecture Search (DARTS)-based approach to neural architecture search (NAS) that uses a cyclic feedback mechanism to train search and evaluation networks concurrently, thereby optimizing the search process by enforcing that the networks produce similar outputs. However, the dissimilarity between the loss functions used by the evaluation networks during the search and retraining phases results in a search-phase evaluation network, a sub-optimal proxy for the final evaluation network utilized during retraining. ICDARTS, a revised algorithm that reformulates the search phase loss functions to ensure …

Context-Aware Temporal Embeddings For Text And Video Data, Ahnaf Farhan

Open Access Theses & Dissertations

Recent years have seen an exponential increase in unstructured data, primarily in the form of text, images, and videos. Extracting useful features and trends from large-scale unstructured datasets -- such as news outlets, scientific papers, and videos like security cameras or body cam recordings -- is faced with substantial challenges of volume, scalability, complexity, and semantic understanding. In analyzing trends, comprehending the temporal context is vital for uncovering patterns and narratives that are not apparent from a single video frame or text document. Despite its importance, many existing data mining and machine learning approaches overlook extracting evolutionary contextual features in …

Deep Learning For Photovoltaic Characterization, Adrian Manuel De Luis Garcia

Graduate Theses and Dissertations

This thesis introduces a novel approach to Photovoltaic (PV) installation segmentation by proposing a new architecture to understand and identify PV modules from overhead imagery. Pivotal to this concept is the creation of a new Transformer-based network, S3Former, which focuses on small object characterization and modelling intra- and inter- object differentiation inside an image. Accurate mapping of PV installations is pivotal for understanding their adoption and guiding energy policy decisions. Drawing insights from current Deep Learning methodologies for image segmentation and building upon State-of-the-Art (SOTA) techniques in solar cell mapping, this work puts forth S3Former with the following enhancements: 1. …

Domain Specific Feature Representation Learning For Diverse Temporal Data, Farhan Asif Chowdhury

Computer Science ETDs

Humans can leverage domain context to recognize novel patterns and categories based on limited known examples. In contrast, computational learning methods are not adept at exploiting context and require sufficient labeled examples to achieve similar accuracy. Many temporal data domain, for example, seismic signals and oil mining sensor data, requires domain expert annotation, which is both costly and time-consuming. The dependency on training data limits the applicability of machine learning algorithms for domains with limited labeled data. This dissertation aims to address this gap by developing temporal mining algorithms that exploit domain context to learn discriminative feature representation from limited …

Machine Learning Modeling Of Polymer Coating Formulations: Benchmark Of Feature Representation Schemes, Nelson I. Evbarunegbe

Masters Theses

Polymer coatings offer a wide range of benefits across various industries, playing a crucial role in product protection and extension of shelf life. However, formulating them can be a non-trivial task given the multitude of variables and factors involved in the production process, rendering it a complex, high-dimensional problem. To tackle this problem, machine learning (ML) has emerged as a promising tool, showing considerable potential in enhancing various polymer and chemistry-based applications, particularly those dealing with high dimensional complexities.

Our research aims to develop a physics-guided ML approach to facilitate the formulations of polymer coatings. As the first step, this …

Deciphering Trends And Tactics: Data-Driven Techniques For Forecasting Information Spread And Detecting Coordinated Campaigns In Social Media, Kin Wai Ng Lugo

USF Tampa Graduate Theses and Dissertations

The main objective of this dissertation is to develop models that predict and investigate the spread of information in social media over time. In this context, we consider topics of discussions as the information that spreads. Thus, we are interested in forecasting the number of messages per day in a future interval of time. We take a data-driven approach, in which we compare our results with real datasets from a multitude of socio-political contexts and from multiple social media platforms, specifically, Twitter and YouTube.

We identified a number of challenges related to forecasting social media time series per topic. First, …

Optimizing Uncertainty Quantification Of Vision Transformers In Deep Learning On Novel Ai Architectures, Erik Pautsch, John Li, Silvio Rizzi, George K. Thiruvathukal, Maria Pantoja

Computer Science: Faculty Publications and Other Works

Deep Learning (DL) methods have shown substantial efficacy in computer vision (CV) and natural language processing (NLP). Despite their proficiency, the inconsistency in input data distributions can compromise prediction reliability. This study mitigates this issue by introducing uncertainty evaluations in DL models, thereby enhancing dependability through a distribution of predictions. Our focus lies on the Vision Transformer (ViT), a DL model that harmonizes both local and global behavior. We conduct extensive experiments on the ImageNet-1K dataset, a vast resource with over a million images across 1,000 categories. ViTs, while competitive, are vulnerable to adversarial attacks, making uncertainty estimation crucial for …

Evaluating Methods For Improving Dnn Robustness Against Adversarial Attacks, Laureano Griffin

USF Tampa Graduate Theses and Dissertations

Deep learning has become more widespread as advances in the field continue. As aresult, making sure deep learning is safe has become a priority. A seemingly normal image with intentional pixel changes can cause a well-trained model to misclassify the image with high confidence. Those kinds of images are called adversarial attacks. Adversarial training has been developed to defend against adversarial attacks. This thesis evaluates different adversarial training methods against a variety of adversarial attacks. The key metrics for evaluation are classification accuracy and training time. This thesis also experiments with an improvement on an existing adversarial training method, the …

Semantic Lung Segmentation From Chest X-Ray Images Using Seg-Net Deep Cnn Model, Dathar Abas Hasan, Umed Hayder Jader

Polytechnic Journal

Implementing an accurate image segmentation to extract the lung shape from X-ray images is a vital step in designing a CAD system that diagnoses various types of chest diseases. Lung segmentation is a complex process due to the blurred regions that separate the lung area and the rest of the image. The conventional image segmentation techniques do not meet the ambitions to achieve precise lung segmentation. In this paper, we utilized the Seg-Net semantic segmentation model as a practical approach to distinguish the lung region pixels in X-ray images. The model involves an encoder network that extracts the data from …

Flacgec: A Chinese Grammatical Error Correction Dataset With Fine-Grained Linguistic Annotation, Hanyue Du, Yike Zhao, Qingyuan Tian, Jiani Wang, Lei Wang, Yunshi Lan, Xuesong Lu

Research Collection School Of Computing and Information Systems

Chinese Grammatical Error Correction (CGEC) has been attracting growing attention from researchers recently. In spite of the fact that multiple CGEC datasets have been developed to support the research, these datasets lack the ability to provide a deep linguistic topology of grammar errors, which is critical for interpreting and diagnosing CGEC approaches. To address this limitation, we introduce FlaCGEC, which is a new CGEC dataset featured with fine-grained linguistic annotation. Specifically, we collect raw corpus from the linguistic schema defined by Chinese language experts, conduct edits on sentences via rules, and refine generated samples manually, which results in 10k sentences …

Emotion-Aware Music Recommendation, Hieu Tran, Tuan Le, Anh Do, Tram Vu, Steven Bogaerts, Brian T. Howard

Computer Science Faculty publications

It is common to listen to songs that match one's mood. Thus, an AI music recommendation system that is aware of the user's emotions is likely to provide a superior user experience to one that is unaware. In this paper, we present an emotion-aware music recommendation system. Multiple models are discussed and evaluated for affect identification from a live image of the user. We propose two models: DRViT, which applies dynamic routing to vision transformers, and InvNet50, which uses involution. All considered models are trained and evaluated on the AffectNet dataset. Each model outputs the user's estimated valence and arousal …