Artificial Intelligence and Robotics Commons^™

Bridging Expert Knowledge With Deep Learning Techniques For Just-In-Time Defect Prediction, Xin Zhou, Donggyun Han, David Lo

Research Collection School Of Computing and Information Systems

Just-In-Time (JIT) defect prediction aims to automatically predict whether a commit is defective or not, and has been widely studied in recent years. In general, most studies can be classified into two categories: 1) simple models using traditional machine learning classifiers with hand-crafted features, and 2) complex models using deep learning techniques to automatically extract features from commit contents. Hand-crafted features used by simple models are based on expert knowledge but may not fully represent the semantic meaning of the commits. On the other hand, deep learning-based features used by complex models represent the semantic meaning of commits but may …

Deep Learning-Based Auto-Segmentation For Liver Yttrium-90 Selective Internal Radiation Therapy, Jun Li, Wookjin Choi, Rani Anne

Department of Radiation Oncology Faculty Papers

The aim was to evaluate a deep learning-based auto-segmentation method for liver delineation in Y-90 selective internal radiation therapy (SIRT). A deep learning (DL)-based liver segmentation model using the U-Net3D architecture was built. Auto-segmentation of the liver was tested in CT images of SIRT patients. DL auto-segmented liver contours were evaluated against physician manually-delineated contours. Dice similarity coefficient (DSC) and mean distance to agreement (MDA) were calculated. The DL-model-generated contours were compared with the contours generated using an Atlas-based method. Ratio of volume (RV, the ratio of DL-model auto-segmented liver volume to manually-delineated liver volume), and ratio of activity (RA, …

Leveraging Distributed Semantics From Deep Learning Architectures For Literature-Based Discovery, Clint A. Cuffy

Theses and Dissertations

Literature-based discovery (LBD) is a scientific process that introduces methods to automatically identify novel insights between non-interacting sets of literature. To date, numerous statistical and machine learning-based methods have been applied in the biomedical domain to find treatments for diseases such as Raynaud's disease, Parkinson's disease, and Multiple Sclerosis. However, the lack of standardized practices and creation of bespoke methodologies produces a scenario where the adoption of LBD remains challenging in real-world systems. Our work addresses these concerns through the improvement of five critical areas: 1) error propagation within LBD's a priori dependent tasks, 2) exploring the integration of modern …

Generalizing Medical Image Segmentation Task With Efficient Deep Learning Models, Abel A. Reyes-Angulo

Dissertations, Master's Theses and Master's Reports

Medical Image Segmentation is a critical task in the field of medical imaging, playing a crucial role in diagnostics, treatment planning, and disease monitoring. The emergence of Deep Learning (DL) has ushered in a new era in Artificial Intelligence (AI), propelling remarkable advancements in key domains like language translation, object recognition, and recommendation systems. This evolution has been accompanied by continuous enhancements in computational efficiency and improvements in predictive accuracy. The introduction of sophisticated algorithms, such as convolutional neural networks (CNNs) and transformers, exemplifies these advancements. DL algorithms have demonstrated exceptional efficacy in medical image segmentation tasks, showcasing the potential …

Applications Of Deep Learning For Optimizing Fingerphoto And Latent Fingerprint Biometrics, Amol Sanjay Joshi

Graduate Theses, Dissertations, and Problem Reports (ETD)

Fingerprint-based biometric recognition remains one of the most dependable and widely adopted approaches for identity verification due to its permanence and distinctiveness. Recent advancements in mobile and contactless imaging have extended fingerprint acquisition beyond controlled environments into unconstrained, real-world conditions through fingerphotos, contactless fingerprint images captured by digital or smartphone cameras. While this paradigm shift enhances accessibility and user convenience, it introduces significant technical challenges. Variations in illumination, focus, and motion blur often degrade ridge patterns, making accurate feature extraction and matching more difficult. Similarly, in forensic applications, latent fingerprints, incomplete or smudged prints lifted from surfaces, pose unique challenges …

Real-Time Feedback-Driven Framework For Automated Cybersickness Mitigation, Md Jahirul Islam

Master's Theses

As technologies are becoming more advanced day by day, the embracement of virtual reality (VR) technology among users is also increasing in daily activities for various purposes, and subsequently, the barrier between the real and virtual world is fading. Despite the versatile uses, cybersickness (CS) is a major problem which is induced among users due to the immersive VR experience. There is a plethora of research findings and methods to measure the users’ CS such as virtual reality sickness questionnaire (VRSQ), simulator sickness questionnaire (SSQ), fast motion scale questionnaire (FMS), and others. Recently, machine learning approaches have also been adopted …

Robust Learning With Probabilistic Relaxation Using Hypothesis-Test-Based Sampling, Zilin Wang

Dissertations and Theses Collection (Open Access)

In recent years, deep learning has been a vital tool in various tasks. The performance of a neural network is usually evaluated by empirical risk minimization. However, robustness issues have gained great concern which can be fatal in safety-critical applications. Adversarial training can mitigate the issue by minimizing the loss of worst-case perturbations of data. It is effective in improving the robustness of the model, but it is too conservative, and the plain performance of the model can be unsatisfying. Probabilistic Robust Learning (PRL) empirically balances the average- and worst-case performance while the robustness of the model is not provable …

Detecting Anomalies In Dynamic Attributed Graphs: An Unsupervised Learning Approach, Austin Hamilton

Electronic Theses and Dissertations

Dynamic attributed graphs, which evolve over time and hold node-specific attributes, are essential in fields like social network analysis, where anomalous node detection is a growing area. Vehicular social networks (VSNs), a subset of these graphs, are ad hoc networks in which vehicles exchange data with one another and with infrastructure. In this dynamic context, identifying anomalous nodes is challenging but crucial for maintaining trust within the network. This work presents an unsupervised deep learning approach for anomalous node detection in VSNs. This model achieved an accuracy of 71% while detecting synthetic anomalies in a simulated network based on real-world …

Real-Time Network Simulations For Ml/Dl Ddos Detection Using Docker, Luis D. Garcia

Master's Theses

As the integration of artificial intelligence (AI) within cybersecurity continues to

grow, machine learning (ML) and deep learning (DL) models are increasingly used to

detect cyber attacks. However, these models are rarely evaluated in real-time attack

scenarios to see how subtle changes from the real networking environment can affect

their predictions. To address this issue, we propose a scalable, platform-independent

Docker testbed specifically designed for simulating real-time Distributed Denial of

Service (DDoS) attack scenarios that allows researchers to deploy and evaluate their

pre-trained, ML and DL detection models. Our framework is simple to configure

and can run across Intel and …

"Deep Learning For Microscope Image Denoising", Nasreen Buhn, Sriya Adunur, Guy Hagen, Jonathan Ventura

College of Engineering Summer Undergraduate Research Program

In order to avoid damaging live cells, optical microscope imaging must be conducted under low-excitation light intensity and/or short exposure times, resulting in low signal-to-noise ratios (SNR). Deep learning methods offer an effective solution for removing microscope noise, utilizing algorithms that are able to reconstruct finer features in low SNR images. This research explores the denoising capability of several deep learning methods based on PSNR and SSIM. Tested methods include traditional approaches (BMED), supervised learning (CARE and Restormer), and unsupervised methods (Noise2Fast, N2V, SSD-Unsupervised, and SASSID). The Restormer model, which employs an encoder-decoder transformer architecture and progressive learning, stood out …

Retrofitting A Legacy Cutlery Washing Machine Using Computer Vision, Hua Leong Fwa

Research Collection School Of Computing and Information Systems

Industry 4.0, the digitalization of manufacturing promises to lead to lowered cost, efficient processes and even discovery of new business models. However, many of the enterprises have huge investments in legacy machines which are not 'smart'. In this study, we thus designed a cost-efficient solution to retrofit a legacy conveyor belt-based cutlery washing machine with a commodity web camera. We then applied computer vision (using both traditional image processing and deep learning techniques) to infer the speed and utilization of the machine. We detailed the algorithms that we designed for computing both speed andutilization. With the existing operational constraints of …

Developing Empathetic Ai: Exploring The Potential Of Artificial Intelligence To Understand And Simulate Family Dynamics And Cultural Identity, Emily Barnes, James Hutson

Faculty Scholarship

The rapid advancement of Artificial Intelligence (AI) has significantly impacted various domains. Yet, the exploration of AI's potential to develop a deep understanding of family culture and identity remains underexplored. This study introduces the concept of "a love of grandma and apple pie" to symbolize the potential of various AI to internalize and appreciate familial relationships, cultural traditions, and personal identity. The proposed study would investigate how an advanced deep learning model, trained on diverse unstructured datasets—including multimedia data from 100 families-could learn and reflect human-like emotions, values, and cultural understanding. Utilizing Convolutional Neural Networks (CNNs) for visual data processing …

The Impact Of Model Variations On The Robustness Of Deep Learning Models In Adversarial Settings, Firuz Juraev, Mohammed Abuhamad, Simon S. Woo, George K. Thiruvathukal, Tamer Abuhmed

Computer Science: Faculty Publications and Other Works

Rapid advancements of deep learning are accelerating adoption in a wide variety of applications, including safety-critical applications such as self-driving vehicles, drones, robots, and surveillance systems. These advancements include applying variations of sophisticated techniques that improve the performance of models. However, such models are not immune to adversarial manipulations, which can cause the system to misbehave and remain unnoticed by experts. The frequency of modifications to existing deep learning models necessitates thorough analysis to determine the impact on models’ robustness. In this work, we present an experimental evaluation of the effects of model modifications on deep learning model robustness using …

Development And Optimization Of A 1-Dimensional Convolutional Neural Network-Based Keyword Spotting Model For Fpga Acceleration, Trysten E. Dembeck

Masters Theses

Spoken Keyword Spotting (KWS) has steadily remained one of the most studied and implemented technologies in human-facing artificially intelligent systems and has enabled them to detect specific keywords in utterances. Modern machine learning models, such as the variants of deep neural networks, have significantly improved the performance and accuracy of these systems over other rudimentary techniques. However, they often demand substantial computational resources, use large parameter spaces, and introduce latencies that limit their real-time applicability and offline use. These speed and memory requirements have become a tremendous problem where faster and more efficient KWS methods dominate and better meet industry …

Transformer-Based Deep Learning Prediction Of 10-Degree Humphrey Visual Field Tests From 24-Degree Data, Min Shi, Anagha Lokhande, Yu Tian, Yan Luo, Mohammad Eslami, Saber Kazeminasab, Tobias Elze, Lucy Shen, Louis Pasquale, Sarah Wellik, Carlos Gustavo De Moraes, Jonathan Myers, Nazlee Zebardast, David Friedman, Michael Boland, Mengyu Wang

Wills Eye Hospital Papers

PURPOSE: To predict 10-2 Humphrey visual fields (VFs) from 24-2 VFs and associated non-total deviation features using deep learning.

METHODS: We included 5189 reliable 24-2 and 10-2 VF pairs from 2236 patients, and 28,409 reliable pairs of macular OCT scans and 24-2 VF from 19,527 eyes of 11,560 patients. We developed a transformer-based deep learning model using 52 total deviation values and nine VF test features to predict 68 10-2 total deviation values. The mean absolute error, root mean square error, and the R2 were evaluation metrics. We further evaluated whether the predicted 10-2 VFs can improve the structure-function relationship …

High Prevalence Of Artifacts In Optical Coherence Tomography With Adequate Signal Strength, Wei-Chun Lin, Aaron Coyner, Charles Amankwa, Abigail Lucero, Gadi Wollstein, Joel Schuman, Hiroshi Ishikawa

Wills Eye Hospital Papers

PURPOSE: This study aims to investigate the prevalence of artifacts in optical coherence tomography (OCT) images with acceptable signal strength and evaluate the performance of supervised deep learning models in improving OCT image quality assessment.

METHODS: We conducted a retrospective study on 4555 OCT images from 546 patients, with each image having an acceptable signal strength (≥6). A comprehensive analysis of prevalent OCT artifacts was performed, and five pretrained convolutional neural network models were trained and tested to infer images based on quality.

RESULTS: Our results showed a high prevalence of artifacts in OCT images with acceptable signal strength. Approximately …

Checklist For Reproducibility Of Deep Learning In Medical Imaging, Mana Moassefi, Yashbir Singh, Gian Marco Conte, Bardia Khosravi, Pouria Rouzrokh, Sanaz Vahdati, Nabile Safdar, Linda Moy, Felipe Kitamura, Amilcare Gentili, Paras Lakhani, Nina Kottler, Safwan Halabi, Joseph Yacoub, Yuankai Hou, Khaled Younis, Bradley Erickson, Elizabeth Krupinski, Shahriar Faghani

Department of Radiology Faculty Papers

The application of deep learning (DL) in medicine introduces transformative tools with the potential to enhance prognosis, diagnosis, and treatment planning. However, ensuring transparent documentation is essential for researchers to enhance reproducibility and refine techniques. Our study addresses the unique challenges presented by DL in medical imaging by developing a comprehensive checklist using the Delphi method to enhance reproducibility and reliability in this dynamic field. We compiled a preliminary checklist based on a comprehensive review of existing checklists and relevant literature. A panel of 11 experts in medical imaging and DL assessed these items using Likert scales, with two survey …

From Graph Theory For Robust Deep Networks To Graph Learning For Multimodal Cancer Analysis, Asim Waqas

USF Tampa Graduate Theses and Dissertations

This dissertation explores the intersection of graph theory and deep learning, focusing on enhancing the robustness of deep neural networks (DNNs) and applying these advancements to complex problems like cancer diagnosis and treatment. We investigate the structural properties of graphs and their influence on neural network performance, particularly in multimodal learning. The work delves into the design space of DNN architectures using graph-theoretic measures, transforming graphs into DNN architectures for various tasks, and examining their robustness against noise and adversarial attacks. The study extends to medical imaging, highlighting advanced DNN architectures like U-Net for brain tumor segmentation. It addresses the …

Machine Learning Multimodal Framework For Fake News Detection And Mitigation, Nada A. Gaballah

Theses and Dissertations

Social media has become our new reality, people wake up every morning and the first thing they do before getting out of bed, is check their social media. Nowadays, people rarely read newspapers, they even rarely watch TV news or listen to radio broadcasts. In recent years, we have witnessed lots of fake news roaming social media every second, with people simply believing it and spreading it even more without checking the credibility of this news. This fake news affected several domains like what happened in the US election in 2016 and again in 2020, the false information about Covid-19 …

Context In Computer Vision: A Taxonomy, Multi-Stage Integration, And A General Framework, Xuan Wang

Dissertations, Theses, and Capstone Projects

Contextual information has been widely used in many computer vision tasks, such as object detection, video action detection, image classification, etc. Recognizing a single object or action out of context could be sometimes very challenging, and context information may help improve the understanding of a scene or an event greatly. However, existing approaches design specific contextual information mechanisms for different detection tasks.

In this research, we first present a comprehensive survey of context understanding in computer vision, with a taxonomy to describe context in different types and levels. Then we proposed MultiCLU, a new multi-stage context learning and utilization framework, …

Morp: Monocular Orientation Regression Pipeline, Jacob Gunderson

Master's Theses

Orientation estimation of objects plays a pivotal role in robotics, self-driving cars, and augmented reality. Beyond mere position, accurately determining the orientation of objects is essential for constructing precise models of the physical world. While 2D object detection has made significant strides, the field of orientation estimation still faces several challenges. Our research addresses these hurdles by proposing an efficient pipeline which facilitates rapid creation of labeled training data and enables direct regression of object orientation from a single image. We start by creating a digital twin of a physical object using an iPhone, followed by generating synthetic images using …

Singleadv: Single-Class Target-Specific Attack Against Interpretable Deep Learning Systems, Eldor Abdukhamidov, Mohammed Abuhamad, George K. Thiruvathukal, Hyoungshick Kim, Tamer Abuhmed

Computer Science: Faculty Publications and Other Works

In this paper, we present a novel Single-class target-specific Adversarial attack called SingleADV. The goal of SingleADV is to generate a universal perturbation that deceives the target model into confusing a specific category of objects with a target category while ensuring highly relevant and accurate interpretations. The universal perturbation is stochastically and iteratively optimized by minimizing the adversarial loss that is designed to consider both the classifier and interpreter costs in targeted and non-targeted categories. In this optimization framework, ruled by the first- and second-moment estimations, the desired loss surface promotes high confidence and interpretation score of adversarial samples. By …

Context Aware Music Recommendation And Playlist Generation, Elias Mann

SMU Journal of Undergraduate Research

There are many reasons people listen to music, and the type of music is largely determined by what the listener may be doing while they listen. For example, one may listen to one type of music while commuting, another while exercising, and yet another while relaxing. Without access to the physiological state of the user, current music recommendation methods rely on collaborative filtering - recommending music based on what other similar users listen to - and content based filtering - recommending songs based on their similarities to songs the user already prefers. With the rise in popularity of smart devices …

Missing Wedge Completion Via Unsupervised Learning With Coordinate Networks, Dave Van Veen, Jesús G Galaz-Montoya, Liyue Shen, Philip Baldwin, Akshay S Chaudhari, Dmitry Lyumkis, Michael F Schmid, Wah Chiu, John Pauly

Faculty, Staff and Students Publications

Cryogenic electron tomography (cryoET) is a powerful tool in structural biology, enabling detailed 3D imaging of biological specimens at a resolution of nanometers. Despite its potential, cryoET faces challenges such as the missing wedge problem, which limits reconstruction quality due to incomplete data collection angles. Recently, supervised deep learning methods leveraging convolutional neural networks (CNNs) have considerably addressed this issue; however, their pretraining requirements render them susceptible to inaccuracies and artifacts, particularly when representative training data is scarce. To overcome these limitations, we introduce a proof-of-concept unsupervised learning approach using coordinate networks (CNs) that optimizes network weights directly against input …

Ai: Is The Future Robots?, Ruthanne Fischer

University Honors College

The goal of this thesis is to analyze how Artificial Intelligence is actively changing. The newly developed technology that is known as Artificial Intelligence can potentially have implications that are not yet known. Artificial Intelligence impacts the business world in many ways. Things such as ChatGPT and Deepfakes are some of the biggest forms of Artificial Intelligence today. It is important to know how Artificial Intelligence is changing every day because no one knows the true implications of a technology like this. It is crucial for companies to understand how this technology can potentially be harmful to their employees. Artificial …

Robust And Trustworthy Deep Learning: Attacks, Defenses And Designs, Bingyin Zhao

All Dissertations

Deep neural networks (DNNs) have achieved unprecedented success in many fields. However, robustness and trustworthiness have become emerging concerns since DNNs are vulnerable to various attacks and susceptible to data distributional shifts. Attacks such as data poisoning and out-of-distribution scenarios such as natural corruption significantly undermine the performance and robustness of DNNs in model training and inference and impose uncertainty and insecurity on the deployment in real-world applications. Thus, it is crucial to investigate threats and challenges against deep neural networks, develop corresponding countermeasures, and dig into design tactics to secure their safety and reliability. The works investigated in this …

Learning Scene Semantics For 3d Scene Retrieval, Natalie Gleason

Honors Theses

This project presents a comprehensive exploration into semantics-driven 3D scene retrieval, aiming to bridge the gap between 2D sketches/images and 3D models. Through four distinct research objectives, this project endeavors to construct a foundational infrastructure, develop methodologies for quantifying semantic similarity, and advance a semantics-based retrieval framework for 2D scene sketch-based and image-based 3D scene retrieval. Leveraging WordNet as a foundational semantic ontology library, the research proposes the construction of an extensive hierarchical scene semantic tree, enriching 2D/3D scenes with encoded semantic information. The methodologies for semantic similarity computation utilize this semantic tree to bridge the semantic disparity between 2D …

Modeling The Spatiotemporal Variations Of The Magnetic Field In Active Regions On The Sun Using Deep Neural Networks, Godwill Asare Mensah Mensah

Open Access Theses & Dissertations

Solar active regions are areas on the Sun's surface that have especially strong magnetic fields. Active regions are usually linked to a number of phenomena that can have serious detrimental consequences on technology and, in turn, human life. Examples of these phenomena include solar flares and coronal mass ejections, or CMEs. The precise predictionof solar flares and coronal mass ejections is still an open problem since the fundamental processes underpinning the formation and development of active regions are still not well understood. One key area of research at the intersection of solar physics and artificial intelligence is deriving insights from …

Evaluation Of An End-To-End Radiotherapy Treatment Planning Pipeline For Prostate Cancer, Mohammad Daniel El Basha, Court Laurence, Carlos Eduardo Cardenas, Julianne Pollard-Larkin, Steven Frank, David T. Fuentes, Falk Poenisch, Zhiqian H. Yu

Dissertations and Theses (Open Access)

Radiation treatment planning is a crucial and time-intensive process in radiation therapy. This planning involves carefully designing a treatment regimen tailored to a patient’s specific condition, including the type, location, and size of the tumor with reference to surrounding healthy tissues. For prostate cancer, this tumor may be either local, locally advanced with extracapsular involvement, or extend into the pelvic lymph node chain. Automating essential parts of this process would allow for the rapid development of effective treatment plans and better plan optimization to enhance tumor control for better outcomes.

The first objective of this work, to automate the treatment …

Representation Learning For Generative Models With Applications To Healthcare, Astronautics, And Aviation, Van Minh Nguyen

Theses and Dissertations

This dissertation explores applications of representation learning and generative models to challenges in healthcare, astronautics, and aviation.

The first part investigates the use of Generative Adversarial Networks (GANs) to synthesize realistic electronic health record (EHR) data. An initial attempt at training a GAN on the MIMIC-IV dataset encountered stability and convergence issues, motivating a deeper study of 1-Lipschitz regularization techniques for Auxiliary Classifier GANs (AC-GANs). An extensive ablation study on the CIFAR-10 dataset found that Spectral Normalization is key for AC-GAN stability and performance, while Weight Clipping fails to converge without Spectral Normalization. Analysis of the training dynamics provided further …