Artificial Intelligence and Robotics Commons^™

Bridging Data Gaps In Retinal Imaging: From Structural Domain Adaptation To Topology-Aware Synthesis, Gözde Merve Demirci

Dissertations, Theses, and Capstone Projects

Comprehensive visualization of the retina is essential for diagnosing and monitoring blinding diseases such as Diabetic Retinopathy and Retinopathy of Prematurity (ROP), where pathological changes often extend beyond a single field of view. Despite significant advances in automated retinal image analysis, clinical deployment remains limited by two fundamental data gaps: a structural learning gap, arising from scarce expert annotations and poor generalization across imaging domains, and a spatial coverage gap, caused by the difficulty of acquiring multi-view retinal images in fragile populations. Although these challenges are often addressed independently, this dissertation argues that they are tightly coupled: accurate, …

Inductive Biases In Field-Level Cosmological Inference From Galaxy Catalogs, James O'Connor Baldwin

Dissertations, Theses, and Capstone Projects

We perform field-level likelihood-free inference of the matter density parameter Ωm from simulated galaxy catalogs using machine learning models with differing inductive biases. Using features extracted from hydrodynamic simulations in the CAMELS suite, we investigate how both observable choice and model architecture govern the extraction of cosmological information. We consider galaxy positions and line-of-sight peculiar velocities, both separately and in combination, and compare permutation-invariant Deep Sets, implemented with either standard multilayer perceptrons (MLPs) or Kolmogorov–Arnold Networks (KANs), to graph neural networks (GNNs) implemented with MLPs, which explicitly encode spatial relations. We evaluate inference performance under both in-distribution and out-of-distribution (OOD) …

6d Rigid Object Pose Estimation Using Deep Learning, Zhujun Li

Dissertations, Theses, and Capstone Projects

6D object pose estimation is the task of determining an object’s 3D rotation and translation with respect to a camera, and plays a critical role in applications such as robotic manipulation, autonomous navigation, and augmented reality. While recent advances in deep learning have substantially improved performance, many existing methods still face limitations in learning robust and generalizable representations. Factors such as variations in object appearance, occlusion, sensor noise, and domain shifts can degrade model accuracy, highlighting the need for more effective representation learning strategies that capture rich geometric and semantic cues for reliable pose estimation across diverse conditions.

This dissertation …

Typeface: Machine-Viewing Gentrification On Storefront Imagery In Bedford-Stuyvesant, Brooklyn, Alexander Mcquilkin

Dissertations, Theses, and Capstone Projects

Gentrification—broadly, the replacement of a less powerful group by a more powerful one in an urban context—is oft-discussed in the popular press, but its definition is much-debated in the urban planning literature. Furthermore, academic treatments of displacement understandably focus on measurable yet fairly abstract indicators like changes in rent or income, whereas neighborhood change is often registered by residents on the ground using visual, but difficult-to-quantify markers like retail turnover. This project uses image recognition technology on a set of storefront photos to index the visual streetscape of a neighborhood, as well as to track changes to that portrait over …

Chatgpt Didn’T Write This: Evaluating The Impact Of Llms With A Case Study In Grading Cuny Language Immersion Program Student Essays, Benjamin Inbar

Dissertations, Theses, and Capstone Projects

This study evaluates the capabilities and limitations of large language models (LLMs), specifically OpenAI’s ChatGPT-4o, in grading essays from students in the City University of New York’s Language Immersion Program. The program serves English language learners with diverse linguistic and demographic backgrounds, offering intensive language instruction to prepare students for academic success in college. Using a dataset of 30 pre- and post-program essays scored by program instructors and ChatGPT-4o under three paradigms, this research explores the alignment between human and AI-generated scores across five rubric-based competency areas. Findings reveal that ChatGPT-4o aligns moderately with human grading, with the strongest agreement …

Advancing Affective Computing: Emotion Recognition And Tracking Across Diverse Contexts (Varied Environments), Shao Liu

Dissertations, Theses, and Capstone Projects

Affective Computing (AC) is an interdisciplinary field that recognizes, interprets, and processes human emotions. Emotions are complex, involving consciousness, physical sensations, and behavioral expressions, and are significant in various domains like mental health, human-computer interaction, and social security. Real-world applications of AC include monitoring drivers’ emotional states to improve road safety and understanding the emotions expressed by artists in visual arts. Traditional methods relying on facial expressions often fall short due to the nuanced nature of emotions, which vary across individuals, cultures, and contexts. Accurate AC systems require sophisticated, multimodal models to handle these variations and external factors like noise …

A Machine Learning Approach To Discovering Physical Models Of Galaxy Formation, Festa Bucinca

Dissertations, Theses, and Capstone Projects

Galaxies are the breathtakingly beautiful starry islands of the Universe. The process of galaxy formation involves the transformation from simple initial conditions in the early Universe to the complex galaxy structures we observe today. Spanning an immense spatial range and tremendous time scales - from the vastness of the Universe to the scale of individual stars - the physics of galaxy formation is both complex and crucial for understanding the Universe we live in. However, despite significant advancements, our theoretical understanding of galaxy formation remains incomplete.

In the era of big data available from hydrodynamical simulations and observations, Machine Learning …

La Vida: Towards A Motivated Goal Reasoning Agent, Ursula Addison

Dissertations, Theses, and Capstone Projects

An autonomous agent deployed to operate over extended horizons in uncertain environments will encounter situations for which it was not designed. A class of these situations involves an invalidation of agent goals and limited guidance in establishing a new set of goals to pursue. An agent will benefit from some mechanism that will allow it to pursue new goals under these circumstances such that the goals are broadly useful in its environment and take advantage of its existing skills while aligning with societal norms. We propose augmenting a goal reasoning agent, i.e., an agent that can deliberate on and self-select …

Assessing Job Vulnerability And Employment Growth In The Era Of Large Language Models (Llms), Prudence P. Brou

Dissertations, Theses, and Capstone Projects

This paper explores the impact of Large Language Models (LLMs) and artificial intelligence (AI) on white-collar occupations in the context of job vulnerability and employment growth. Utilizing the Kaggle dataset "Occupation Salary and Likelihood of Automation," the study employs a data-driven approach to analyze trends across states. Through interactive data visualization, the project aims to provide actionable insights for affected workers, businesses, and policymakers navigating the changing dynamics of the workforce amidst technological advancements.

Context In Computer Vision: A Taxonomy, Multi-Stage Integration, And A General Framework, Xuan Wang

Dissertations, Theses, and Capstone Projects

Contextual information has been widely used in many computer vision tasks, such as object detection, video action detection, image classification, etc. Recognizing a single object or action out of context could be sometimes very challenging, and context information may help improve the understanding of a scene or an event greatly. However, existing approaches design specific contextual information mechanisms for different detection tasks.

In this research, we first present a comprehensive survey of context understanding in computer vision, with a taxonomy to describe context in different types and levels. Then we proposed MultiCLU, a new multi-stage context learning and utilization framework, …

Deep Learning-Based Human Action Understanding In Videos, Elahe Vahdani

Dissertations, Theses, and Capstone Projects

The understanding of human actions in videos holds immense potential for technological advancement and societal betterment. This thesis explores fundamental aspects of this field, including action recognition in trimmed clips and action localization in untrimmed videos. Trimmed videos contain only one action instance, with moments before or after the action excluded from the video. However, the majority of videos captured in unconstrained environments, often referred to as untrimmed videos, are naturally unsegmented. Untrimmed videos are typically lengthy and may encompass multiple action instances, along with the moments preceding or following each action, as well as transitions between actions. In the …

Thermodynamics Of Learning With Parametric Probabilistic Models, Shervin Sadat Parsi

Dissertations, Theses, and Capstone Projects

This study delves into the learning process within the Probabilistic Parametric Models (PPMs) framework from a unique thermodynamic perspective. By exploring the core concepts of thermodynamics and its innate connection with information theory, we showcase how this interdisciplinary approach can effectively contribute to the domain of machine learning. In the initial chapter, we establish the link between the learning problem in PPMs and a thermodynamic process by reframing various elements of the learning process within the context of thermodynamics. We introduce novel information-theoretic measurements that provide insights into the information learned in both the parameter space and the overall performance …

Out-Of-Distribution Generalization Of Deep Learning To Illuminate Dark Protein Functional Space, Tian Cai

Dissertations, Theses, and Capstone Projects

Dark protein illumination is a fundamental challenge in drug discovery where majority human proteins are understudied, i.e. with only known protein sequence but no known small molecule binder. It's a major road block to enable drug discovery paradigm shift from single-targeted which looks to identify a single target and design drug to regulate the single target to multi-targeted in a Systems Pharmacology perspective. Diseases such as Alzheimer's and Opioid-Use-Disorder plaguing millions of patients call for effective multi-targeted approach involving dark proteins. Using limited protein data to predict dark protein property requires deep learning systems with OOD generalization capacity. Out-of-Distribution (OOD) …

Optimization And Application Of Graph Neural Networks, Shuo Zhang

Dissertations, Theses, and Capstone Projects

Graph Neural Networks (GNNs) are widely recognized for their potential in learning from graph-structured data and solving complex problems. However, optimal performance and applicability of GNNs have been an open-ended challenge. This dissertation presents a series of substantial advances addressing this problem. First, we investigate attention-based GNNs, revealing a critical shortcoming: their ignorance of cardinality information that impacts their discriminative power. To rectify this, we propose Cardinality Preserved Attention (CPA) models that can be applied to any attention-based GNNs, which exhibit a marked improvement in performance. Next, we introduce the Directional Node Pair (DNP) descriptor and the Robust Molecular Graph …

Structural Anomaly Detection, Shoufu Luo

Dissertations, Theses, and Capstone Projects

As computer systems become more complex and powerful, the threat of sophisticated and persistent computer attacks increases dramatically. Traditional intrusion detection systems that rely on log analysis struggle to keep pace with these evolving threats, as the attacking trails are often buried in high-volume and high-velocity legitimate activities in the system. Despite tremendous progress in applying machine learning techniques to anomaly-based intrusion detection, such methods continue to suffer from a high false positive rate due to the diversity and variability of individual behavior.To address this problem, this thesis proposes a new framework for detecting structural anomalies in computer systems. The …

Evaluating Neural Networks As Cognitive Models For Learning Quasi-Regularities In Language, Xiaomeng Ma

Dissertations, Theses, and Capstone Projects

Many aspects of language can be categorized as quasi-regular: the relationship between the inputs and outputs is systematic but allows many exceptions. Common domains that contain quasi-regularity include morphological inflection and grapheme-phoneme mapping. How humans process quasi-regularity has been debated for decades. This thesis implemented modern neural network models, transformer models, on two tasks: English past tense inflection and Chinese character naming, to investigate how transformer models perform quasi-regularity tasks. This thesis focuses on investigating to what extent the models' performances can represent human behavior. The results show that the transformers' performance is very similar to human behavior in many …

The Interaction Of Different Primary Producers And Physical And Chemical Dynamics Of An Urban Shallow Lake, Majid Sahin

Dissertations, Theses, and Capstone Projects

An artificial urban shallow lake, Prospect Park Lake (PPL), is situated on a terminal moraine in Brooklyn New York, and supplied with municipal water treated with ortho-phosphates. The constant input of the phosphate nutrient is the primary source of eutrophication in the lake. The numerous pools along the water course houses various aquatic phototrophs, which influence the water quality and the state of the system, driving conditions into favoring the survival of their species. In the first half of the dissertation, the focus of the project is on analyzing how the different primary producers in different regions of PPL affect …

Finite Gaussian Neurons: Defending Against Adversarial Attacks By Making Neural Networks Say "I Don’T Know", Felix Grezes

Dissertations, Theses, and Capstone Projects

In this work, I introduce the Finite Gaussian Neuron (FGN), a novel neuron architecture for artificial neural networks aimed at protecting against adversarial attacks.
Since 2014, artificial neural networks have been known to be vulnerable to adversarial attacks, which can fool the network into producing wrong or nonsensical outputs by making humanly imperceptible alterations to inputs. While defenses against adversarial attacks have been proposed, they usually involve retraining a new neural network from scratch, a costly task.

My works aims to:
- easily convert existing models to Finite Gaussian Neuron architecture,
- while preserving the existing model's behavior on real …

Data-Centric Machine Learning For Speech And Audio, Ali Raza Syed

Dissertations, Theses, and Capstone Projects

There is growing recognition of the importance of data-centric methods for building machine learning systems. Data-centric methods assume a fixed model and iterate over the data to improve system performance. This is in contrast to traditional model-centric approaches, which assume a fixed dataset and iterate over models for the same ends. Data-centric machine learning is driven by the observation that, beyond the size of the training data, model performance depends on factors such as the quality of the annotations, and whether the data are representative of conditions in which models will be deployed. This is particularly of interest in the …

Influence Level Prediction On Social Media Through Multi-Task And Sociolinguistic User Characteristics Modeling, Denys Katerenchuk

Dissertations, Theses, and Capstone Projects

Prediction of a user’s influence level on social networks has attracted a lot of attention as human interactions move online. Influential users have the ability to influence others’ behavior to achieve their own agenda. As a result, predicting users’ level of influence online can help to understand social networks, forecast trends, prevent misinformation, etc. The research on user influence in social networks has attracted much attention across multiple disciplines, from social sciences to mathematics, yet it is still not well understood. One of the difficulties is that the definition of influence is specific to a particular problem or a domain, …

Identifying, Evaluating And Applying Importance Maps For Speech, Viet Anh Trinh

Dissertations, Theses, and Capstone Projects

Like many machine learning systems, speech models often perform well when employed on data in the same domain as their training data. However, when the inference is on out-of-domain data, performance suffers. With a fast-growing number of applications of speech models in healthcare, education, automotive, automation, etc., it is essential to ensure that speech models can generalize to out-of-domain data, especially to noisy environments in real-world scenarios. In contrast, human listeners are quite robust to noisy environments. Thus, a thorough understanding of the differences between human listeners and speech models is urgently required to enhance speech model performance in noise. …

Representation Learning For Chemical Activity Predictions, Mohamed S. Ayed

Dissertations, Theses, and Capstone Projects

Computational prediction of a phenotypic response upon the chemical perturbation on a biological system plays an important role in drug discovery and many other applications. Chemical fingerprints derived from chemical structures are a widely used feature to build machine learning models. However, the fingerprints ignore the biological context, thus, they suffer from several problems such as the activity cliff and curse of dimensionality. Fundamentally, the chemical modulation of biological activities is a multi-scale process. It is the genome-wide chemical-target interactions that modulate chemical phenotypic responses. Thus, the genome-scale chemical-target interaction profile will more directly correlate with in vitro and in …

Piecewise Linear Manifold Clustering, Artyom Diky

Dissertations, Theses, and Capstone Projects

This work studies the application of topological analysis to non-linear manifold clustering. A novel method, that exploits the data clustering structure, allows to generate a topological representation of the point dataset. An analysis of topological construction under different simulated conditions is performed to explore the capabilities and limitations of the method, and demonstrated statistically significant improvements in performance. Furthermore, we introduce a new information-theoretical validation measure for clustering, that exploits geometrical properties of clusters to estimate clustering compressibility, for evaluation of the clustering goodness-of-fit without any prior information about true class assignments. We show how the new validation measure, when …

Solving Multiple Inference In Graphical Models, Cong Chen

Dissertations, Theses, and Capstone Projects

For inference problems in graphical models, much effort has been directed at algorithms for obtaining one single optimal prediction. In practice, the data is often noisy or incomplete, which makes one single optimal solution unreliable. To address this problem, multiple Inference is proposed to find several best solutions, M-Best, where multiple hypotheses are preferred for advanced reasoning. People use oracle accuracy as an evaluation criterion expecting one of the solutions has high accuracy with the ground truth. It has been shown that it is beneficial for the top solutions to be diverse. Approaches for solving diverse multiple inference are proposed …

Adversarial Training For Skill Learning In A Mobile Robot, Todd W. Flyr

Dissertations, Theses, and Capstone Projects

Machine Learning in mobile robotics is sometimes hampered by the difficulties associated with the creation of a large corpus of labeled data that most neural network based learning algorithms demand. In recent years, advances in the field of machine learning have been facilitated via the creation of large collaboratively-created labeled training datasets that researchers can use as the basis for experiments to validate and improve their candidate neural network architectures. For the field of robotics, however, tasks are so disparate and the physical devices so varied that in most cases the creation of collaborative benchmark datasets are impractical. Obtaining data …

Learn Biologically Meaningful Representation With Transfer Learning, Di He

Dissertations, Theses, and Capstone Projects

Machine learning has made significant contributions to bioinformatics and computational biol­ogy. In particular, supervised learning approaches have been widely used in solving problems such as bio­marker identification, drug response prediction, and so on. However, because of the limited availability of comprehensively labeled and clean data, constructing predictive models in super­ vised settings is not always desirable or possible, especially when using data­hunger, red­hot learning paradigms such as deep learning methods. Hence, there are urgent needs to develop new approaches that could leverage more readily available unlabeled data in driving successful machine learning ap­ plications in this area.

In my dissertation, …

Metareasoning, Opportunistic Exploration, And Explanations For Autonomous Indoor Navigation, Raj Korpan

Dissertations, Theses, and Capstone Projects

Autonomous indoor navigation is an important task for mobile robots deployed without a map in real-world environments, such as museums or offices. While it travels, an autonomous robot navigator must contend with lack of prior knowledge, sensor noise, actuator error, and inquisitive people. This dissertation addresses these challenges with a cognitively-based hierarchical reasoning architecture that incorporates learning, exploration, reactivity, planning, heuristics, and explanations. Evaluation by simulation in large, complex, indoor environments shows that a robot controller can successfully navigate without a detailed map of every obstruction's location when it performs limited initial global exploration and plans in its learned spatial …

Speech Enhancement Using Speech Synthesis Techniques, Soumi Maiti

Dissertations, Theses, and Capstone Projects

Traditional speech enhancement systems reduce noise by modifying the noisy signal to make it more like a clean signal, which suffers from two problems: under-suppression of noise and over-suppression of speech. These problems create distortions in enhanced speech and hurt the quality of the enhanced signal. We propose to utilize speech synthesis techniques for a higher quality speech enhancement system. Synthesizing clean speech based on the noisy signal could produce outputs that are both noise-free and high quality. We first show that we can replace the noisy speech with its clean resynthesis from a previously recorded clean speech dictionary from …

3d Object Detection, Instance Segmentation And Classification From 3d Range And 2d Color Images, Xiaoke Shen

Dissertations, Theses, and Capstone Projects

We address the problem of 3D object detection and instance segmentation by proposing a novel object segmentation and detection system. First, we detect 2D objects based on RGB, Depth only, or RGB-D images. A 3D convolutional-based system, named Frustum VoxNet, is proposed. This system 1) generates frustums from 2D detection results, 2) proposes 3D candidate voxelized images for each frustum, and uses a 3D convolutional neural network (CNN) based on these candidates voxelized images to perform the 3D instance segmentation and object detection. Although the volumetric data representation is widely used for 3D object classification, there are fewer works on …

A New Feature Selection Method Based On Class Association Rule, Sami A. Al-Dhaheri

Dissertations, Theses, and Capstone Projects

Feature selection is a key process for supervised learning algorithms. It involves discarding irrelevant attributes from the training dataset from which the models are derived. One of the vital feature selection approaches is Filtering, which often uses mathematical models to compute the relevance for each feature in the training dataset and then sorts the features into descending order based on their computed scores. However, most Filtering methods face several challenges including, but not limited to, merely considering feature-class correlation when defining a feature’s relevance; additionally, not recommending which subset of features to retain. Leaving this decision to the end-user may …