Data Science Commons^™

From Attention To Reasoning: Beyond Accuracy In Multimodal Ai, Wayner Barrios

Dartmouth College Ph.D Dissertations

Multimodal large language models have achieved impressive performance on vision-language benchmarks by integrating visual encoders with large language models. Yet a critical gap persists between benchmark accuracy and genuine multimodal understanding: current evaluation frameworks assess performance by final answers alone, rewarding confident predictions while leaving systematic reasoning failures undetected.

This thesis addresses this gap through a unified framework that progresses from understanding to reasoning, using video as the most comprehensive multimodal testbed. Video inherently combines vision, audio, and language with temporal dynamics and massive token redundancy; techniques developed for video's comprehensive challenges transfer naturally to simpler multimodal tasks.

On understanding …

Enhancing Financial Audit Operations Through Ai Anomaly Detection, Nadya Cousin, Rebekah Garza, Talisa Gomez

Posters - 2026

❖ Financial auditing plays a critical role in ensuring accuracy, regulatory compliance, and fraud detection in financial reporting

❖ Traditional audit approaches rely heavily on sampling and manual review processes, limiting their ability to scale with increasing data complexity

❖ The rapid growth of high-volume, high-velocity financial data (big data) has exposed significant limitations in traditional auditing, including:

• Incomplete data coverage
• Delayed anomaly detection
• Increased risk of material misstatements

❖ These limitations create a need for scalable, automated, and data-driven audit solutions

❖ Artificial Intelligence (AI), particularly anomaly detection models, enables:

•  Full-population testing
•  Real-time pattern recognition
•  Proactive risk identification

Low-Complexity Structured Neural Networks And Their Usage In Image And Signal Processing, Adam Kuzmicki

Doctoral Dissertations and Master's Theses

Conventional neural networks face significant challenges due to high computational costs, large parameter counts, and reliance on backpropagation, which restricts their application in resource-constrained and real-time settings. To address these challenges, this thesis proposes three structured neural network (NN) architectures grounded in the theories of sparse and self-contained factorizations of transforms, with applications to image compression, reconstruction, classification, encryption, and also adaptive wideband multi-beam beamforming. The first neural network architecture, named DCTrix-Net, replaces conventional spatial con- volution with highly sparse factorization of the discrete Cosine transform (DCT) complemented by Toeplitz-structured weight initialization, achieving at least 97% FLOP reduction over CNNs, …

Using Ai-Based Predictive Scheduling To Improve Patient Flow And Reduce Wait Times In Healthcare Clinics, Oscar Martinez

Posters - 2026

• Healthcare systems face increasing challenges in patient access and wait times
• Average wait times for specialist care continue to rise, creating:
  • Delays in treatment
  • Reduced patient satisfaction
  • Increased system inefficiencies (Sanford, 2025)
• A major contributor is operational bottlenecks, defined as:
  • Points of congestion that slow or disrupt service flow
• Hospitals typically operate under process layouts, which:
  • Handle diverse patient needs
  • Reduce specialization efficiency
• Contributing factors to bottlenecks:
  • Physician shortages and burnout
  • Administrative burden
  • Inefficient scheduling systems (Moura & Pinho, 2025)
• AI offers potential solutions through:
  • Predictive scheduling
  • Automation of administrative processes
  • Data-driven optimization of patient flow

Optimizing Retail Grocery Inventory Using Ai And Large Language Models: Evidence On Forecast Accuracy, Waste Reduction, And Cost Efficiency, Robert Miller, Stephen Garcia, Brandon Ermis

Posters - 2026

Aim: To evaluate how AI and LLMs improve forecasting accuracy, reduce waste, and enhance inventory decision-making

A General Weighting Theory For Ensemble Learning: Beyond Variance Reduction Via Spectral And Geometric Structure, Ernest Fokoue

Articles

Ensemble learning is traditionally justified as a variance-reduction strategy, explaining its strong performance for unstable predictors such as decision trees. This explanation, however, does not account for ensembles constructed from intrinsically stable estimators-including smoothing splines, kernel ridge regression, Gaussian process regression, and other regularized reproducing kernel Hilbert space (RKHS) methods whose variance is already tightly controlled by regularization and spectral shrinkage. This paper develops a general weighting theory for ensemble learning that moves beyond classical variance-reduction arguments. We formalize ensembles as linear operators acting on a hypothesis space and endow the space of weighting sequences with geometric and spectral constraints. …

Human Subject Studies For The Alignment Of Llm-As-A-Judge Evaluation Metric For Science News, Gabriel Vega Osborne

Knowledge and Creativity Expo

Science news has become an important vehicle to disseminate scientific breakthroughs, discoveries, and technological innovations. With the advancement of large language models and related AI models, it is possible to automatically generate science news from scientific papers, extending the reader population from domain scientists to a broader scope. However, how to evaluate the quality of the generated news warrants research. Traditional token based metrics have been shown to fail to evaluate the semantics and nuances of science news. Inspired by the fact that a major goal of science news is to educate readers with new knowledge, we thus propose knowledge …

Addressing The Problems Of Data Variations, Quality, And Scarcity In Training Deep Neural Networks, Jian Sun

Electronic Theses and Dissertations

The performance of deep neural networks (DNNs) is strongly influenced by the characteristics and quality of the underlying datasets. This Ph.D. dissertation addresses three pervasive data challenges-imbalance, quality degradation, and scarcity-that commonly hinder the effectiveness of DNNs in computer vision (CV) and natural language processing (NLP) applications.

Class imbalance remains one of the most frequent causes of degraded model generalization. While Focal Loss effectively mitigates inter-class imbalance by assigning higher weights to minority classes, it struggles with intra-class imbalance, particularly in video datasets where longer clips dominate feature representation. To address this, I implement and utilize …

Kms-Net: Kolmogorov–Arnold-Based Multi-Scale Attention Network For Cardiac Segmentation, Abid Mehmood, Hassan Ali, David Noule Tolno, Sery Gahouidi Thierry S, Muhammad Saeed, Naeem Ahmed

Research & Publications

Accurate segmentation of cardiac structures in 2D echocardiography is essential for diagnosing cardiovascular disease and computing clinical metrics such as chamber volumes and ejection fraction. Conventional U-Net architectures excel at extracting local spatial features but struggle with long-range dependencies inherent in noisy ultrasound images, while pure Transformer-based models capture global context at the expense of fine boundary detail. To address these limitations, we propose KMS-Net, a novel hybrid segmentation architecture that integrates Kolmogorov–Arnold Networks (KANs), a class of learnable, spline-based function approximators that replace fixed activation functions with trainable nonlinear mappings, alongside multi-scale attention mechanisms. Specifically, spline-based KAN layers (grid …

Anomaly Detection For Multi-System Bug Triage, Gibran Miguel Zavala Gamero, Hayoung Cheon, Mustafa Iqbal

SMU Data Science Review

Large-scale software systems produce vast volumes of logs and telemetry, making manual incident triage slow and error prone. This study presents an unsupervised anomaly detection pipeline that fuses logs, metrics, and traces through late fusion. Using Hybrid Ensemble modeling with Isolation Forest, and Long Short-Term Memory (LSTM) Deep Learning model, the system detects cross-service anomalies producing and assigning a composite triage score reflecting severity and impact. Ranked alerts are categorized into Critical, High, or Medium priorities for review. A retrieval-augmented generation (RAG) layer enriches results with contextual summaries for explainable triage. Evaluated on synthetic multi-service datasets, the pipeline …

Availability Model To Evaluate Ai Data Centers’ Role In Grid Stability, Troy Mcsimov, Trevor S. Kunz, Jeffrey Billo

SMU Data Science Review

The United States has made it clear; it is imperative that the US wins the global AI race. This paper focuses on one of the most challenging puzzle pieces surfaced at the POWER Data Center conference (San Antonio, Sept. 30.); for Electric Reliability Council of Texas (ERCOT) the limiting factor is not generation alone but the need to balance generation and load to preserve grid reliability.

The regulatory landscape fundamentally changed with the passage of Texas Senate Bill 6 in June 2025, which mandates new large loads must "contribute to the recovery of the interconnecting electric utility’s costs" (Texas Legislature, …

Using Ensemble Disagreement To Stabilize Conformal Prediction Under Distribution Shift, Patrick D. Murphy

Master's Theses

Semantic segmentation of eelgrass from drone imagery is crucial for coastal habitat monitoring, restoration, and management, as these habitats continue to see rapid changes due to climate change and human influence. However, the reliability of generalizing a deployed classification model relies on both high-accuracy segmentation as well as robust uncertainty quantification that holds up when conditions change over years or locations. Conformal prediction (CP) is a method that converts a classifier's output into prediction sets with a guaranteed average coverage level for in-distribution data. However, the “vanilla” conformal score can often under-cover in hard or out-of-distribution (OOD) regions under drift. …

Cerebral Documents And Algorithmic Sensemaking: Searching For Expressions In Human And Artificial Cognitive Collaborations, Rebekah L. Cowell

Proceedings from the Document Academy

Generative Artificial Intelligences (AIs) and current advanced large language models (LLMs) are algorithmically designed to generate text-based conversations as conversational agents (CAs), by replicating human language and conversational communication. Pairing human cognition with generative computationally coded cognition. We have never been here before: cerebral and artificial information collaborations and processing producing expressions that may or may not become visible as second-hand/secondary source documents.

Sensemaking or sense(un)making is a unique autonomous human drive cognitively, our information processing is sensemaking in action and expressions and articulations are evidence of the sensemaking cycle. Documentation [expressed or articulated through various mediums] are a product …

Typeface: Machine-Viewing Gentrification On Storefront Imagery In Bedford-Stuyvesant, Brooklyn, Alexander Mcquilkin

Dissertations, Theses, and Capstone Projects

Gentrification—broadly, the replacement of a less powerful group by a more powerful one in an urban context—is oft-discussed in the popular press, but its definition is much-debated in the urban planning literature. Furthermore, academic treatments of displacement understandably focus on measurable yet fairly abstract indicators like changes in rent or income, whereas neighborhood change is often registered by residents on the ground using visual, but difficult-to-quantify markers like retail turnover. This project uses image recognition technology on a set of storefront photos to index the visual streetscape of a neighborhood, as well as to track changes to that portrait over …

Computational Clinical Judgment: Predicting Risk With Large Language Models, Hannah Laqueur, Ryan W. Copus

Faculty Works

For seventy years, research has shown actuarial methods outperform clinical judgment. Yet actuarial approaches have limitations: they generally rely on structured data; cannot exploit rare case-specific details; have limited accuracy where outcome data are scarce or incomplete; and cannot offer case-level justifications. Large language models (LLMs) offer a different approach. Like actuarial methods, they aggregate information algorithmically, but like clinicians, they bring general knowledge and can provide case-level justifications. We prompted seven LLMs to assess rearrest risk from 113 parole hearing transcripts and compared their predictions to a machine learning model trained on 4,000 cases with 91 administrative variables. GPT-5 …

Systematic Approaches To Characterizing Vulnerabilities And Enhancing Robustness Of Text And Vision-Language Models, Poojitha Thota

Computer Science and Engineering Dissertations

The proliferation of artificial intelligence (AI) across critical domains, including news summarization, privacy-policy analysis, and medical decision support, has raised growing concerns about the security and robustness of these systems against adversarial manipulation. This dissertation investigates adversarial robustness in generative AI by addressing three key research goals: (1) characterizing adversarial vulnerabilities across generative models, (2) developing systematic defenses to improve the robustness of generative models, and (3) designing deployment-time safeguards for securing LLM interactions.

Towards the first goal, we characterize adversarial vulnerabilities across text-based and multimodal systems. In abstractive text summarization, we show that inference-time perturbations can exploit lead bias …

Previsit Ai: A Retrieval-Augmented Generation For Patient Readiness In Clinical Encounters, Rolande Umuhoza

All Graduate Theses, Dissertations, and Other Capstone Projects

With healthcare systems under growing pressure from rising patient volumes and shrinking consultation windows, improving how patients communicate with physicians has become essential to delivering quality care. Yet patients routinely arrive at appointments unable to clearly describe their symptoms, recall their medical history, or articulate concerns, contributing to miscommunication, diagnostic inefficiency, and pre-visit anxiety. This study introduces PreVisit AI, a conversational system designed to address this gap through structured, knowledge-based patient preparation. The system is built on a Retrieval-Augmented Generation (RAG) architecture combining HuggingFace sentence embeddings (all-MiniLM-L6-v2), a Chroma vector store, and Google’s Gemini language model over a curated seven-document …

Impact Of Cross-Client Heterogeneity In Federated Learning For Real-World Plant Disease Classification, Ritesh Janga

All Graduate Theses, Dissertations, and Other Capstone Projects

Deep learning applications are being adopted in agricultural image analysis that include challenges of data privacy and limited institutional data and heterogeneity of different types of architectures. However, Federated Learning is a model that allows collaborative training on data that does not have to be shared among parties. Therefore, Federated Learning is an effective method of collaborative training; however, its comparative effectiveness as compared to individual (local) training on diverse architectures has never been examined in an agricultural context. The objective of this study was to examine Federated Learning for the purpose of crop disease classification on extreme non-IID distributed …

Data Defines Success: Algorithm For Dataset Quality Assessment In Deep Learning For Malware Detection, Matei Ionescu

Theses and Dissertations

The field of artificial intelligence is based upon the premise of constructing architectures through which to propagate training data. However, the majority of existing research literature is focused on architecture. While necessary, the attention devoted to the architecture should not so precipitously exceed that of the data. It should be noted that this disparity is not without reasonable cause. Data quality is often exceedingly difficult to verify due to particularities of the field or subfield; LLM repositories of text are distinct from image recognition pictures of dog breeds which are distinct from EEG waveforms of human brains which are distinct …

Network-Aware Airline-Specific Flight Delay Prediction Using Tree-Based Ensemble Models, Mary Dufie Afrane

College of Graduate Studies: Theses & Dissertations

Flight delays pose persistent challenges to the efficiency and reliability of air transportation systems, affecting airlines, airports, regulators, and passengers alike. As traffic demand grows and operational environments become increasingly interconnected, accurately predicting both departure and arrival delays has become crucial for effective planning and mitigation. This study presents a network-aware, airline-specific framework for predicting flight delays in U.S. domestic air transportation systems using tree-based ensemble machine learning models. A large-scale dataset of 1.98 million flights, enriched with weather information, is used to develop predictive models for both departure and arrival delays. To capture the structural and operational complexity of …

Explainable Physics-Based Constraints On Reinforcement Learning For Accelerator Optimization, Jonathan Colen, Malachi Schram, Kishansingh Rajput, Armen Kasparian

Data Science Faculty Publications

We present a reinforcement learning (RL) framework for optimizing particle accelerator experiments that builds explainable physics-based constraints on agent behavior. The goal is to increase transparency and trust by letting users verify that the agent’s decision-making process incorporates suitable physics. Our algorithm uses a learnable surrogate function for physical observables, such as energy, and uses them to fine-tune how actions are chosen. This surrogate can be represented by a neural network or by an interpretable sparse dictionary model. We test our algorithm on a range of particle accelerator optimization environments designed to emulate the Continuous Electron Beam Accelerator Facility at …

Exploring The Synergy Between Very Large Transformer And Lstm Models For Effective Medical Captioning From Videos To Text: The Impact Of Captioning In Healthcare, R. V. Aswiga, Moin Ahmed Zahir

Data Science Faculty Publications

In today’s rapidly evolving digital landscape, the demand for accurate and contextually relevant subtitles for image and video content, particularly in the medical domain, is increasingly critical. Despite the proliferation of visual data across various platforms, existing captioning systems often struggle due to variations in visual settings, complex temporal relationships, and nuanced semantics. Additionally, challenges such as limited datasets, privacy issues, and specialized annotation requirements make medical image captioning particularly difficult. To tackle these challenges, we investigate cutting-edge deep learning methodologies, specifically Transfer Learning and Transformer models, through a comparative analysis. Specifically, we focus on Transfer Learning through the MedVisionCapturer …

Advancing Hate Speech Detection: Binary And Multiclass Approaches From Traditional Methods To Parameter-Efficient And Ontology-Guided Language Models, Mahmoud Abusaqer

Graduate Theses/Dissertations

The widespread proliferation of hate speech on social media platforms poses significant challenges for content moderation and user safety, requiring automated systems that are simultaneously accurate, efficient, and capable of fine-grained distinctions. This thesis investigates hate speech detection through five published manuscripts organized into two complementary threads: binary detection (hateful vs. non-hateful) and multiclass detection across demographic targeting categories. The binary thread progresses from a broad 38-model baseline spanning traditional machine learning, deep learning, and transformer architectures (where RoBERTa reaches 91.48% accuracy and CatBoost remains competitive at 88.60%) to parameter-efficient adaptation, in which Low-Rank Adaptation (LoRA) of large language models …

Srgan-Based Deep Learning Framework For Wind Turbine Damage Detection From Sentinel-2 Imagery, Kübra Çakir, Onur Elma, Murat Kuzlu

Engineering Technology Faculty Publications

The operational reliability of wind turbines is critical for sustainable energy production in smart grids. This study proposes a remote monitoring approach using perceptually enhanced satellite imagery. Sentinel-2 multispectral data (10 m resolution) has been processed with a Super-Resolution Generative Adversarial Network (SRGAN) to improve visual quality to a perceptual resolution of 30 cm. Although true spatial refinement is not achieved, the sharper structural details enhance classification accuracy. The data set comprises 15,000 images—10,000 SRGAN-enhanced and 5000 augmented through rotation, zoom in, increasing brightness, noise addition, and blurring. A custom Convolutional Neural Network (CNN) has been trained to classify turbines …

Graph-Based And Graph-Transformer Representation Learning For Healthcare Data, Rui Wang

Electronic Theses & Dissertations (2024 - present)

Healthcare data exhibit complex structures, including heterogeneous clinical entities, sparse observations, and longitudinal patient trajectories. Effectively modeling such data remains a fundamental challenge in computational healthcare research. Traditional machine learning approaches often rely on flat feature representations that fail to capture relationships among clinical events, limiting their ability to model complex healthcare processes. These challenges motivate structured learning frameworks that capture both relational structure and temporal dynamics in healthcare data. This dissertation develops a series of graph-based representation learning approaches, extended through graph-transformer architectures for modeling complex healthcare data. Such data can be represented as graphs, where nodes correspond to …

Ai-Assisted Surface-Enhanced Raman Spectroscopy For Cardiovascular Diagnostics: From Plasmonic Materials To Clinical Translation, Anju Joshi, Gymama Slaughter

Center for Bioelectronics Publications

Raman spectroscopy (SERS) has emerged as a powerful analytical technique, offering molecular fingerprint specificity and ultrasensitive detection of cardiac biomarkers. Recent advances in plasmonic nanostructures, surface functionalization strategies, and flexible sensing platforms have significantly improved the analytical performance of SERS-based biosensors. In parallel, the integration of artificial intelligence (AI) and machine learning has enabled robust interpretation of complex spectral datasets, facilitating automated biomarker classification and improved diagnostic accuracy in heterogeneous biological environments. Despite these advances, the field remains fragmented, with limited integration between nanomaterial design, biomarker selection, and data-driven analysis, and persistent challenges related to reproducibility, standardization, and clinical validation. …

Swimming In Uncertainty: Filling Data Gaps And Providing An Educational Platform For Beach Water Quality At Tybee Island, Georgia, Lukas Roberson

College of Graduate Studies: Theses & Dissertations

@font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-536870145 1107305727 0 0 415 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:""; margin:0in; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman",serif; mso-fareast-font-family:"Times New Roman";}.MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; mso-font-kerning:0pt; mso-ligatures:none;}div.WordSection1 {page:WordSection1;}

Swimming in beaches water contaminated with high levels of bacteria can make you sick. Current monitoring at the public beaches on Tybee Island consists of weekly monitoring and enumeration of fecal indicator bacteria that takes 24 hours for results. If the number of bacteria exceed regulatory limits, a public health advisory is issued, and affected waters are retested until …

Healthcare Digital Twins: A Methodological Literature Review On Integrating Iot And Ai For Personalized Medicine And Predictive Care, Sara Shahnazinia, Mahsa Tavasoli, Abdolhossein Sarrafzadeh, Ali Karimoddini

Electrical & Computer Engineering Faculty Publications

Digital Twin (DT) technology has the potential to revolutionize healthcare delivery and enhance patient outcomes through personalized and precision medicine, simulation models for operations and interventions, and drug discovery. However, successful implementation of DTs in Internet of Things (IoT) and artificial intelligence (AI) healthcare is contingent upon addressing key challenges such as privacy, ethics, and robust data security. This paper presents a methodological literature review of DT applications in healthcare, systematically analyzing the current state of research, key enabling technologies, and implementation challenges. The review summarizes DT categorization approaches (application-based, technology-based, and real-time function-based); delineates core DT components such as …

A Novel Federated Llm Framework For Distributed Traffic Modelling In Intelligent Transportation Systems, Seerat Kaur

Theses and Dissertations (Comprehensive)

Intelligent transportation systems (ITS) depend on accurate traffic prediction to support congestion management, infrastructure planning, and real-time operational decisions. Despite substantial progress in data-driven forecasting, several challenges continue to limit practical deployment: traffic data is distributed across independent regional authorities, making centralized aggregation infeasible, standard federated aggregation strategies ignore traffic-specific characteristics that meaningfully affect model quality, and existing models produce only numerical outputs without interpretable reasoning that urban planners can act upon. This thesis addresses these challenges through four contributions that collectively advance privacy-preserving, explainable, and scalable traffic forecasting.

The first contribution provides a systematic review of 129 peer-reviewed publications, …

An Explainable Transformer Framework For Sentiment Analysis In Aviation Workforce Data, Sovon Chakraborty, Protiva Das, Fahmid Al Farid, Fuyad Hasan Bhoyan, Farig Yousuf Sadeque, Jia Uddin, Hezerul Abdul Karim

Computer Science Faculty Publications

Aviation is one of the predominant sectors that contribute significantly to the global economy. With the advent of technology, this industry is witnessing a paradigm shift towards data-driven approaches. The morale of the airline employees is barely noticed, which causes fatigue and depression. Furthermore, these mental health issues can be active reasons for destructive accidents. In this research, the authors are focused on collecting insightful information on aviation employees from Glassdoor.com. Moreover, the authors focus on analyzing the sentiments of the employees of renowned aviation companies. Primarily, the authors scraped necessary data from Glassdoor.com and created a dataset named JetJobJoy …