Artificial Intelligence and Robotics Commons^™

Students-Generative Ai Interaction Patterns And Its Impact On Academic Writing, Jinhee Kim, Sang-Soog Lee, Rita Detrick, Jialin Wang, Na Li

STEMPS Faculty Publications

Considering both the transformative opportunities and challenges presented by generative AI (GenAI) in academic writing, effectively integrating GenAI into the academic setting becomes a significant need requiring prioritization. Yet, there is limited understanding regarding the nature of interactions between different types of students, what behavioral patterns students exhibit during a student-GenAI interaction (SAI) on a given task, and how these different SAI patterns relate to the actual writing task performance. This study, therefore, aimed to identify SAI patterns of academic writing tasks depending on students’ level of AI literacy and examine the differences in academic writing performance between the identified …

Students' Perception Of Generative Ai-Assisted Collaborative Argumentation, Jinhee Kim, Seongryeong Yu, Rita Detrick, Liangjie Fan, Na Li

STEMPS Faculty Publications

The rapid scaling of generative artificial intelligence (GenAI) technology presents opportunities for personalised learning experiences and facilitates collaborative learning, including collaborative argumentation (CA). However, empirical research examining students' perceptions of GenAI-assisted CA within classroom contexts remains limited. This study explored university students' experiences with GenAI-assisted CA through in-depth interviews with 36 students following a CA activity using a ChatGPT4-embedded argumentation platform developed by the research team. Findings indicate that students viewed GenAI as serving multiple roles, including tool, facilitator, teaching assistant and machine buddy. Students perceived that GenAI-assisted CA could empower task performance and create a collaborative learning environment. Meanwhile, …

Standardization Of Neuromuscular Reflex Analysis—Role Of Fine-Tuned Vision-Language Model Consortium And Openai Gpt-Oss Reasoning Llm-Enabled Decision Support System, Eranga Bandara, Ross Gore, Sachin Shetty, Ravi Mukkamala, Christopher K. Rhea, Brittany S. Samulski, Amin Hass, Atmaram Yarlagadda, Shaifali Kaushik, Malith De Silva, Andriy Maznychenko, Inna Sokolowska, Kasun De Zoysa

VMASC Publications

Background/Objectives: Accurate assessment of neuromuscular reflexes, such as the Hoffmann reflex (H-reflex), plays a critical role in sports science, rehabilitation, and clinical neurology. Conventional interpretation of H-reflex electromyography (EMG) waveforms is subject to inter-rater variability and interpretive bias, limiting reliability and standardization. This study aims to develop an automated, interpretable, and robust agentic AI–driven framework for H-reflex waveform analysis. Methods: We propose a fine-tuned Vision–Language Model (VLM) consortium combined with a reasoning Large Language Model (LLM)–enabled decision support system for automated H-reflex interpretation. Multiple VLMs were fine-tuned on curated datasets of H-reflex EMG waveform images annotated with expert clinical observations, …

Generative Ai And Llm Applications In Renewable Energy And Smart Grids: A Systematic Review For The Sustainable Energy Transition, Umit Cali, Ugur Halden, Merlinda Andoni, Ferhat Ozgur Catak, Si Chen, Benoit Couraud, Emre Kantar, Samuel Knapper, Ibrahim Kucukdemiral, Huseyin Kusetogullari, Murat Kuzlu, Yashar Mousavi, Sonam Norbu, Taha Selim Ustun, David Flynn

Engineering Technology Faculty Publications

The global energy transition toward decarbonization and digitalization requires advanced methods to manage decentralized, data-intensive cyber-physical energy systems. This systematic review analyzes 106 research studies on Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) in renewable energy and smart grids, organized into seven application clusters covering forecasting, system design, operation, reliability, data and cybersecurity, and energy markets. The review situates these applications within a Cyber-Physical-Social Systems (CPSS) framework. Results show that GANs dominate current applications (47.2%), followed by LLMs (10.4%) and VAEs (9.4%), with growing adoption of diffusion and score-based models (7.5% each). Selected studies report improved probabilistic forecasting …

Adaptive Multi-Grade Deep Learning For Highly Oscillatory Fredholm Integral Equations Of The Second Kind, Jie Jiang, Yuesheng Xu

Mathematics & Statistics Faculty Publications

This paper studies the use of Multi-Grade Deep Learning (MGDL) for solving highly oscillatory Fredholm integral equations of the second kind. We provide rigorous error analyses of continuous and discrete MGDL models, showing that the discrete model retains the convergence and stability of its continuous counterpart under sufficiently small quadrature error. We identify the DNN training error as the primary source of approximation error, motivating a novel adaptive MGDL algorithm that selects the network grade based on training performance. Numerical experiments with highly oscillatory (including wavenumber 500) and singular solutions confirm the accuracy, effectiveness and robustness of the proposed approach.

Artificial Intelligence And Machine Learning In Smart Vaginal Formulation Development, Deborah A. Ogundemuren, Vivek Agrahari, Andrew P. Wong, Carolina Herrera, Margaret O. Ilomuanya, Gustavo F. Doncel

CONRAD Publications

Vaginal drug delivery in women's health remains underutilized and insufficiently studied, largely due to the complexity and dynamic nature of the vaginal microenvironment. Variations in vaginal pH, hormonal levels, and microbiota composition introduce significant biological variability, complicating formulation design and contributing to inconsistent therapeutic outcomes and poor patient adherence. Conventional vaginal formulations often fail to account for these individual differences, highlighting the need for more adaptive and predictive approaches. Emerging advances in artificial intelligence (AI) and machine learning (ML) offer promising strategies to address these challenges by enabling multi-parameter, data-driven formulation development that explicitly considers biological variability. Despite their transformative …

Designing Ai Systems To Support A Productive-Failure-Based Learning: Insights From Adult Learners On Ai Applications And Ai System Design Principles, Jinhee Kim, Xi Lin, Seongryeong Yu, Rita Detrick

STEMPS Faculty Publications

Emerging capabilities of generative artificial intelligence (GenAI) offer significant potential to support productive failure (PF)-based learning, which engages adult learners (ALs) in exploring problems before instruction and learning from their initial attempts. However, the effective use of AI to support multifaceted areas of PF-based learning, including problem generation, exploration, consolidation, and knowledge assembly, is limited. Furthermore, AI design principles to support PF-based learning remain under-researched. This study, therefore, aims to investigate ALs’ perceptions of AI applications in enhancing PF-based learning and to explore the essential design principles of AI systems for PF-based learning. To achieve these aims, the study conducted …

Srgan-Based Deep Learning Framework For Wind Turbine Damage Detection From Sentinel-2 Imagery, Kübra Çakir, Onur Elma, Murat Kuzlu

Engineering Technology Faculty Publications

The operational reliability of wind turbines is critical for sustainable energy production in smart grids. This study proposes a remote monitoring approach using perceptually enhanced satellite imagery. Sentinel-2 multispectral data (10 m resolution) has been processed with a Super-Resolution Generative Adversarial Network (SRGAN) to improve visual quality to a perceptual resolution of 30 cm. Although true spatial refinement is not achieved, the sharper structural details enhance classification accuracy. The data set comprises 15,000 images—10,000 SRGAN-enhanced and 5000 augmented through rotation, zoom in, increasing brightness, noise addition, and blurring. A custom Convolutional Neural Network (CNN) has been trained to classify turbines …

Interpretable Machine Learning For Bridge-Pier Scour Prediction And Flood Resilience, Adil Khan, Dalya Ismael

Engineering Technology Faculty Publications

Bridge-pier scour is a leading cause of flood-induced bridge failure, yet practice still lacks transparent, physics-informed tools that link data-driven prediction with design guidance. This study develops an interpretable, physics-aware machine-learning framework to predict equilibrium scour depth and translate those predictions into actionable strategies for flood-resilient infrastructure. Using the 2014 U.S. Geological Survey Pier-Scour Database (569 laboratory cases), five models: Gradient Boosting, AdaBoost (Tree), XGBoost, Gaussian Process (RBF kernel), and Kernel Ridge (polynomial), were trained and evaluated with K-fold cross-validation. Model performance was evaluated using R², RMSE, and MAE. Gradient Boosting performed best, achieving training and testing R² of 0.99 …

The Devil Is In The Det[Ai]Ls: Ai Agents, Ghost Students, And The Crisis Of Verified Presence In An Agentic Ai World, Aras Bozkurt, Helen Crompton, Caroline Fell Kurban

STEMPS Faculty Publications

The transition from reactive Generative Artificial Intelligence (GenAI) to agentic AI systems marks a categorical shift in digital education, moving beyond simple content generation to goal-oriented, autonomous execution. This paper explores the emergence of the “ghost student”: a digital surrogate created by the coupling of Large Language Models (the “mind”) and agentic AI browsers (the “body”). These entities are capable of navigating Learning Management Systems (LMS), engaging with content, and completing assessments with human-like mimicry, often rendering the actual learner’s presence optional. We argue that this phenomenon creates a verification gap that traditional proctoring and detection tools are structurally unable …

Instructional Designers' Reflections On Generative Ai Use For Scenario-Based And Performance-Centered Learning: A Collective Autoethnography, Nour El Houda Maache, Dan V. Dao, Jiyoon Jung, Jayanth Nadheri, Nari Kim, Chikezie Ozuzu, Xinyue Ren

STEMPS Faculty Publications

This study examines how instructional designer-instructors (IDIs) use and evaluate generative artificial intelligence (GenAI) when designing scenario-based and performance-centered authentic learning in higher education. Using a collective autoethnography (CAE) approach, the study draws on semi-structured interviews and reflective narratives from five IDIs with varied professional experience. Findings indicate that GenAI enhanced design capacity by accelerating scenario development, translating complex content, and supporting scenario-based and performance-based task construction. At the same time, participants reported limitations related to contextual misalignment, output unreliability, and the cognitive demands of prompt refinement. Across cases, effective integration depended on sustained human oversight, disciplinary judgment, and ethical …

Chatgpt In Secondary English Language Learning: Educators' And Students' Literacy, Perceptions, And Experiences, Victoria Brenes, Sierra Outerbridge, Xinyue Ren

STEMPS Faculty Publications

The increasing use of generative artificial intelligence (GenAI) has shown the potential of transforming teaching and learning practices in various educational settings, such as in English language learning (ELL). As English language learners (ELLs) often experience many challenges and barriers in schools in the United States, it is urgent to leverage the educational affordances of GenAI in fostering the effectiveness of ELL. Given the limited research investigating GenAI adoption, especially ChatGPT literacy within K-12 ELL, this convergent mixed methods research aims to investigate students' and teachers’ perceptions of using ChatGPT and their ChatGPT literacy in secondary ELL contexts. We will …

Redefining Educational Technology: A Critical Collaborative Inquiry, Aras Bozkurt, Helen Crompton, Robert Farrow, Agnes Kukulska-Hulme, Jon Dron, Richard West, Agnieszka (Aga) Palalas, Matth Bower, Junhong Xiao, Ahmed Tlili, Danah Henriksen, Angelica Pazurek, Henk Huijser, Thomas K. F. Chiu, Petar Jandrić, Katy Jordan, John Curry, Royce Kimmons, Mutlu Cukurova, Thomas Reeves, Gwo-Jen Hwang, Peter Shea, Jason Lodge, Martin Weller, Davy Ng, Tutaleni Iita Asino

STEMPS Faculty Publications

Educational technologists have not settled on a fixed definition of the field and likely never will. However, attempting to define the field helps to understand the epistemological meanings that shape what the field sees, values, and considers worth pursuing. Through a critical historical review spanning over a century, alongside theoretical engagement with the concepts of entanglement and distributed agency, this paper identifies three key insufficiencies in current educational technology frameworks. These are the persistence of an instrumental-facilitative paradigm that treats technology as a resource deployed by human agents; the theoretical dissolution of the pedagogy-technology dichotomy that existing definitions have not …

A Comparison Of Conversational Chatbots And The Internet For Consumer Information Search, Wondwesen Tafesse, Yoseph Mamo

STEMPS Faculty Publications

This study compares consumer perceptions of conversational chatbots and the internet for information search. While the internet is a mature platform, conversational chatbots represent an emerging technology, and insight into how consumers view them in relation to the internet for information search is lacking. Drawing on the information source utility perspective, the study builds a comparative model based on four key dimensions: information currency, information customisation, information trustworthiness, and media richness. Additionally, the study investigates consumers’ prior experience with conversational chatbots as a moderating factor. Data was collected from 191 respondents recruited through MTurk. Paired sample t-tests assessed mean differences …

A Large Language Model-Based Academic Advising Assistant For Engineering Technology Students, Murat Kuzlu, Vukica M. Jovanovic, Katherine Smith, Otilia Popescu, Adel El-Shahat, Ryan Cotton, Kayla Marie Seegers, William Austin Henderson

Engineering Technology Faculty Publications

In recent years, Artificial Intelligence (AI)-based solutions, particularly Large Language Models (LLMs), have been applied to a variety of domains, such as energy, finance, transportation, healthcare, and education. Among these domains, education has become increasingly popular due to strong interest among educators and students. This study proposes an academic advising assistant system that uses LLMs to help Engineering Technology (ET) students plan their course load based on their educational history, departmental course offerings, and personal constraints, such as their preferred semester course load. The proposed LLM-based academic advising assistant system maintains a database of students' course histories and upcoming course …

Improving Medical Diagnostics With Vision-Language Models: Convex Hull-Based Uncertainty Analysis, Ferhat Ozgur Catak, Murat Kuzlu, Taylor Patrick, Michel Audette

Engineering Technology Faculty Publications

In recent years, vision-language models (VLMs) have been applied to various fields, including healthcare, education, finance, and manufacturing, with remarkable performance. However, concerns remain regarding VLMs’ consistency and uncertainty, particularly in critical applications such as healthcare, which demand a high level of trust and reliability. This paper proposes a novel approach to evaluate uncertainty in VLMs’ responses using a convex hull approach on a healthcare application for visual question answering (VQA). For any VLM, temperature refers to a sampling parameter used in probabilistic generation, which controls the randomness of the model’s output. The LLM-CXR model is selected as the medical …

Artificial Intelligence (Ai) In Educating Next Generation Of Engineering Technology Students, Adel El-Shahat, Murat Kuzlu, Vukica M. Jovanovic, Katherine Smith, Abdullah Al Mamun, Otilia Popescu

Engineering Technology Faculty Publications

Artificial Intelligence (AI) is transforming education, particularly for electrical engineering technology (EET) students, by presenting adaptive learning, immediate responses, and unconventional tools. Therefore, this paper proposes investigating modern learning to employ AI in educating future electrical engineering technology students. Firstly, the paper explores how to shape AI knowledge for EET students, supplying them with hands-on skills in AI tasks, clarifying coding, data analysis, and AI ethical usage. Then, as educators, what are the efficient AI tools to utilize in teaching, such as tailored tutoring, automated code assessment, AI-driven design/simulation, lecture dictation, and smart content creation? Key tools, for instance, Google …

A Methodological Framework For Modernizing Engineering Course Content With Large Language Models, Katherine Smith, Dalya Ismael, Otilia Popescu, Murat Kuzlu, Adel El-Shahat, Vukica M. Jovanovic

Engineering Technology Faculty Publications

The rapid evolution of technology presents challenges for engineering educators. While the core engineering methods often remain relevant over time, course materials rapidly become outdated in presentation and pedagogical approach. This paper presents a methodological framework for using large-language models (LLMs) to modernize engineering course content with a case study in an advanced technical analysis course.

The methodology follows a phased approach that is designed to be repeatable and verify the accuracy and completeness of course content. During the first phase, an LLM is used to map outdated text-heavy content to a modern format using a LaTeX template. The second …

Generative Ai-Driven Optimization In Flexible And Reconfigurable Manufacturing Systems, Salah Hammedi, Hicham Chaoui, Lotfi Nabli

Electrical & Computer Engineering Faculty Publications

Flexible and Reconfigurable Manufacturing Systems (FRMSs) are essential for coping with variability in modern production environments; however, efficient scheduling and rapid reconfiguration remain challenging. This paper presents a hybrid optimization framework that integrates Colored Petri Net (CPN) modeling with Generative Artificial Intelligence (GenAI) to enhance scheduling performance and system adaptability. The CPN formalism ensures verifiable modeling of system dynamics, while a transformer-based generative model produces candidate scheduling and reconfiguration strategies. Simulation experiments were conducted under static, dynamic, and adaptive scenarios, including machine breakdowns and dynamic job arrivals. Performance was evaluated using makespan, mean flow time, machine utilization, and reconfiguration latency. …

Multi-Grade Deep Learning, Yuesheng Xu

Mathematics & Statistics Faculty Publications

Deep learning requires solving a nonconvex optimization problem of a large size to learn a deep neural network (DNN). The current deep learning model is of a single-grade, that is, it trains a DNN end-to-end, by solving a single nonconvex optimization problem. When the layer number of the neural network is large, it is computationally challenging to carry out such a task efficiently. The complexity of the task comes from learning all weight matrices and bias vectors from one single nonconvex optimization problem of a large size. Inspired by the human education process which arranges learning in grades, we …

Mg-Spair: Multi-Grade Sparse-Guided Implicit Representation For Training-Data-Free Image Restoration, Jianmin Liao, Lei Huang, Ronglong Fang, Ashley Prater-Bennette, Lixin Shen, Yuesheng Xu

Mathematics & Statistics Faculty Publications

MG-SpaIR is a training-data-free framework for restoring a clean image from a single observation corrupted by a mixture of blur, downsampling, noise, and missing pixels. Building on implicit neural representations (INRs), we introduce a multi-grade residual hierarchy that progressively refines the reconstruction from low to high spatial frequencies across grades, improving representational fidelity and mitigating spectral limitations. To stabilize reconstruction optimization and suppress INR-induced artifacts, we further propose an explicit sparse proximal regularization (e.g., ℓ0 type) applied directly in the high-resolution image domain, which discourages spurious high-frequency patterns while preserving sharp structures. The resulting optimization is solved efficiently via a …

Short-Term Response Mechanisms Of Water Quantity And Quality Of Daihai Lake Under Temperature-Driven Changes, Hao Zhang, Xiaohong Shi, Xianhua Li, Junping Lu, Ruizhong Gao, Xixi Wang, Shuhao Zhang, Longmei Xie, Yu Liu

Civil & Environmental Engineering Faculty Publications

Temperature-driven mechanisms involving complex feedback and lag that affect the evolution of hydrological processes and ecological functions in cold- and arid-region lakes represent a core scientific issue in current hydrology and lake ecology research. In this study, based on month-scale temperature and environmental factor data from Daihai Lake in Inner Mongolia from January to December 2023, statistical methods (redundancy analysis, Tukey's test analysis, correlation analysis, structural equation modeling), time series analysis methods (dynamic time warping), and machine learning methods (random forest) were combined. A hierarchical and phased response framework was constructed that encompassed driver identification, path tracing, lag characterization, and …

Application Paths Of Semantic Modeling In Financial Fraud Detection And Risk Identification, Victor P. Gauthier, Daniel S. Wu

Computer Science Faculty Publications

Financial fraud and risk pose significant threats to economic stability and individual well-being. Traditional detection methods often struggle to keep pace with increasingly sophisticated fraudulent schemes. Semantic modeling, which focuses on understanding the meaning and relationships within data, offers a promising avenue for enhancing fraud detection and risk identification. This review paper explores the application paths of semantic modeling in this domain. We begin with a historical overview of fraud detection techniques, highlighting the limitations of traditional approaches. Subsequently, we delve into core themes, including knowledge graph-based fraud detection and semantic rule-based inference for risk assessment. We then compare and …

Sage: Spatially Aware Gene Selection And Dual-View Embedding Fusion For Domain Identification In Spatial Transcriptomics, Yi He, Yunpei Xu, Liqing Ding, Hong-Dong Li, Yaohang Li, Shaokai Wang

Computer Science Faculty Publications

Despite enabling high-resolution mapping of gene expression within tissues, spatial transcriptomics (ST) still faces challenges in accurately segmenting spatial domains due to complex tissue architecture and limitations of current methods. Most approaches rely on local spatial priors, lack gene-level interpretability, and fall short in capturing structure-discriminative genes or long-range functional relationships, limiting their ability to resolve biologically meaningful architectures. We present Spatially Aware Gene selection and dual-view Embedding fusion (SAGE), a unified and reproducible framework for domain identification in spatial transcriptomics that combines topic-driven gene selection with dual-view embedding fusion to address these gaps. SAGE integrates non-negative matrix factorization (NMF)-based …

Adaptive Self-Attention For Enhanced Segmentation Of Adult Gliomas In Multi-Modal Mri, Evan P. Savaria, Jiangwen Sun

Computer Science Faculty Publications

Every year there are an estimated 80,000–90,000 new glioma cases, highlighting the need for reliable imaging-based decision support. Although deep learning has improved tumor sub-region segmentation, many state-of-the-art models fail to fully capture complementary information across T1, T1Gd, T2, and FLAIR MRI modalities and often operate as “black boxes,” limiting physician trust when precise delineation is critical for surgical planning, radiation targeting, and treatment monitoring. To address these limitations, we propose AIMS, an Adaptive Integrated Multi-Modal Segmentation framework that maintains modality-specific feature streams and employs adaptive self-attention within a hierarchical CNN-Transformer architecture to prioritize and fuse multi-modal MRI features. We …

Quantum Machine Learning Models: Principles, Frameworks, And Computational Challenges, K. A. Jayabalaji, S. Venkata Anand, Dineshkumar Rajendran, Prasanta Chatterjee Biswas, Sardor Omonov, Rubaid Ashfaq

Computer Science Faculty Publications

Quantum machine learning (QML) has become an optimistic avenue of harnessing quantum computation in data-driven modeling, especially of issues with high dimensionality and complicated correlations. Current methods are generally based on fixed or over-parameterized quantum circuits, and hence restricted to scalability as well as unproductive optimization in real-world hardware. This chapter introduces a hybrid quantum-classical learning system that is adaptive and provides principled quantum data encoding, architecture-conscious variational circuit design and resource-optimal optimization. The technique is based on the concepts of quantum architecture search and subspace-preserving transformations to trade expressiveness with trainability, and discretize the quantum model into a classical …

Integration Of Hybrid Quantum-Neuromorphic Ai With Cloud, Edge, And High-Performance Computing Environments, Arun B. Prasad, Ajay Prasad, Dineshkumar Rajendran, Anurag Tiwari, T. Akilan, Islombek Khushvaktov

Computer Science Faculty Publications

The convergence of quantum computing, neuromorphic learning, and distributed cloud infrastructures has occurred very rapidly, and intelligent systems are now providing new opportunities, but the challenge of instability, complexity of orchestration, and noise sensitivity remains in the way of practical integration. The proposed work is based on a hybrid quantum and neuromorphic architecture, which is the integration of event-based neuromorphic adaptation and quantum-assisted global optimization, orchestrated by cloud-HPC. The architecture presents the thermodynamically regularized learning and resourceful task scheduling to the probabilistic search and the continuous local adaptation. Experimental evaluation across financial modeling, medical imaging, and physical system prediction shows …

Design And Analysis Of Modern Quantum Neural Network Architectures For Intelligent Systems, Lakshmi Chandrakanth Kasireddy, Prabhakara Rao Kapula, Dineshkumar Rajendran, Neha Bharani, Srikanth Pulipeti, Islombek Khushvaktov

Computer Science Faculty Publications

Quantum neural networks (QNNs) offer a principled pathway for integrating quantum computation with machine learning through superposition- and entanglement-based representations. This chapter proposes an architecture-aware design and evaluation framework for modern QNNs, emphasizing robustness and system feasibility alongside predictive performance. Multiple architectures variational QNNs, quantum convolutional neural networks, tensor-network hybrids, and fully quantum models—are assessed under a unified protocol. Experimental analysis shows that the proposed architecture-search–guided QNN achieves 91.8% classification accuracy and an F1-score of 0.914, outperforming fixed-template variational QNNs by approximately 5.6 percentage points. Under depolarizing noise with probability p = 0.10, the proposed model retains 85.3% accuracy, whereas …

Cognitive Prosthetic: An Ai-Enabled Multimodal System For Episodic Recall In Knowledge Work, Lawrence Obiuwevwi, Krzystof J. Rechowicz, Vikas Ashok, Sachin Shetty, Sampath Jayarathna

Computer Science Faculty Publications

Modern knowledge workplaces increasingly strain human episodic memory as individuals navigate fragmented attention, overlapping meetings, and multimodal information streams. Existing workplace tools provide partial support through note-taking or analytics but rarely integrate cognitive, physiological, and attentional context into retrievable memory representations. This paper presents the Cognitive Prosthetic Multimodal System (CPMS)—an AI-enabled proof-of-concept designed to support episodic recall in knowledge work through structured episodic capture and natural language retrieval. CPMS synchronizes speech transcripts, physiological signals, and gaze behavior into temporally aligned, JSON-based episodic records processed locally for privacy. Beyond data logging, the system includes a web-based retrieval interface that allows users …

Modeling Joint Visual Attention In Naturalistic Dyadic Interactions, Kuushini Thennakoon, Yasasi Abeysinghe, Bhanuka Mahanama, Vikas Ashok, Sampath Jayarathna

Computer Science Faculty Publications

Joint visual attention (JVA) provides important insight into how individuals coordinate attention during social interaction. Egocentric eye tracking enables the study of JVA in natural, multi-user settings. This work presents a multi-stage framework to identify and analyze JVA using egocentric video and gaze data. The approach consists of three steps: spatiotemporal tube-based visual similarity, gaze-guided object detection, and attention pattern analysis using the ambient–focal coefficient K. Results show that object-focused collaborative activities exhibit high JVA, with object detection capturing higher joint attention than visual similarity, whereas conversation-based or independent activities show lower and more fragmented joint attention. Analysis of K …