Robotics Commons^™

Enhancing Emotional Accuracy And Behavioral Novelty In Robot Behaviors: A Generative-Discriminative Approach, Rista Baral

Boise State University Theses and Dissertations

Recent advancements in language modeling have improved robotic emotion expressiveness, yet several challenges remain. Many existing robotic expression models rely on fixed rules and static frameworks, which limit their ability to capture the dynamic nature of emotional expression. Additionally, these systems often struggle to balance emotional accuracy with generating novel and varied behaviors. These limitations underscore the need for method capable of delivering both emotionally accurate and diverse robot behaviors.

In this thesis, we addresses these challenges by introducing a new framework for expressive behavior generation in robots. We develop a generative model that generates robot behavior sequences that align …

Bridging Virtual Robots And Physical Tasks Via Augmented Reality, Xiangfei Kong

USF Tampa Graduate Theses and Dissertations

Entry to human-robot interaction research, e.g., conducting empirical experiments, faces a significant economic barrier due to the high cost of physical robots, ranging from thousands to tens of thousands. This cost issue also severely limits the field’s ability to replicate user studies and reproduce the results to verify their reliability, thus offering more confidence to incorporate these findings. Although virtual reality (VR) user studies present a potential solution, it is unclear whether we can confidently transfer the findings to physical robots and physical environments because VR isolates both the physical robot and the physical world where robots operate. To address …

Exploring The Limits Of Multimodal Foundation Models For Visual Temporal Reasoning And Gesture Recognition Tasks, Ziyao Shangguan

Computer Science Theses

Multimodal foundation models (MFMs) have demonstrated impressive capabilities in static vision-language tasks such as image captioning, video summarization, and cross modal retrieval. However, their ability to reason over time—especially in gesture-rich video inputs—remains limited. This thesis investigates the temporal reasoning capabilities of MFMs in the context of gesture understanding, a critical component for enabling more expressive human-robot interaction. Through a preliminary study, we show that prompting-based strategies offer only marginal improvements in temporal reasoning, despite producing accurate frame-by-frame descriptions.

To more rigorously evaluate these limitations, we introduce TOMATO, a benchmark designed to assess visual …

Modelling And Mitigating Interlocutor Confusion In Situated Human-Avatar And Human-Robot Interaction, Na Li

Dissertations

Human-Robot Interaction (HRI) is an important but challenging field focused on improving the interaction between humans and robots, to make the interaction more intelligent and effective. However, building a natural conversational HRI is an interdisciplinary challenge for scholars, engineers, and designers. Achieving successful conversational interaction with a social robot necessitates not only observing a user’s active participation in the interaction but also being aware of their emotional and attitudinal states as the interaction progresses. On the topic of attitudinal states, one field that has received little attention to date is monitoring the user for possible confusion states. Confusion is a …

Multi-Robot Symbolic Task And Motion Planning Leveraging Human Trust Models: Theory And Applications, Huanfei Zheng

All Dissertations

Multi-robot systems (MRS) can accomplish more complex tasks with two or more robots and have produced a broad set of applications. The presence of a human operator in an MRS can guarantee the safety of the task performing, but the human operators can be subject to heavier stress and cognitive workload in collaboration with the MRS than the single robot. It is significant for the MRS to have the provable correct task and motion planning solution for a complex task. That can reduce the human workload during supervising the task and improve the reliability of human-MRS collaboration. This dissertation relies …

Considering The Context To Build Theory In Hci, Hri, And Hmc: Explicating Differences In Processes Of Communication And Socialization With Social Technologies, Andrew Gambino, Bingjie Liu

Human-Machine Communication

The proliferation and integration of social technologies has occurred quickly, and the specific technologies with which we engage are ever-changing. The dynamic nature of the development and use of social technologies is often acknowledged by researchers as a limitation. In this manuscript, however, we present a discussion on the implications of our modern technological context by focusing on processes of socialization and communication that are fundamentally different from their interpersonal corollary. These are presented and discussed with the goal of providing theoretical building blocks toward a more robust understanding of phenomena of human-computer interaction, human-robot interaction, human-machine communication, and interpersonal …

Human-Machine Communication Scholarship Trends: An Examination Of Research From 2011 To 2021 In Communication Journals, Riley J. Richards, Patric R. Spence, Chad Edwards

Human-Machine Communication

Despite a relatively short history, the modern-day study of communication has grown into multiple subfields. To better understand the relationship between Human-Machine Communication (HMC) research and traditional communication science, this study examines the published scholarship in 28 communication-specific journals from 2011–2021 focused on human-machine communication (HMC). Findings suggest limited prior emphasis of HMC research within the 28 reviewed journals; however, more recent trends show a promising future for HMC scholarship. Additionally, HMC appears to be diverse in the specific context areas of research in the communication context. Finally, we offer future directions of research and suggestions for the development of …

Transferring Studies Across Embodiments: A Case Study In Confusion Detection, Na Li, Robert J. Ross

Articles

Human-robot studies are expensive to conduct and difficult to control, and as such researchers sometimes turn to human-avatar interaction in the hope of faster and cheaper data collection that can be transferred to the robot domain. In terms of our work, we are particularly interested in the challenge of detecting and modelling user confusion in interaction, and as part of this research programme, we conducted situated dialogue studies to investigate users' reactions in confusing scenarios that we give in both physical and virtual environments. In this paper, we present a combined review of these studies and the results that we …

Sharing Stress With A Robot: What Would A Robot Say?, Honson Y. Ling, Elin A. Björling

Human-Machine Communication

With the prevalence of mental health problems today, designing human-robot interaction for mental health intervention is not only possible, but critical. The current experiment examined how three types of robot disclosure (emotional, technical, and by-proxy) affect robot perception and human disclosure behavior during a stress-sharing activity. Emotional robot disclosure resulted in the lowest robot perceived safety. Post-hoc analysis revealed that increased perceived stress predicted reduced human disclosure, user satisfaction, robot likability, and future robot use. Negative attitudes toward robots also predicted reduced intention for future robot use. This work informs on the possible design of robot disclosure, as well as …

Building A Stronger Casa: Extending The Computers Are Social Actors Paradigm, Andrew Gambino, Jesse Fox, Rabindra A. Ratan

Human-Machine Communication

The computers are social actors framework (CASA), derived from the media equation, explains how people communicate with media and machines demonstrating social potential. Many studies have challenged CASA, yet it has not been revised. We argue that CASA needs to be expanded because people have changed, technologies have changed, and the way people interact with technologies has changed. We discuss the implications of these changes and propose an extension of CASA. Whereas CASA suggests humans mindlessly apply human-human social scripts to interactions with media agents, we argue that humans may develop and apply human-media social scripts to these interactions. Our …

Xylo-Bot: A Therapeutic Robot-Based Music Platform For Children With Autism, Huanghao Feng

Electronic Theses and Dissertations

Children with Autism Spectrum Disorder (ASD) experience deficits in verbal and nonverbal communication skills, including motor control, emotional facial expressions, and eye gaze / joint attention. This Ph.D. dissertation focuses on studying the feasibility and effectiveness of using a social robot, called NAO, and a toy music instrument, xylophone, at modeling and improving the social responses and behaviors of children with ASD. In our investigation, we designed an autonomous social interactive music teaching system to fulfill this mission.

A novel modular robot-music teaching system consisting of three modules is presented. Module 1 provides an autonomous self-awareness positioning system for the …

Effects Of Control Device And Task Complexity On Performance And Task Shedding During A Robotic Arm Task, Shelby K. Long

Psychology Theses & Dissertations

The use of robotic arms across domains is increasing, but the relationship between control features and performance is not fully understood. The goal of this research was to investigate the difference in task performance when using two different control devices at high and low task complexities when participants can shed tasks to automation. In this experiment, 40 undergraduates (24 females) used two control devices, a Leap Motion controller and an Xbox controller, to teleoperate a robotic arm in a high or low complexity peg placement task. Simultaneously, participants were tasked with scanning images for tanks. During the experiment, participants had …

Using Eye And Head Movements As A Control Mechanism For Tele-Operating A Ground-Based Robot And Its Payload, Kathryn C. Hicks

Computational Modeling & Simulation Engineering Theses & Dissertations

To date, eye and head tracking has been used to indicate users' attention patterns while performing a task or as an aid for disabled persons, to allow hands-free interaction with a computer. The increasing accuracy and the reduced cost of eye- and head-tracking equipment make utilizing this technology feasible for explicit control tasks, especially in cases where there is confluence between the visual task and control.

The goal of this research was to investigate the use of eye-tracking as a more natural interface for the control of a camera-equipped, remotely operated robot in tasks that require the operator to simultaneously …

Toward A Framework For Levels Of Robot Autonomy In Human-Robot Interaction, Jenay Beer, A. D. Fisk, W. A. Rogers

Faculty Publications

Autonomy is a critical construct related to human-robot interaction (HRI) and varies widely across robot platforms. Levels of robot autonomy (LORA), ranging from teleoperation to fully autonomous systems, influence the way in which humans and robots interact with one another. Thus, there is a need to understand HRI by identifying variables that influence—and are influenced by—robot autonomy. Our overarching goal is to develop a framework for LORA in HRI. To reach this goal, our framework draws links between HRI and human-automation interaction, a field with a long history of studying and understanding human-related variables. The construct of autonomy is reviewed …

A Midsummer Night’S Dream (With Flying Robots), Robin Murphy, Dylan Shell, Amy Guerin, Brittany Duncan, Benjamin Fine, Kevin Pratt, Takis Zourntos

School of Computing: Faculty Publications

Seven flying robot “fairies” joined human actors in the Texas A&M production of William Shakespeare’s A Midsummer Night’s Dream. The production was a collaboration between the departments of Computer Science and Engineering, Electrical and Computer Engineering, and Theater Arts. The collaboration was motivated by two assertions. First, that the performing arts have principles for creating believable agents that will transfer to robots. Second, the theater is a natural testbed for evaluating the response of untrained human groups (both actors and the audience) to robots interacting with humans in shared spaces, i.e., were believable agents created? The production used two types …