Graphics and Human Computer Interfaces Commons^™

Beyond The Interface: Human Perceptions Of Generative-Ai Chatbots As Conversational Partners, Browning W.E. Blair

All Theses

Generative AI (gen-AI) chatbots are becoming embedded in everyday communicative life, yet it remains unclear whether users perceive these systems as socially reciprocative conversational partners. Therefore, this study examines how young adults understand and interact with gen-AI chatbots, focusing on perceptions of conversational partnership, anthropomorphism, politeness, discomfort, and technical understanding. Guided by the CASA framework, Media Equation Theory, and the uncanny valley hypothesis, this study employed four semi-structured, online focus groups with 15 undergraduate students and recent college graduates in the United States. Findings indicate that participants did not broadly perceive gen-AI chatbots as conversational partners in the interpersonal sense. …

A Macrocognitive Design Taxonomy For Simulation-Based Training Systems: Bridging Cognitive Theory And Human-Computer Interaction, Jessica M. Johnson

Virginia Digital Maritime Center (VDMC) Faculty Publications

Simulation-based training systems are increasingly deployed to prepare learners for complex, safety-critical, and dynamic work environments. While advances in computing have enabled immersive and data-rich simulations, many systems remain optimized for procedural accuracy and surface-level task performance rather than the macrocognitive processes that underpin adaptive expertise. Macrocognition encompasses higher-order cognitive processes that are essential for performance transfer beyond controlled training conditions. When these processes are insufficiently supported, training systems risk fostering brittle strategies and negative training effects. This paper introduces a macrocognitive design taxonomy for simulation-based training systems derived from a large-scale meta-analysis examining the transfer of macrocognitive skills from …

Memebuddy: Dialog-Style Audio Representations For Engaging Non-Visual Meme Experiences, Chirag Bhansali, Vikas Ashok, Hae-Na Lee

Computer Science Faculty Publications

Image memes are a pervasive form of online communication, widely used to convey humor, opinions, and cultural references. Prior work has explored making memes accessible to blind users, primarily through auto-generated descriptive captions. While these approaches improve comprehensibility and sometimes incorporate prosodic or emotional cues, they often fail to capture the humor, narrative structure, and contextual nuances that make memes engaging. We present MemeBuddy, a system that models memes as dialog, generating structured, multi-turn audio representations using role-based speakers. MemeBuddy reinterprets a meme as a conversation between two speakers, integrating extracted meme text with contextual knowledge implicitly inferred by a …

Gaze Transition Entropy And Automation Trust In Multitasking Workspace, Yusuke Yamani, Austin Jackson, Tetsuya Sato, Feyishola Ashimi, Michael S. Politowicz, Eric T. Chancey, Makoto Itoh

Psychology Faculty Publications

Safe flight operation requires visual scanning across multiple displays in a cockpit, which collectively represent the state of the aircraft and supporting automation. Trust is a crucial factor that drives human-automation interaction, and recent work has suggested a relationship between an operator's visual attention and automation trust. One index that captures predictability of eye movements between different areas of interest is gaze transition entropy. The current work reanalyzed data from Sato et al., which examined eye movement patterns and trust in automation associated with the system monitoring task of the Multi-Attribute Task Battery. Results showed credible positive correlations between the …

A Real-Time Approach To Capture Ambient And Focal Attention In Visual Search, Gavindya Jayawardena, Yasith Jayawardana, Yasasi Abeysinghe, Bhanuka Mahanama, Sampath Jayarathna, Jacek Gwizdka

Computer Science Faculty Publications

During visual search, individuals’ attention shifts between ambient and focal states in response to task demands and stimuli. The ambient/focal coefficient K is a statistically validated measure of these states, computed offline from fixation duration and saccade amplitude data. While current methods compute K offline, real-time computation could enable applications such as monitoring user attention, creating attention-adaptive user interfaces, and optimizing graphics rendering. However, real-time computation of K requires stable estimates for the parameters of fixation duration and saccade amplitude distributions. Since these distributions are heavy-tailed, the real-time estimates exhibit high variance and slow convergence. To overcome this, we propose …

Nexus: Network Exploration For Exploiting Unsafe Sequences In Multi-Turn Llm Jailbreaks, Javad Rafiei Asl, Sidhant Narula, Mohammad Ghasemigol, Eduardo Blanco, Daniel Takabi

School of Cybersecurity Faculty Publications

Large Language Models (LLMs) have revolutionized natural language processing, yet remain vulnerable to jailbreak attacks—particularly multi-turn jailbreaks that distribute malicious intent across benign exchanges, thereby bypassing alignment mechanisms. Existing approaches often suffer from limited exploration of the adversarial space, rely on hand-crafted heuristics, or lack systematic query refinement. We propose NEXUS (Network Exploration for eXploiting Unsafe Sequences), a modular framework for constructing, refining, and executing optimized multi-turn attacks. NEXUS comprises: (1) ThoughtNet, which hierarchically expands a harmful intent into a structured semantic network of topics, entities, and query chains; (2) a feedback-driven Simulator that iteratively refines and prunes these chains …

The Interacting Roles Of Attention Allocation And Trust In Highly Automated Aam Environments, Yusuke Yamani

Psychology Faculty Publications

[First slide]

Mechanisms of attentive visual processing

• Attention control
• Visual search
• Eye movement
• Aging and individual differences

Limits of human performance in applied environment

• Complex displays
• Machine operation
• Surface transportation
• Advanced air mobility
• Nuclear operation

Methods to ameliorate human cognitive performance

• Human-machine interface
• Human autonomy/AI teaming
• Human-systems integration
• Training

Research Project Review: Human-Computer Interactions, Kylie E. Garcia

The Agora

A review of a body of research conducted by Dr. Gain Park, an assistant professor in the Department of Journalism and Media Studies at New Mexico State University. This review contains a summary of Dr. Park's research on human-computer interactions, commentary on its contributions and significance, as well as insights from Dr. Park.

Differences In Student-Ai Interaction Process On A Drawing Task: Focusing On Students' Attitude Towards Ai And The Level Of Drawing Skills, Jinhee Kim, Yoonhee Ham, Sang-Soog Lee

STEMPS Faculty Publications

Recent advances and applications of artificial intelligence (AI) have increased the opportunities for students to interact with AI in their learning tasks. Although various fields of scholarly research have investigated human-AI collaboration, the underlying processes of how students collaborate with AI in a student-AI teaming scenario have been scarcely investigated. To develop effective AI applications in education, it is necessary to understand differences in the student-AI interaction (SAI) process depending on students' characteristics. The present study attempts to fill this gap by exploring the differences in the SAI process amongst students with varying drawing proficiencies and attitudes towards AI in …

Scene Understanding And Spatial Analysis Using Scene Graph Enhanced By Hall's Proxemics Zones In Smart Homes, Debaleen Das Spandan

Graduate Theses/Dissertations

Voice-controlled smart assistants have received widespread popularity. It plays a pivotal role in smart homes by providing a natural and convenient interface for interacting with smart devices. However, these assistants are unable to serve persons with physical disabilities and speech impairments. Therefore, non-verbal communication methods, such as eye tracking, gesture recognition, and context awareness can complement and overcome some of these limitations to enhance user experience in smart homes. To address this issue, I am investigating non-verbal communication methods to make smart home technology more accessible and intuitive. In this research, I focus on proxemics, i.e., the study of distance …

Leveraging Artificial Intelligence For Team Cognition In Human-Ai Teams, Beau Schelble

All Dissertations

Advances in artificial intelligence (AI) technologies have enabled AI to be applied across a wide variety of new fields like cryptography, art, and data analysis. Several of these fields are social in nature, including decision-making and teaming, which introduces a new set of challenges for AI research. While each of these fields has its unique challenges, the area of human-AI teaming is beset with many that center around the expectations and abilities of AI teammates. One such challenge is understanding team cognition in these human-AI teams and AI teammates' ability to contribute towards, support, and encourage it. Team cognition is …

Anatoview: Using Interactive 3d Visualizations With Augmented Reality Support For Laypersons’ Medical Education In Informed Consent Processes, Michelle Chen

Dartmouth College Master’s Theses

AnatoView is an interactive multimedia educational application that visualizes medical procedures in three-dimensional (3D), augmented reality (AR) space. By providing visual and spatial information of medical procedures, AnatoView acts as a learning supplement for laypersons/patients in informed consent (IC) processes — wherein instructional content is traditionally limited to purely spoken explanations that lead to poor patient comprehension. We design a mixed study and conduct a randomized, controlled trial with 15 laypersons as participants: administering a traditional IC process to a control group, and an IC process supplemented by the use of AnatoView to experimental groups. As a primary outcome, medical …

Immersive Learning Environments For Computer Science Education, Dillon Buchanan

Electronic Theses and Dissertations

This master's thesis explores the effectiveness of an educational intervention using an interactive notebook to support and supplement instruction in a foundational-level programming course. A quantitative, quasi-experimental group comparison method was employed, where students were placed into either a control or a treatment group. Data was collected from assignment and final grades, as well as self-reported time spent using the notebook. Independent t-tests and correlation were used for data analysis. Results were inconclusive but did indicate that the intervention had a possible effect. Further studies may explore better efficacy, implementation, and satisfaction of interactive notebooks across a larger population and …

An Ambiguous Technique For Nonvisual Text Entry, Dylan C. Gaines

Dissertations, Master's Theses and Master's Reports

Text entry is a common daily task for many people, but it can be a challenge for people with visual impairments when using virtual touchscreen keyboards that lack physical key boundaries. In this thesis, we investigate using a small number of gestures to select from groups of characters to remove most or all dependence on touch locations. We leverage a predictive language model to select the most likely characters from the selected groups once a user completes each word.

Using a preliminary interface with six groups of characters based on a Qwerty keyboard, we find that users are able to …

Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, Koki Kusano

Research Collection School Of Computing and Information Systems

Inaccessible e-commerce websites and mobile applications exclude people with visual impairments (PVI) from online shopping. Customer-to-customer (C2C) marketplaces, a form of e-commerce where trading happens not between businesses and customers but between customers, could pose a unique set of challenges in the interactions that the platform brings about. Through online questionnaire and remote interviews, we investigate problems experienced by people with low vision disabilities in common C2C scenarios. Our study with low vision participants (N = 12) reveal both previously known general accessibility issues (e.g., web and mobile interface accessibility) and C2C specific accessibility issues (e.g., inability to confirm item …

The Illusion Of Agency In Human–Computer Interaction, Michael Madary

College of the Pacific Faculty Articles

This article makes the case that our digital devices create illusions of agency. There are times when users feel as if they are in control when in fact they are merely responding to stimuli on the screen in predictable ways. After the introduction, the second section of the article offers examples of illusions of agency that do not involve human–computer interaction in order to show that such illusions are possible and not terribly uncommon. The third and fourth sections of the article cover relevant work from empirical psychology, including the cues that are known to generate the sense of agency. …

Towards Location-Independent Eyes-Free Text Entry, Dylan C. Gaines

Dissertations, Master's Theses and Master's Reports

We propose an interface for eyes-free text entry using an ambiguous technique and conduct a preliminary user study. We find that user are able to enter text at 19.09 words per minute (WPM) with a 2.08% character error rate (CER) after eight hours of practice. We explore ways to optimize the ambiguous groupings to reduce the number of disambiguation errors, both with and without familiarity constraints. We find that it is feasible to reduce the number of ambiguous groups from six to four. Finally, we explore a technique for presenting word suggestions to users using simultaneous audio feedback. We find …

Visualization Research Lab At Hkust, Yong Wang

Research Collection School Of Computing and Information Systems

HKUST VisLab (http://vis.cse.ust.hk/) is one of the leading research labs in the field of data visualization and human-computer interaction worldwide. The lab is dedicated to conducting cutting-edge research on data visualization and human-computer interaction to facilitate data exploration and analytics in various application domains, including E-learning, urban computing, social media and industry 4.0. Starting from its foundation by Prof. Huamin Qu in August 2004, the mission of HKUST VisLab is to build an excellent visualization research center and foster data visualization research and talent cultivation in Asia, as there were very few visualization researchers in Asia around 2004.

The Meaning Of Red And Green In User Interfaces For The Color Deficient, Bassel Hamieh

University Honors Theses

Around 108 million web users are color blind which is a problem when the way we communicate over the web or interfaces is through the use of color. Red and green are two colors that are especially heavily used in interface design because of their strong symbolic associations; red being a sign to warn or stop and green being the opposite. This has a large effect on red-green color blind people who are not able to perceive either of those colors correctly. Many solutions exist that aim to help through color differentiation but none take into account color symbolism. With …

Transfer Learning Approach To Multiclass Classification Of Child Facial Expressions, Megan A. Witherow, Manar D. Samad, Khan M. Iftekharuddin

Electrical & Computer Engineering Faculty Publications

The classification of facial expression has been extensively studied using adult facial images which are not appropriate ground truths for classifying facial expressions in children. The state-of-the-art deep learning approaches have been successful in the classification of facial expressions in adults. A deep learning model may be better able to learn the subtle but important features underlying child facial expressions and improve upon the performance of traditional machine learning and feature extraction methods. However, unlike adult data, only a limited number of ground truth images exist for training and validating models for child facial expression classification and there is a …

Patient Engagement In The Medical Facility Waiting Room Using Gamified Healthcare Information Delivery, Raheel Hassan, Nathan W. Twyman, Fiona F. Nah, Keng Siau

Research Collection School Of Computing and Information Systems

This study explores the proposition that medical facility waiting rooms are an opportune setting to engage with and educate patients while they are waiting for care. In collaboration with emergency department (ED) personnel, we developed ER Hero, a tablet-based application for waiting rooms that introduces patients to ED professionals and operations through mini-games and story-like interaction. We evaluated this prototype with human participants to determine how well it performed when compared to paper-based information disclosure presenting the same information. Participants using the application exhibited increased ED knowledge, decreased nervousness, and increased interest. The gamified application outperformed a paper-based approach on …

Patient Engagement In The Medical Facility Waiting Room Using Gamified Healthcare Information Delivery, Raheel Hassan, Nathan W. Twyman, Fiona Fui-Hoon Nah, Keng Siau

Research Collection School Of Computing and Information Systems

This study explores the proposition that medical facility waiting rooms are an opportune setting to engage with and educate patients while they are waiting for care. In collaboration with emergency department (ED) personnel, we developed ER Hero, a tablet-based application for waiting rooms that introduces patients to ED professionals and operations through mini-games and story-like interaction. We evaluated this prototype with human participants to determine how well it performed when compared to paper-based information disclosure presenting the same information. Participants using the application exhibited increased ED knowledge, decreased nervousness, and increased interest. The gamified application outperformed a paper-based approach on …

Smart Living For Elderly: Design And Human-Computer Interaction Considerations, Ranjana Sharma, Fiona Fui-Hoon Nah, Kavya Sharma, Teja S. Katta, Natalie Pang, Alvin Yong

Research Collection School Of Computing and Information Systems

To address aging challenges, we examine the concept of smart living and its applications for the elderly. Smart living refers to improving quality of life by transforming environments to become more intelligent and adaptable to users. In this paper, we discuss how smart living applications can help to address the needs of the elderly, as well as the design and human-computer interaction considerations for such applications.

Infotainment Interface Design For Automobiles, Ian R. Buck

Scholarly Horizons: University of Minnesota, Morris Undergraduate Journal

In an increasingly connected, mobile world, situations where users do not interact with their digital lives are becoming few and far between. This can be a problem in situations that demand a user's attention for their safety. Driving is one such situation, and it is doubly important because a significant portion of the western population drives on a daily basis. Researchers have tested different interface designs with the goal of finding one that demands the least cognitive load while still allowing the user to perform the desired task efficiently. In this paper interfaces incorporating auditory cues, voice dictation, and air …

A Function-To-Task Process Model For Adaptive Automation System Design, Jason M. Bindewald, Michael E. Miller, Gilbert L. Peterson

Faculty Publications

Adaptive automation systems allow the user to complete a task seamlessly with a computer performing tasks at which the human operator struggles. Unlike traditional systems that allocate functions to either the human or the machine, adaptive automation varies the allocation of functions during system operation. Creating these systems requires designers to consider issues not present during static system development. To assist in adaptive automation system design, this paper presents the concept of inherent tasks and takes advantage of this concept to create the function-to-task design process model. This process model helps the designer determine how to allocate functions to the …

Peering Into The Discourse Of Industrial Design Training Through A Sustainability Lens, Norman M. Su, Haodan Tan, Eli Blevis

Design Thinking Research Symposium

Now well established in HCI, the lens of sustainability may be applied to educational practices in industrial design and interaction design. By sustainability, we mean to include notions of mitigation of the environmental effects of climate change. In this paper, we present an analysis of student projects in a junior and senior industrial design class dataset. Drawing from discourse analysis, we examine how the industrial design classroom serves as a space to socially construct the philosophies and goals inherent in “good” design. We then examine how the lens of sustainability is implicated into the industrial design “way” as espoused by …

Exploration Of Human-Computer Interaction (Hci) Applications In Hospitality Industry, Jia Wei

UNLV Theses, Dissertations, Professional Papers, and Capstones

Purpose: The purpose of this professional paper is to explore and identify the best practices in Human-Computer Interaction (HCI) applications and use within the hospitality industry.

Statement of Objectives: To accomplish the stated purpose, first, the paper will provide the universal definition of HCI applications and define HCI within the context of hospitality industry. Second, the paper will gather and categorize information, such as the standards of designing HCI applications, from other fields of study such as informatics. Third, the paper will gather existing examples of HCI applications across different areas within the hospitality industry, such as the front desk …

A Visualization Approach For Message Passing In Parallel Computing Systems, Arunkumar Sadasivan

UNLV Theses, Dissertations, Professional Papers, and Capstones

Information Visualization has been used as an effective method in transforming textual information into a visual form that enables users to effectively communicate and understand the information. MPI (Message Passing Interface) usually involves a large amount of data, which necessitates exploring innovative ideas to visualize such data effectively.

In this thesis, we implement a graph visualization tool called MPROV to effectively visualize communication patterns of message passing using log files from MPI applications. The tool provides several interaction techniques to effectively reduce the amount of data displayed on the screen. We have also developed protocol conformance checking, which verifies the …

Pysafe: An Interdisciplinary Approach To Interface Design, Matthew Adam Richardson

Theses Digitization Project

The purpose of this project is two fold : examine the prevailing wisdom of the interaction design field over the past forty years and look at emerging perspectives, then apply the established principles along with new trends in a practical application. Includes source code.

Introduction: Human-Computer Interaction Studies In Management Information Systems, Fiona Fui-Hoon Nah, P. Zhang, S. Mccoy

Research Collection School Of Computing and Information Systems

Human-computer interaction (HCI) is a critical area of research and practice that spans several disciplines, including industrial engineering, management information systems (MIS), computer science, information science, psychology, sociology, and anthropology. HCI research within the MIS discipline has distinct features shaped by the evolution and current state of MIS. Originating as applied computer science in the 1970s, MIS gradually transformed into a more social science–oriented discipline (Baskerville & Myers, 2002). MIS is broadly defined as “the effective design, delivery, and use of information systems in organizations” (Keen, 1980, p. 14). The two distinguishing features of MIS compared to other "homes" of …