Graphics and Human Computer Interfaces Commons^™

Wisdom Of Nature: Finding Tao In Water, Ziyu Xu

Senior Projects Fall 2022

Senior Project submitted to The Division of Arts of Bard College.

A New Way To Make Music: Processing Digital Audio In Virtual Reality, Gavin E. Payne

Senior Projects Spring 2022

The work of this project attempts to provide new methods of creating music with technology. The product, Fields, is a functional piece of virtual reality software, providing users an immersive and interactive set of tools used to build and design instruments in a modular manner. Each virtual tool is analogous to musical hardware such as guitar pedals, synthesizers, or samplers, and can be thought of as an effect or instrument on its own. Specific configurations of these virtual audio effects can then be played to produce music, and then even saved by the user to load up and play with …

The Effect Of Touch Simulation In Virtual Reality Shopping, Ha Kyung Lee, Namhee Yoon, Dooyoung Choi

STEMPS Faculty Publications

This study aims to explore the effect of touch simulation on virtual reality (VR) store satisfaction mediated by VR shopping self-efficacy and VR shopping pleasure. The moderation effects of the autotelic and instrumental need for touch between touch simulation and VR store satisfaction are also explored. Participants wear a head-mounted display VR device (Oculus Go) in a controlled laboratory environment, and their VR store experience is recorded as data. All participants’ responses (n = 58) are analyzed using SPSS 20.0 for descriptive statistics, reliability analysis, exploratory factor analysis, and the Process macro model analysis. The results show that touch simulation …

"What's In A Name?”: The Use Of Instructional Design In Overcoming Terminology Barriers Associated With Dark Patterns, Andrea Curley, Damian Gordon, Dympna O'Sullivan

Conference Papers

Many users experience a phenomena when they are shopping on-line where they feel they are being pressured to either spend more money than they had intended, or to share more personal data than they wanted. In academic circles we use the term “Dark Patterns” to describe these deceptive practices, and categorize them as being within the discipline of User Experience (Narayanan, 2020). As academics it is important to name phenomena, and to categorize them, so that we can discuss and analyze these issues. However, this particular topic is one that all users should be made aware of when interacting online, …

Eye Movement And Pupil Measures: A Review, Bhanuka Mahanama, Yasith Jayawardana, Sundararaman Rengarajan, Gavindya Jayawardena, Leanne Chukoskie, Joseph Snider, Sampath Jayarathna

Computer Science Faculty Publications

Our subjective visual experiences involve complex interaction between our eyes, our brain, and the surrounding world. It gives us the sense of sight, color, stereopsis, distance, pattern recognition, motor coordination, and more. The increasing ubiquity of gaze-aware technology brings with it the ability to track gaze and pupil measures with varying degrees of fidelity. With this in mind, a review that considers the various gaze measures becomes increasingly relevant, especially considering our ability to make sense of these signals given different spatio-temporal sampling capacities. In this paper, we selectively review prior work on eye movements and pupil measures. We first …

Multi-User Eye-Tracking, Bhanuka Mahanama

Computer Science Faculty Publications

The human gaze characteristics provide informative cues on human behavior during various activities. Using traditional eye trackers, assessing gaze characteristics in the wild requires a dedicated device per participant and therefore is not feasible for large-scale experiments. In this study, we propose a commodity hardware-based multi-user eye-tracking system. We leverage the recent advancements in Deep Neural Networks and large-scale datasets for implementing our system. Our preliminary studies provide promising results for multi-user eye-tracking on commodity hardware, providing a cost-effective solution for large-scale studies.

Toward A Real-Time Index Of Pupillary Activity As An Indicator Of Cognitive Load, Gavindya Jayawardena, Yasith Jayawardana, Sampath Jayarathna, Jonas Högström, Thomas Papa, Deepak Akkil, Andrew T. Duchowski, Vsevolod Peysakhovich, Izabela Krejtz, Nina Gehrer, Krzysztof Krejtz

Computer Science Faculty Publications

The Low/High Index of Pupillary Activity (LHIPA), an eye-tracked measure of pupil diameter oscillation, is redesigned and implemented to function in real-time. The novel Real-time IPA (RIPA) is shown to discriminate cognitive load in re-streamed data from earlier experiments. Rationale for the RIPA is tied to the functioning of the human autonomic nervous system yielding a hybrid measure based on the ratio of Low/High frequencies of pupil oscillation. The paper's contribution is drawn from provision of documentation of the calculation of the RIPA. As with the LHIPA, it is possible for researchers to apply this metric to their own experiments …

Using Ai-Enabled Gaze Tracking System To Understand Comprehension Patterns Of Computer Science Students, Bradley Boswell

Electronic Theses and Dissertations

Academic institutions and instructors lack the ability to accurately assess the moment-to-moment attentiveness of students in classrooms where students’ faces are obscured by computer monitors. This can cause the lectures of Computer Science, Information Technology, or other lab-based courses to be incorrectly-paced, which can lead to students having an overall poorer grasp of the subject material. We propose a system for real-time accurate detection of classroom attentiveness using monitor-mounted webcams and eye trackers, along with a Convolutional Neural Network machine learning model (NiCATS). Through the use of a neural network, we produce an initial attentiveness score based on student webcam …

Interpretable Machine Learning For Self-Service High-Risk Decision Making, Charles Recaido

All Master's Theses

This research contributes to interpretable machine learning via visual knowledge discovery in General Line Coordinates (GLC). The concepts of hyperblocks as interpretable dataset units and GLC are combined to create a visual self-service machine learning model. Two variants of GLC known as Dynamic Scaffold Coordinates (DSC) are proposed. DSC1 and DSC2 can map in a lossless manner multiple dataset attributes to a single two-dimensional (X, Y) Cartesian plane using a dynamic scaffolding graph construction algorithm.

Hyperblock analysis is used to determine visually appealing dataset attribute orders and to reduce line occlusion. It is shown that hyperblocks can generalize decision tree …

Balancing User Experience For Mobile One-To-One Interpersonal Telepresence, Kevin Pfeil

Electronic Theses and Dissertations, 2020-

The COVID-19 virus disrupted all aspects of our daily lives, and though the world is finally returning to normalcy, the pandemic has shown us how ill-prepared we are to support social interactions when expected to remain socially distant. Family members missed major life events of their loved ones; face-to-face interactions were replaced with video chat; and the technologies used to facilitate interim social interactions caused an increase in depression, stress, and burn-out. It is clear that we need better solutions to address these issues, and one avenue showing promise is that of Interpersonal Telepresence. Interpersonal Telepresence is an interaction paradigm …

Evaluating Technology-Mediated Collaborative Workflows For Telehealth, Christopher Bondy Ph.D., Pengcheng Shi, Pamela Grover Md, Vicki Hanson, Linlin Chen, Rui Li

Articles

Goals: This paper discusses the need for a predictable method to evaluate gains and gaps of collaborative technology-mediated workflows and introduces an evaluation framework to address this need. Methods: The Collaborative Space Analysis Framework (CS-AF), introduced in this research, is a cross-disciplinary evaluation method designed to evaluate technology-mediated collaborative workflows. The 5-step CS-AF approach includes: (1) current-state workflow definition, (2) current-state (baseline) workflow assessment, (3) technology-mediated workflow development and deployment, (4) technology-mediated workflow assessment, (5) analysis, and conclusions. For this research, a comprehensive, empirical study of hypertension exam workflow for telehealth was conducted using the CS-AF approach. Results: The CS-AF …

What Interactive Web Features Are Most Used, Travis Tyler

Experiential Learning Projects

The use of interactive features in websites has become common place on the internet. People use these tools to help navigate and understand the content related to that website. However, due the large variety of websites it can be tricky to understand what features are best to utilize based on the topic of your site. This paper seeks to address this issue by researching how users interact and utilized different features on different websites. Research is gathered via scholarly articles and direct data gathered from volunteers. This data shows users tend to favor more interactive tools to help with navigation, …

Evaluation Of Gpu Acceleration For Wrf–Sfire, Joshua Benz

Master's Projects

WRF–SFIRE is an open source, atmospheric–wildfire model that couples the WRF model with the level set fire spread model to simulate wildfires in real time. This model has many applications and more scientific questions can be asked and answered if the model can be run faster. Nvidia has put a lot of effort into easing the barrier of entry for accelerating applications with their tools to be run on GPUs. Various physical simulations have been successfully ported to utilize GPUs and have benefited from the speed increase. In this research, we take a look at WRF-SFIRE and try to use …

Winter 2021

In The Loop

2021 Emmy Nominees; Animator Tapped by Cartoon Network; IndieCade Horizons 2021; Hack4Space; Security Daemons Prevail; Role Models: DePaul Originals Game Studio students build industry-level skills that benefit themselves and others; Frames and Fortune: Eugene Bush programmed his indie video studio with patience and planning; Reality Check: Heather Snyder Quinn augments reality to question systems of unchecked power

Comparative Analysis Of Rgb-Based Eye-Tracking For Large-Scale Human-Machine Applications, Brett Thaman, Trung Cao

Posters-at-the-Capitol

Gaze tracking has become an established technology that enables using an individual’s gaze as an input signal to support a variety of applications in the context of Human-Computer Interaction. Gaze tracking primarily relies on sensing devices such as infrared (IR) cameras. Nevertheless, in the recent years, several attempts have been realized at detecting gaze by acquiring and processing images acquired from standard RGB cameras. Nowadays, there are only a few publicly available open-source libraries and they have not been tested extensively. In this paper, we present the result of a comparative analysis that studied a commercial eye-tracking device using IR …

Markdown To Question & Test Interoperability, Su Kim

Master's Projects

As the classroom setting shifted to a virtual one as a result of Covid-19, numerous software are readily available to accommodate for the change, including Canvas, the online course management system. Canvas has a core feature that allows teachers to generate and administer quizzes for students through their interface, but it does not fully utilize the potential with online exams. The first step to exploring this potential is this project, known as Markdown to Question & Test Interoperability (M2QTI). Based on the QTI specifications, this tool lets users to plan and write quizzes in Markdown format. Combined with Canvas’s ability …

Component Damage Source Identification For Critical Infrastructure Systems, Nathan Davis

Graduate Theses and Dissertations

Cyber-Physical Systems (CPS) are becoming increasingly prevalent for both Critical Infrastructure and the Industry 4.0 initiative. Bad values within components of the software portion of CPS, or the computer systems, have the potential to cause major damage if left unchecked, and so detection and locating of where these occur is vital. We further define features of these computer systems and create a use-based system topology. We then introduce a function to monitor system integrity and the presence of bad values as well as an algorithm to locate them. We then show an improved version, taking advantage of several system properties …

Contrastive Learning For Unsupervised Auditory Texture Models, Christina Trexler

Computer Science and Computer Engineering Undergraduate Honors Theses

Sounds with a high level of stationarity, also known as sound textures, have perceptually relevant features which can be captured by stimulus-computable models. This makes texture-like sounds, such as those made by rain, wind, and fire, an appealing test case for understanding the underlying mechanisms of auditory recognition. Previous auditory texture models typically measured statistics from auditory filter bank representations, and the statistics they used were somewhat ad-hoc, hand-engineered through a process of trial and error. Here, we investigate whether a better auditory texture representation can be obtained via contrastive learning, taking advantage of the stationarity of auditory textures to …

Self-Supervised Learning Disentangled Group Representation As Feature, Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, Hanwang Zhang

Research Collection School Of Computing and Information Systems

A good visual representation is an inference map from observations (images) to features (vectors) that faithfully reflects the hidden modularized generative factors (semantics). In this paper, we formulate the notion of “good” representation from a group-theoretic view using Higgins’ definition of disentangled representation [38], and show that existing Self-Supervised Learning (SSL) only disentangles simple augmentation features such as rotation and colorization, thus unable to modularize the remaining semantics. To break the limitation, we propose an iterative SSL algorithm: Iterative Partition-based Invariant Risk Minimization (IP-IRM), which successfully grounds the abstract semantics and the group acting on them into concrete contrastive learning. …

Vireo @ Trecvid 2021 Ad-Hoc Video Search, Jiaxin Wu, Phuong Anh Nguyen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

In this paper, we summarize our submitted runs and results for Ad-hoc Video Search (AVS) task at TRECVid 2020