Graphics and Human Computer Interfaces Commons^™

Human-Centered Machine Learning With Interpretable Visual Knowledge Discovery, Lincoln Huber

All Master's Theses

This research advances interpretable machine learning (ML) by introducing hyperblocks (HBs) as a structured, rule-based approach for creating transparent and accurate models using meaningful numeric attributes directly interpretable to end users. Key techniques, including Parallel Hyperblock Creation, Interactive Hyperblock Creation, Level n Hyperblock Creation, and k-Nearest Neighbor Hyperblock, provide a framework that ensures domain experts can meaningfully engage with the model’s decision-making process through lossless visualizations using General Line Coordinates (GLC). Case studies with the Wisconsin Breast Cancer and MNIST datasets demonstrated HBs' effectiveness in handling high-risk and complex classification tasks, offering interpretable accuracy that traditional models struggle to achieve. …

Demonstrating Canvas-Based Processing Of Multiple Camera Streams At The Edge, Ila Gokarn, Hemanth Sabbella, Yigong Hu, Tarek Abdelzaher, Archan Misra

Research Collection School Of Computing and Information Systems

We demonstrate criticality-aware canvas-based processing of multiple concurrent camera streams at the resource constrained edge to show substantial improvement in the accuracy-throughput trade-off. The proposed system focuses the available computation resources on select Regions of Interest (RoI) across all the camera streams by (i) extracting RoI from the input camera stream (ii) 2D bin packing the RoI on a canvas frame and (iii) batching and inferring upon these constructed composite canvas frames with a YOLOv5 object detection model. Our experiments show that such canvas-based processing can (i) sustain real-time processing throughput of 23 FPS per camera across 6 concurrent input …

Effects Of Mindfulness And Emotion Regulation On Aesthetics: A Theoretical Model From Hedonic Perspective Of Processing Fluency, Geng-Bao Lin, Fiona Fui-Hoon Nah, Choon Ling Sia

Research Collection School Of Computing and Information Systems

Research has shown that processing fluency positively impacts perceived aesthetics, with pleasure mediating the relationship. Considering the important role of pleasure, we propose studying the role of emotion regulation in moderating the mediated relationship from processing fluency to perceived aesthetics. Based on our hypotheses, individuals’ emotion regulation strategies are expected to have moderating effects on the relationship between processing fluency and perceived aesthetics such that cognitive reappraisal positively moderates the relationship from processing fluency to pleasure, and expressive suppression negatively moderates the relationship from pleasure to perceived aesthetics. Trait mindfulness is also expected to influence perceived aesthetics through emotion regulation …

An Analysis Of Precision: Occlusion And Perspective Geometry’S Role In 6d Pose Estimation, Jeffrey Choate, Derek Worth, Scott Nykl, Clark N. Taylor, Brett J. Borghetti, Christine M. Schubert Kabban

Faculty Publications

Achieving precise 6 degrees of freedom (6D) pose estimation of rigid objects from color images is a critical challenge with wide-ranging applications in robotics and close-contact aircraft operations. This study investigates key techniques in the application of YOLOv5 object detection convolutional neural network (CNN) for 6D pose localization of aircraft using only color imagery. Traditional object detection labeling methods suffer from inaccuracies due to perspective geometry and being limited to visible key points. This research demonstrates that with precise labeling, a CNN can predict object features with near-pixel accuracy, effectively learning the distinct appearance of the object due to perspective …

Garbage In ≠ Garbage Out: Exploring Gan Resilience To Image Training Set Degradations, Nicholas Crino, Bruce A. Cox, Nathan B. Gaw

Faculty Publications

Generative Adversarial Networks (GANs) have received immense attention in recent years due to their ability to capture complex, high-dimensional data distributions without the need for extensive labeling. Since their conception in 2014, a wide array of GAN variants have been proposed featuring alternative architectures, optimizers, and loss functions with the goal of improving performance and training stability. This manuscript focuses on quantifying the resilience of a GAN architecture to specific modes of image degradation. We conduct systematic experimentation to empirically determine the effects of 10 fundamental image degradation modes, applied to the training image dataset, on the Fréchet inception distance …

Improving Usability Of Data Charts In Multimodal Documents For Low Vision Users, Yash Prakash, Akshay Kolgar Nayak, Shoaib Mohammed Alyaan, Pathan Aseef Khan, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Data chart visualizations and text are often paired in news articles, online blogs, and academic publications to present complex data. While chart visualizations offer graphical summaries of the data, the accompanying text provides essential context and explanation. Associating information from text and charts is straightforward for sighted users but presents significant challenges for individuals with low vision, especially on small-screen devices such as smartphones. The visual nature of charts coupled with the layout of the text inherently makes it difficult for low vision users to mentally associate chart data with text and comprehend the content due to their dependence on …

A-Disetrac Advanced Analytic Dashboard For Distributed Eye Tracking, Yasasi Abeysinghe, Bhanuka Mahanama, Gavindya Jayawardena, Yasith Jayawardena, Mohan Sunkara, Andrew T. Duchowski, Vikas Ashok, Sampath Jayarathna

Computer Science Faculty Publications

Understanding how individuals focus and perform visual searches during collaborative tasks can help improve user engagement. Eye tracking measures provide informative cues for such understanding. This article presents A-DisETrac, an advanced analytic dashboard for distributed eye tracking. It uses off-the-shelf eye trackers to monitor multiple users in parallel, compute both traditional and advanced gaze measures in real-time, and display them on an interactive dashboard. Using two pilot studies, the system was evaluated in terms of user experience and utility, and compared with existing work. Moreover, the system was used to study how advanced gaze measures such as ambient-focal coefficient K …

All In One Place: Ensuring Usable Access To Online Shopping Items For Blind Users, Yash Prakash, Akshay Kolgar Nayak, Mohan Sunkara, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Perusing web data items such as shopping products is a core online user activity. To prevent information overload, the content associated with data items is typically dispersed across multiple webpage sections over multiple web pages. However, such content distribution manifests an unintended side effect of significantly increasing the interaction burden for blind users, since navigating to-and-fro between different sections in different pages is tedious and cumbersome with their screen readers. While existing works have proposed methods for the context of a single webpage, solutions enabling usable access to content distributed across multiple webpages are few and far between. In this …

Enhancing Research Productivity: Seamless Integration Of Personal Devices And Hpc Resources With The Cybershuttle Notebook Gateway, Yasith Jayawardana, Dimuthu Wannipurage, Eroma Abeysinghe, Suresh Marru

Computer Science Faculty Publications

Scientists often utilize personal laptops and workstations for initial research stages and turn to high-performance computing (HPC) supercomputers for compute-intensive tasks. However, seamless transitions between these environments are vital for enhancing productivity and accelerating research progress. Our paper presents the Cybershuttle Notebook Gateway, an open-source framework crafted to streamline this transition, optimize resource utilization, and reduce time-to-science for researchers. Leveraging JupyterLab, the framework extends kernel mechanics for seamless provisioning and connection to remote HPC cluster kernels. We delve into its architecture, which separates user authentication, kernel provisioning, and remote file system access. Additionally, we highlight practical capabilities like analyzing network …

Poster, Performed: Understanding Public Opinions Of Authorship In Generative Artificial Intelligence Models Via Analogy, Wylie Z. Kasai

Dartmouth College Master’s Theses

Over the last decade, generative artificial intelligence models have advanced significantly and provided the public with several tools to create new works of art. However, the true authorship of these works has been debated due to their training on web-scraped data. Serving as an analogy to these larger models, Poster, Performed is an interactive artificial intelligence exhibition project that uses image assets submitted by the public to create poster compositions with custom image processing algorithms. During the course of a four-day exhibition, visitors were asked to identify the exhibition’s primary artist from five options: (1) participants who submitted image assets, …

Bias-Aware Gaze Uniformity Assessment In Group Images, Omkar Kulkarni

Electronic Theses & Dissertations (2024 - present)

Today, more than 5 billion photos are captured every day, with smartphones generating over 94\% of these images. However, despite advancements in technology, achieving aesthetically pleasing group photos remains challenging, especially when it comes to aligning the direction of everyone’s gaze. While current methods focus on facial features, they often fail to ensure consistent gaze direction. The introduction of the iPhone's Live mode, which captures a 1.5-second video snippet along with still images, complicates the selection of the best key photo due to its subjective nature and a lack of publicly available data, especially during the pandemic.

To address these …

Trust: The Feature That Vending Machines And Atms Share, But Simplygo Lacks, Sun Sun Lim

Research Collection College of Integrative Studies

The article discussed the intricacies of trust in the SimplyGo debacle and highlighted how the design of physical interfaces like vending machines and ATMs and digital interfaces from apps like Grab, Parking.sg and ShopBack have critical features to instil trust. People need to be reassured that their transactions have proceeded as they should, and thay have not been short-changed.

Visual Analysis Of Enterprise Network Public Opinion Events And Research On Crisis Public Relations Strategy：Taking The “Haitian Soy Sauce Incident”As An Example, Danlin Xie, Xisheng Hu, Weishu Yang

Journal of Scientific Information Research

[Purpose/significance]In the information age with decentralized discourse power, the channels for the public to publicly express their opinions and participate in topic discussions are increasing. The exchange and dissemination of online public opinion on low-ignition events will undoubtedly increase the heat of the event and bring greater pressure and challenges to corporate crisis public relations. [Method/process]Taking "Haitian soy sauce event" as an example, this paper discusses the stage of network public opinion dissemination of the event, uses ROST CM to carry out high-frequency word statistics and public sentiment tendency analysis, and uses Ucinet and Gephi to analyze the social network …

Glance To Count: Learning To Rank With Anchors For Weakly-Supervised Crowd Counting, Zheng Xiong, Liangyu Chai, Wenxi Liu, Yongtuo Liu, Sucheng Ren, Shengfeng He

Research Collection School Of Computing and Information Systems

Crowd image is arguably one of the most laborious data to annotate. In this paper, we devote to reduce the massive demand of densely labeled crowd data, and propose a novel weakly-supervised setting, in which we leverage the binary ranking of two images with highcontrast crowd counts as training guidance. To enable training under this new setting, we convert the crowd count regression problem to a ranking potential prediction problem. In particular, we tailor a Siamese Ranking Network that predicts the potential scores of two images indicating the ordering of the counts. Hence, the ultimate goal is to assign appropriate …

Dynamic Meta-Path Guided Temporal Heterogeneous Graph Neural Networks, Yugang Ji, Chuan Shi, Yuan Fang

Research Collection School Of Computing and Information Systems

Graph Neural Networks (GNNs) have become the de facto standard for representation learning on topological graphs, which usually derive effective node representations via message passing from neighborhoods. Although GNNs have achieved great success, previous models are mostly confined to static and homogeneous graphs. However, there are multiple dynamic interactions between different-typed nodes in real-world scenarios like academic networks and e-commerce platforms, forming temporal heterogeneous graphs (THGs). Limited work has been done for representation learning on THGs and the challenges are in two aspects. First, there are abundant dynamic semantics between nodes while traditional techniques like meta-paths can only capture static …

Programming By Voice, Sadia Nowrin

Dissertations, Master's Theses and Master's Reports

Programmers typically rely on a keyboard and mouse for input, which poses significant challenges for individuals with motor impairments, limiting their ability
to effectively input programs. Voice-based programming offers a promising alternative,
enabling a more inclusive and accessible programming environment. Insights from interviews with motor-impaired programmers revealed that memorizing unnatural commands in existing voice-based programming systems led to frustration. In this work, we explore how programmers naturally speak a single line of code and present a comprehensive methodology for a voice programming system aimed at making programming more accessible for diverse users. To achieve this, we adopted a two-step pipeline. …

The Impact Of Eye-Tracking On Mixed Reality Typing, Cecilia Schmitz

Dissertations, Master's Theses and Master's Reports

Accuracy and speed are pivotal when typing. We hypothesized that the lack of tactile feedback on midair mixed reality keyboards may adversely impact typing performance, especially when ten fingers are used to type. We examined the differences in performance when participants typed on a virtual keyboard using just their index fingers versus all ten fingers. The keyboard was deterministic (without auto-correct), relied only on the headset's egocentric cameras for sensing, and included symbol keys. We used a novel eye-tracking technique to mitigate accidental key presses. The technique was successful at reducing error rates, though participants still typed faster using their …

‘I Know It When I See It’– Developing Quality Schedules Considering Subjective Or Unspecified Criteria, Douglas L. Moody

Publications and Research

Most timetabling problems have a given objective function to measure the quality of a solution. However, users may have a “I know it when I see it” recognition of a quality schedule, without specifying the complete basis for their judgment. In this situation, the objective function cannot be exclusively used as a solution quality measurement. This work presents an AI based approach to aid in categorizing the solution’s quality when the users have not explicitly defined all factors used in their criteria.