Graphics and Human Computer Interfaces Commons^™

‘I Know It When I See It’– Developing Quality Schedules Considering Subjective Or Unspecified Criteria, Douglas L. Moody

Publications and Research

Most timetabling problems have a given objective function to measure the quality of a solution. However, users may have a “I know it when I see it” recognition of a quality schedule, without specifying the complete basis for their judgment. In this situation, the objective function cannot be exclusively used as a solution quality measurement. This work presents an AI based approach to aid in categorizing the solution’s quality when the users have not explicitly defined all factors used in their criteria.

Demonstrating Canvas-Based Processing Of Multiple Camera Streams At The Edge, Ila Gokarn, Hemanth Sabbella, Yigong Hu, Tarek Abdelzaher, Archan Misra

Research Collection School Of Computing and Information Systems

We demonstrate criticality-aware canvas-based processing of multiple concurrent camera streams at the resource constrained edge to show substantial improvement in the accuracy-throughput trade-off. The proposed system focuses the available computation resources on select Regions of Interest (RoI) across all the camera streams by (i) extracting RoI from the input camera stream (ii) 2D bin packing the RoI on a canvas frame and (iii) batching and inferring upon these constructed composite canvas frames with a YOLOv5 object detection model. Our experiments show that such canvas-based processing can (i) sustain real-time processing throughput of 23 FPS per camera across 6 concurrent input …

Glance To Count: Learning To Rank With Anchors For Weakly-Supervised Crowd Counting, Zheng Xiong, Liangyu Chai, Wenxi Liu, Yongtuo Liu, Sucheng Ren, Shengfeng He

Research Collection School Of Computing and Information Systems

Crowd image is arguably one of the most laborious data to annotate. In this paper, we devote to reduce the massive demand of densely labeled crowd data, and propose a novel weakly-supervised setting, in which we leverage the binary ranking of two images with highcontrast crowd counts as training guidance. To enable training under this new setting, we convert the crowd count regression problem to a ranking potential prediction problem. In particular, we tailor a Siamese Ranking Network that predicts the potential scores of two images indicating the ordering of the counts. Hence, the ultimate goal is to assign appropriate …

Virtual Reality & Pilot Training: Existing Technologies, Challenges & Opportunities, Tim Marron, Niall Dungan, Brian Mac Namee, Anna Donnla O'Hagan

Journal of Aviation/Aerospace Education & Research

The introduction of virtual reality (VR) to flying training has recently gained much attention, with numerous VR companies, such as Loft Dynamics and VRpilot, looking to enhance the training process. Such a considerable change to how pilots are trained is a subject that warrants careful consideration. Examining the effect that VR has on learning in other areas gives us an idea of how VR can be suitably applied to flying training. Some of the benefits offered by VR include increased safety, decreased costs, and increased environmental sustainability. Nevertheless, some challenges ahead for developers to consider are negative transfer of learning, …

Bias-Aware Gaze Uniformity Assessment In Group Images, Omkar Kulkarni

Electronic Theses & Dissertations (2024 - present)

Today, more than 5 billion photos are captured every day, with smartphones generating over 94\% of these images. However, despite advancements in technology, achieving aesthetically pleasing group photos remains challenging, especially when it comes to aligning the direction of everyone’s gaze. While current methods focus on facial features, they often fail to ensure consistent gaze direction. The introduction of the iPhone's Live mode, which captures a 1.5-second video snippet along with still images, complicates the selection of the best key photo due to its subjective nature and a lack of publicly available data, especially during the pandemic.

To address these …

Consistent Monte Carlo Methods For Non-Linear Applications In Light Transport, Zackary T. Misso

Dartmouth College Ph.D Dissertations

The study of light transport focuses on describing the propagation of light from emitters to sensors through accurately describing the interactions light can undergo with everything in between. Physically-based rendering is the process of applying the laws of light transport to formulate practical algorithms which simulate the flow of light for the purpose of synthesizing images of virtual environments.

Unfortunately, there are very few interesting scene configurations which can be computed analytically. Instead, modern solutions predominantly rely on Monte Carlo integration to stochastically estimate the transfer of light since the process is both unbiased and consistent. Meaning, it is expected …

Towards Dynamic Context Detection From Voice Commands And Conversations With Smart Assistants In Smart Homes, Jeniya Sultana

Graduate Theses/Dissertations

Voice-enabled interactions have become increasingly popular with the rise of voice assistants. Identifying contexts or meanings from voice commands and conversations with smart assistants can contribute to the autonomous control of smart home devices and appliances. To improve automation, there is a growing need for efficient context detection that eliminates the need to memorize voice commands. To address this need, I followed a two-step approach in my research. In the first step, I developed a unique context recognition model using a transformer, an attention mechanism, and a fully connected neural network. I trained this model on a conversational dataset of …

Scene Understanding And Spatial Analysis Using Scene Graph Enhanced By Hall's Proxemics Zones In Smart Homes, Debaleen Das Spandan

Graduate Theses/Dissertations

Voice-controlled smart assistants have received widespread popularity. It plays a pivotal role in smart homes by providing a natural and convenient interface for interacting with smart devices. However, these assistants are unable to serve persons with physical disabilities and speech impairments. Therefore, non-verbal communication methods, such as eye tracking, gesture recognition, and context awareness can complement and overcome some of these limitations to enhance user experience in smart homes. To address this issue, I am investigating non-verbal communication methods to make smart home technology more accessible and intuitive. In this research, I focus on proxemics, i.e., the study of distance …

Garbage In ≠ Garbage Out: Exploring Gan Resilience To Image Training Set Degradations, Nicholas Crino, Bruce A. Cox, Nathan B. Gaw

Faculty Publications

Generative Adversarial Networks (GANs) have received immense attention in recent years due to their ability to capture complex, high-dimensional data distributions without the need for extensive labeling. Since their conception in 2014, a wide array of GAN variants have been proposed featuring alternative architectures, optimizers, and loss functions with the goal of improving performance and training stability. This manuscript focuses on quantifying the resilience of a GAN architecture to specific modes of image degradation. We conduct systematic experimentation to empirically determine the effects of 10 fundamental image degradation modes, applied to the training image dataset, on the Fréchet inception distance …

Programming By Voice, Sadia Nowrin

Dissertations, Master's Theses and Master's Reports

Programmers typically rely on a keyboard and mouse for input, which poses significant challenges for individuals with motor impairments, limiting their ability
to effectively input programs. Voice-based programming offers a promising alternative,
enabling a more inclusive and accessible programming environment. Insights from interviews with motor-impaired programmers revealed that memorizing unnatural commands in existing voice-based programming systems led to frustration. In this work, we explore how programmers naturally speak a single line of code and present a comprehensive methodology for a voice programming system aimed at making programming more accessible for diverse users. To achieve this, we adopted a two-step pipeline. …

Enhancing Students’ User Experience With A Code Critiquer, Laura E. Albrant

Dissertations, Master's Theses and Master's Reports

This thesis explores the role of human factors in the realm of code critiquers and students’ experiences with them. Across three studies, the work utilized Design Thinking to improve the user experience of WebTA for introductory engineering students learning MATLAB. The first two studies gathered observational and interview data to empathize, define, and ideate a new user interface (UI). Said UI was prototyped and then tested with the third study. Overall, the surveys’ data suggests that most students found the new design to be more appealing, useful, and purposeful; however, there is still plenty of room for improvement. Additionally, analysis …

Tracking People Across Ultra Populated Indoor Spaces By Matching Unreliable Wi-Fi Signals With Disconnected Video Feeds, Quang Hai Truong, Dheryta Jaisinghani, Shubham Jain, Arunesh Sinha, Jeong Gil Ko, Rajesh Krishna Balan

Research Collection School Of Computing and Information Systems

Tracking in dense indoor environments where several thousands of people move around is an extremely challenging problem. In this paper, we present a system — DenseTrack for tracking people in such environments. DenseTrack leverages data from the sensing modalities that are already present in these environments — Wi-Fi (from enterprise network deployments) and Video (from surveillance cameras). We combine Wi-Fi information with video data to overcome the individual errors induced by these modalities. More precisely, the locations derived from video are used to overcome the localization errors inherent in using Wi-Fi signals where precise Wi-Fi MAC IDs are used to …

Dynamic Meta-Path Guided Temporal Heterogeneous Graph Neural Networks, Yugang Ji, Chuan Shi, Yuan Fang

Research Collection School Of Computing and Information Systems

Graph Neural Networks (GNNs) have become the de facto standard for representation learning on topological graphs, which usually derive effective node representations via message passing from neighborhoods. Although GNNs have achieved great success, previous models are mostly confined to static and homogeneous graphs. However, there are multiple dynamic interactions between different-typed nodes in real-world scenarios like academic networks and e-commerce platforms, forming temporal heterogeneous graphs (THGs). Limited work has been done for representation learning on THGs and the challenges are in two aspects. First, there are abundant dynamic semantics between nodes while traditional techniques like meta-paths can only capture static …

Effects Of Mindfulness And Emotion Regulation On Aesthetics: A Theoretical Model From Hedonic Perspective Of Processing Fluency, Geng-Bao Lin, Fiona Fui-Hoon Nah, Choon Ling Sia

Research Collection School Of Computing and Information Systems

Research has shown that processing fluency positively impacts perceived aesthetics, with pleasure mediating the relationship. Considering the important role of pleasure, we propose studying the role of emotion regulation in moderating the mediated relationship from processing fluency to perceived aesthetics. Based on our hypotheses, individuals’ emotion regulation strategies are expected to have moderating effects on the relationship between processing fluency and perceived aesthetics such that cognitive reappraisal positively moderates the relationship from processing fluency to pleasure, and expressive suppression negatively moderates the relationship from pleasure to perceived aesthetics. Trait mindfulness is also expected to influence perceived aesthetics through emotion regulation …

Random Walk Methods For Geometry Representation Agnostic Transport, Dario R. Seyb

Dartmouth College Ph.D Dissertations

In computer graphics, we use geometry representations to model a wide range of virtual scenes—from the fantastical worlds shown in animated movies to intricate mechanical parts.
These representations provide the context for transport problems—light transport is used to produce images of virtual scenes and diffusive transport to simulate distributions of quantities like heat.

There are many types of representations each with their own advantages.
For example, explicit ones make it easy to directly manipulate surfaces, while implicit representations allow for intuitive modeling by non-technical users and straightforward integration into machine learning systems.
Unfortunately, many algorithms that work on these digital …

Trust: The Feature That Vending Machines And Atms Share, But Simplygo Lacks, Sun Sun Lim

Research Collection College of Integrative Studies

The article discussed the intricacies of trust in the SimplyGo debacle and highlighted how the design of physical interfaces like vending machines and ATMs and digital interfaces from apps like Grab, Parking.sg and ShopBack have critical features to instil trust. People need to be reassured that their transactions have proceeded as they should, and thay have not been short-changed.

All In One Place: Ensuring Usable Access To Online Shopping Items For Blind Users, Yash Prakash, Akshay Kolgar Nayak, Mohan Sunkara, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Perusing web data items such as shopping products is a core online user activity. To prevent information overload, the content associated with data items is typically dispersed across multiple webpage sections over multiple web pages. However, such content distribution manifests an unintended side effect of significantly increasing the interaction burden for blind users, since navigating to-and-fro between different sections in different pages is tedious and cumbersome with their screen readers. While existing works have proposed methods for the context of a single webpage, solutions enabling usable access to content distributed across multiple webpages are few and far between. In this …

Enhancing Research Productivity: Seamless Integration Of Personal Devices And Hpc Resources With The Cybershuttle Notebook Gateway, Yasith Jayawardana, Dimuthu Wannipurage, Eroma Abeysinghe, Suresh Marru

Computer Science Faculty Publications

Scientists often utilize personal laptops and workstations for initial research stages and turn to high-performance computing (HPC) supercomputers for compute-intensive tasks. However, seamless transitions between these environments are vital for enhancing productivity and accelerating research progress. Our paper presents the Cybershuttle Notebook Gateway, an open-source framework crafted to streamline this transition, optimize resource utilization, and reduce time-to-science for researchers. Leveraging JupyterLab, the framework extends kernel mechanics for seamless provisioning and connection to remote HPC cluster kernels. We delve into its architecture, which separates user authentication, kernel provisioning, and remote file system access. Additionally, we highlight practical capabilities like analyzing network …

An Analysis Of Precision: Occlusion And Perspective Geometry’S Role In 6d Pose Estimation, Jeffrey Choate, Derek Worth, Scott Nykl, Clark N. Taylor, Brett J. Borghetti, Christine M. Schubert Kabban

Faculty Publications

Achieving precise 6 degrees of freedom (6D) pose estimation of rigid objects from color images is a critical challenge with wide-ranging applications in robotics and close-contact aircraft operations. This study investigates key techniques in the application of YOLOv5 object detection convolutional neural network (CNN) for 6D pose localization of aircraft using only color imagery. Traditional object detection labeling methods suffer from inaccuracies due to perspective geometry and being limited to visible key points. This research demonstrates that with precise labeling, a CNN can predict object features with near-pixel accuracy, effectively learning the distinct appearance of the object due to perspective …