Graphics and Human Computer Interfaces Commons^™

Rethinking Multi-View Representation Learning Via Distilled Disentangling, Guanzhou Ke, Bo Wang, Xiaoli Wang, Shengfeng He

Research Collection School Of Computing and Information Systems

Multi-view representation learning aims to derive robust representations that are both view-consistent and view-specific from diverse data sources. This paper presents an in-depth analysis of existing approaches in this domain, highlighting a commonly overlooked aspect: the redundancy between view-consistent and view-specific representations. To this end, we propose an innovative framework for multi-view representation learning, which incorporates a technique we term 'distilled disentangling'. Our method introduces the concept of masked cross-view prediction, enabling the extraction of compact, high-quality view-consistent representations from various sources without incurring extra computational overhead. Additionally, we develop a distilled disentangling module that efficiently filters out consistency-related information …

Inceptionnext: When Inception Meets Convnext, Weihao Yu, Pan Zhou, Shuicheng Yan, Xinchao Wang

Research Collection School Of Computing and Information Systems

Inspired by the long-range modeling ability of ViTs, large-kernel convolutions are widely studied and adopted recently to enlarge the receptive field and improve model performance, like the remarkable work ConvNeXt which employs 7×7 depthwise convolution. Although such depthwise operator only consumes a few FLOPs, it largely harms the model efficiency on powerful computing devices due to the high memory access costs. For example, ConvNeXtT has similar FLOPs with ResNet-50 but only achieves ∼ 60% throughputs when trained on A100 GPUs with full precision. Although reducing the kernel size of ConvNeXt can improve speed, it results in significant performance degradation, which …

Consistent3d: Towards Consistent High-Fidelity Text-To-3d Generation With Deterministic Sampling Prior, Zike Wu, Pan Zhou, Xuanyu Yi, Xiaoding Yuan, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Score distillation sampling (SDS) and its variants have greatly boosted the development of text-to-3D generation, but are vulnerable to geometry collapse and poor textures yet. To solve this issue, we first deeply analyze the SDS and find that its distillation sampling process indeed corresponds to the trajectory sampling of a stochastic differential equation (SDE): SDS samples along an SDE trajectory to yield a less noisy sample which then serves as a guidance to optimize a 3D model. However, the randomness in SDE sampling often leads to a diverse and unpredictable sample which is not always less noisy, and thus is …

Let’S Think Outside The Box: Exploring Leap-Of-Thought In Large Language Models With Multimodal Humor Generation, Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, Pan Zhou

Research Collection School Of Computing and Information Systems

Chain-of-Thought (CoT) [2, 3] guides large language models (LLMs) to reason step-by-step, and can motivate their logical reasoning ability. While effective for logical tasks, CoT is not conducive to creative problem-solving which often requires out-of-box thoughts and is crucial for innovation advancements. In this paper, we explore the Leap-of-Thought (LoT) abilities within LLMs — a nonsequential, creative paradigm involving strong associations and knowledge leaps. To this end, we study LLMs on the popular Oogiri game which needs participants to have good creativity and strong associative thinking for responding unexpectedly and humorously to the given image, text, or both, and thus …

Few-Shot Learner Parameterization By Diffusion Time-Steps, Zhongqi Yue, Pan Zhou, Richang Hong, Hanwang Zhang, Sun Qianru

Research Collection School Of Computing and Information Systems

Even when using large multi-modal foundation models, few-shot learning is still challenging—if there is no proper inductive bias, it is nearly impossible to keep the nuanced class attributes while removing the visually prominent attributes that spuriously correlate with class labels. To this end, we find an inductive bias that the time-steps of a Diffusion Model (DM) can isolate the nuanced class attributes, i.e., as the forward diffusion adds noise to an image at each time-step, nuanced attributes are usually lost at an earlier time-step than the spurious attributes that are visually prominent. Building on this, we propose Time-step Few-shot (TiF) …

Diffusion Time-Step Curriculum For One Image To 3d Generation, Xuanyu Yi, Zike Wu, Qingshan Xu, Pan Zhou, Joo Hwee Lim, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Score distillation sampling (SDS) has been widely adopted to overcome the absence of unseen views in reconstructing 3D objects from a single image. It leverages pretrained 2D diffusion models as teacher to guide the reconstruction of student 3D models. Despite their remarkable success, SDS-based methods often encounter geometric artifacts and texture saturation. We find out the crux is the overlooked indiscriminate treatment of diffusion time-steps during optimization: it unreasonably treats the studentteacher knowledge distillation to be equal at all time-steps and thus entangles coarse-grained and fine-grained modeling. Therefore, we propose the Diffusion Time-step Curriculum one-image-to-3D pipeline (DTC123), which involves both …

Jollygesture: Exploring Dual-Purpose Gestures In Vr Presentations, Gun Woo Warren Park, Anthony Tang, Fanny Chevalier

Research Collection School Of Computing and Information Systems

Virtual reality (VR) offers new opportunities for presenters to use expressive body language to engage their audience. Yet, most VR presentation systems have adopted control mechanisms that mimic those found in face-to-face presentation systems. We explore the use of gestures that have dual-purpose: first, for the audience, a communicative purpose; second, for the presenter, a control purpose to alter content in slides. To support presenters, we provide guidance on what gestures are available and their effects. We realize our design approach in JollyGesture, a VR technology probe that recognizes dual-purpose gestures in a presentation scenario. We evaluate our approach through …

Impact Of Similarities In Gender And Physical Appearance Between User And Embodied Conversational Agents On Trustworthiness, Empathy, And Service Evaluation, Sookyoung Park

Dartmouth College Master’s Theses

Embodied conversational agents (ECAs) have significantly enhanced human-machine interactions and show considerable potential in various industries such as customer service, education, healthcare, entertainment, and finance [1, 2]. This study explores the impact of similarities in gender and physical appearance between ECAs and users on the perceptions of trustworthiness, empathy, and service evaluation within the context of counselor ECAs. We conducted a within-subject experiment (n=50), using a 2x2 factorial arrangement, that varied the gender and the physical appearance of four distinct AI avatars. Participants interacted with each avatar, completing a post-experiment survey and participating in semi-structured interviews. Our findings indicate that …

Embodied Visions: Interactive Installations That Reimagine Bodily Presence In Digital Imaging Apparatuses As Shadows, Yunzi Shi

Dartmouth College Master’s Theses

Contextualized within a history of technological development, the evolution of imaging devices and technologies is accompanied by the abstraction of spatial relationships between the body of the observer, the apparatus, and physical reality, which leads to disembodying experiences for the observing subject. Compared with devices and interactive experiences, critical reflection on the epistemological impact of digital imaging devices has less priority in computational imaging and human-computer interaction research. Taking an artistic approach, this thesis describes Embodied Visions, an exhibition featuring three interactive installations exploring the technical infrastructure for imaging and reflecting on the (dis)embodied experiences in the digital age. …

Inchi Isotopologue And Isotopomer Specifications, Hunter N. B. Moseley, Philippe Rocca-Serra, Reza M. Salek, Masanori Arita, Emma L. Schymanski

Markey Cancer Center Faculty Publications

This work presents a proposed extension to the International Union of Pure and Applied Chemistry (IUPAC) International Chemical Identifier (InChI) standard that allows the representation of isotopically‑resolved chemi‑ cal entities at varying levels of ambiguity in isotope location. This extension includes an improved interpretation of the current isotopic layer within the InChI standard and a new isotopologue layer specification for representing chemical intensities with ambiguous isotope localization. Both improvements support the unique isotopically‑ resolved chemical identification of features detected and measured in analytical instrumentation, specifically nuclear magnetic resonance and mass spectrometry.

Scientific contribution

This new extension to the InChI standard …

Manipulative, Dark, And Unethical Design Practices In Ui & Ux Design, Ryan Edward Brown

Honors Theses

This thesis examines the pervasive and detrimental effects of manipulative user interface and user experience design (UI/UX) practices on individuals and society. Focusing on three critical areas – accessibility, dark patterns, and polarization – the study employs a mixed-methods approach, combining findings from a comprehensive literature review, an analysis of specific design patterns and methods, and a survey of user experiences.
The literature review covers topics such as the importance of accessibility in design education, the prevalence of dark patterns in mobile and desktop sites, the role of personalization algorithms in shaping user experiences, and the formation of echo chambers …

Companionship, Romance, And Self-Perception With Conversational Chatbots, Jonathan Windsor

Departmental Honors & Graduate Capstone Projects

Serving as a metaphorical gateway transcending the communicative barriers of physical relationships in interpersonal dialogues, artificial imators of human behavior and speech, also known as conversational chatbots; a simulation of human knowledge and existence in a bi-directional conversation, functions as a rhetor of expression. Spanning from contexts of professional to romantic, I serve to dissect and critically analyze the nuances of human-machine relationships based on pre-established literature, inviting ethical considerations and biases in their design and marketing. Corporate influences spark pre-established servitude-esque relationships with conversational agents. Professional applications, both task-oriented and emotionally based alike, paint a mixed picture of …

Simulating Cross-Scale Solid-Fluid Interaction Phenomena, Jinyuan Liu

Dartmouth College Ph.D Dissertations

Solid-fluid interactions are ubiquitous in nature, and accurate simulation methods are essential for realistic animation, industrial design, and engineering analysis. Com- pared to large-scale coupling phenomena, simulating fine-scale interactions poses extra challenges due to factors such as surface tension, material wettability, and geometric complexity. In this thesis, we pursue novel methodologies to accurately model in- terfacial dynamics between surface-tension fluids and codimensional solids, involving capillary interactions, controllable wettability, and robust contact behaviors. Our ini- tial approach involves developing a novel three-way coupling method, which utilizes a thin liquid membrane, modelled as a simplicial mesh, to facilitate accurate momen- tum transfer, …

Research Project Review: Human-Computer Interactions, Kylie E. Garcia

The Agora

A review of a body of research conducted by Dr. Gain Park, an assistant professor in the Department of Journalism and Media Studies at New Mexico State University. This review contains a summary of Dr. Park's research on human-computer interactions, commentary on its contributions and significance, as well as insights from Dr. Park.

An Exploration Of Procedural Methods In Game Level Design, Hector Salinas

Computer Science and Computer Engineering Undergraduate Honors Theses

Video games offer players immersive experiences within intricately crafted worlds, and the integration of procedural methods in game level designs extends this potential by introducing dynamic, algorithmically generated content that could stand on par with handcrafted environments. This research highlights the potential to provide players with engaging experiences through procedural level generation, while potentially reducing development time for game developers.

Through a focused exploration on two-dimensional cave generation techniques, this paper aims to provide efficient solutions tailored to this specific environment. This exploration encompasses several procedural generation methods, including Midpoint Displacement, Random Walk, Cellular Automata, Perlin Worms, and Binary Space …

Deep Reinforcement Learning Guided Improvement Heuristic For Job Shop Scheduling, Cong Zhang, Zhiguang Cao, Wen Song, Yaoxin Wu, Jie Zhang

Research Collection School Of Computing and Information Systems

Recent studies in using deep reinforcement learning (DRL) to solve Job-shop scheduling problems (JSSP) focus on construction heuristics. However, their performance is still far from optimality, mainly because the underlying graph representation scheme is unsuitable for modelling partial solutions at each construction step. This paper proposes a novel DRL-guided improvement heuristic for solving JSSP, where graph representation is employed to encode complete solutions. We design a Graph-Neural-Network-based representation scheme, consisting of two modules to effectively capture the information of dynamic topology and different types of nodes in graphs encountered during the improvement process. To speed up solution evaluation during improvement, …

Diffusion-Based Negative Sampling On Graphs For Link Prediction, Yuan Fang, Yuan Fang

Research Collection School Of Computing and Information Systems

Link prediction is a fundamental task for graph analysis with important applications on the Web, such as social network analysis and recommendation systems, etc. Modern graph link prediction methods often employ a contrastive approach to learn robust node representations, where negative sampling is pivotal. Typical negative sampling methods aim to retrieve hard examples based on either predefined heuristics or automatic adversarial approaches, which might be inflexible or difficult to control. Furthermore, in the context of link prediction, most previous methods sample negative nodes from existing substructures of the graph, missing out on potentially more optimal samples in the latent space. …

Swapvid: Integrating Video Viewing And Document Exploration With Direct Manipulation, Taichi Murakami, Kazuyuki Fujita, Kotaro Hara, Kazuki Takashima, Yoshifumi Kitamura

Research Collection School Of Computing and Information Systems

Videos accompanied by documents—document-based videos—enable presenters to share contents beyond videos and audience to use them for detailed content comprehension. However, concurrently exploring multiple channels of information could be taxing. We propose SwapVid, a novel interface for viewing and exploring document-based videos. SwapVid seamlessly integrates a video and a document into a single view and lets the content behaves as both video and a document; it adaptively switches a document-based video to act as a video or a document upon direct manipulation (e.g., scrolling the document, manipulating the video timeline). We conducted a user study with twenty participants, comparing SwapVid …

Automated Cinematographer For Vr Viewing Experiences, Zihan Wu

Dartmouth College Master’s Theses

As the virtual reality (VR) industry continues to evolve, the question of how to effectively capture VR experiences for an audience remains a challenge. The predominant method of showcasing VR applications through first-person recordings lacks cinematic interest, failing to capture other viewpoints and the essence of the moment. Meanwhile, manually setting up cameras and editing videos requires technical expertise on behalf of the user. In this paper, we propose the use of machine learning (ML) to automatically select the most compelling predefined viewpoint in a VR environment, at any given moment. Our models, trained on actor motion and voice volume, …

Privacy Protection In Mobile Photography With Face Cloaking, Rithyka Heng

Computer Science and Computer Engineering Undergraduate Honors Theses

In a world of increasing connectivity, privacy is becoming ever-more difficult to maintain. People have little control over the capture of their image while in public and have even less control over the online sharing or posting of their image. This leaves many people vulnerable to being tracked or profiled via their image’s presence in other people’s photos. This thesis implements and evaluates an approach to privacy protection that involves the photographers protecting the privacy of bystanders. Because most photographs are now being taken by smartphones, a mobile application is decidedly the technology that would best achieve widespread adoption and …

Instant3d: Instant Text-To-3d Generation, Ming Li, Pan Zhou, Jia-Wei Liu, Jussi Keppo, Shuicheng Yan, Xiangyu Xu

Research Collection School Of Computing and Information Systems

Text-to-3D generation has attracted much attention from the computer vision community. Existing methods mainly optimize a neural field from scratch for each text prompt, relying on heavy and repetitive training cost which impedes their practical deployment. In this paper, we propose a novel framework for fast text-to-3D generation, dubbed Instant3D. Once trained, Instant3D is able to create a 3D object for an unseen text prompt in less than one second with a single run of a feedforward network. We achieve this remarkable speed by devising a new network that directly constructs a 3D triplane from a text prompt. The core …

The Impact Of Avatar Completeness On Embodiment And The Detectability Of Hand Redirection In Virtual Reality, Martin Feick, Andre Zenner, Simon Seibert, Anthony Tang, Antonio Krüger

Research Collection School Of Computing and Information Systems

To enhance interactions in VR, many techniques introduce offsets between the virtual and real-world position of users’ hands. Nevertheless, such hand redirection (HR) techniques are only effective as long as they go unnoticed by users—not disrupting the VR experience. While several studies consider how much unnoticeable redirection can be applied, these focus on mid-air floating hands that are disconnected from users’ bodies. Increasingly, VR avatars are embodied as being directly connected with the user’s body, which provide more visual cue anchoring, and may therefore reduce the unnoticeable redirection threshold. In this work, we studied more complete avatars and their effect …

Vaid: Indexing View Designs In Visual Analytics System, Lu Ying, Aoyu Wu, Haotian Li, Zikun Deng, Ji Lan, Jiang Wu, Yong Wang, Huamin Qu, Dazhen Deng, Yingcai Wu

Research Collection School Of Computing and Information Systems

Visual analytics (VA) systems have been widely used in various application domains. However, VA systems are complex in design, which imposes a serious problem: although the academic community constantly designs and implements new designs, the designs are difficult to query, understand, and refer to by subsequent designers. To mark a major step forward in tackling this problem, we index VA designs in an expressive and accessible way, transforming the designs into a structured format. We first conducted a workshop study with VA designers to learn user requirements for understanding and retrieving professional designs in VA systems. Thereafter, we came up …

3-D Reconstruction For Underwater Robots With A Monocular Camera And Lights, Monika Roznere

Dartmouth College Ph.D Dissertations

Before a robot can act, it must perceive its environment. Though, this is not a simple task when considering the challenges in underwater domains -- poor visibility conditions, limited sensor configurations, and lack of readily accessible localization. Underwater robots have, nevertheless, improved dramatically with more extensive sensor and navigation equipment. Robot and sensor use have enabled us to explore all reaches of our oceans. On the other hand, these same robots are not easily accessible or transferable to many practical tasks, including fishery management, infrastructure maintenance, disaster response, site conservation, and ecological surveys. There is a growing need for robots …

An Empirical Study On The Efficacy Of Llm-Powered Chatbots In Basic Information Retrieval Tasks, Naja Faysal

Electronic Theses, Projects, and Dissertations

The rise of conversational user interfaces (CUIs) powered by large language models (LLMs) is transforming human-computer interaction. This study evaluates the efficacy of LLM-powered chatbots, trained on website data, compared to browsing websites for finding information about organizations across diverse sectors. A within-subjects experiment with 165 participants was conducted, involving similar information retrieval (IR) tasks using both websites (GUIs) and chatbots (CUIs). The research questions are: (Q1) Which interface helps users find information faster: LLM chatbots or websites? (Q2) Which interface helps users find more accurate information: LLM chatbots or websites?. The findings are: (Q1) Participants found information significantly faster …

Multi-Script Handwriting Identification By Fragmenting Strokes, Joshua Jude Thomas

Graduate Theses and Dissertations (2019 - present)

This study tests the effectiveness of Multi-Script Handwriting Identification after simplifying character strokes, by segmenting them into sub-parts. Character simplification is performed through splitting the character by branching-points and end-points, a process called stroke fragmentation in this study. The resulting sub-parts of the character are called stroke fragments and are evaluated individually to identify the writer. This process shares similarities with the concept of stroke decomposition in Optical Character Recognition which attempts to recognize characters through the writing strokes that make them up. The main idea of this study is that the characters of different writing‑scripts (English, Chinese, etc.) may …

Exploring The Relationship Between Anxiety And Virtual Reality Sickness, David Wesley Woolverton

Graduate Theses and Dissertations (2019 - present)

As virtual reality (VR) becomes more commonly used in education, it is important to understand the technology’s weakness and mitigate any potential negative effects on student success. One adverse side-effect of VR use is simulation-induced motion sickness, known in the context of VR as VR sickness. Previous research by Howard and Van Zandt (2021) found that possessing a phobia had a significant positive correlation with VR sickness, but only if the phobia is triggered by the simulation, suggesting that symptoms are actually connected to the anxiety the phobia induces. This study explored the hypothesized correlation between anxiety and VR sickness, …

Next-Generation Crop Monitoring Technologies: Case Studies About Edge Image Processing For Crop Monitoring And Soil Water Property Modeling Via Above-Ground Sensors, Nipuna Chamara

Dissertations and Doctoral Documents, University of Nebraska-Lincoln, 2023–

Artificial Intelligence (AI) has advanced rapidly in the past two decades. Internet of Things (IoT) technology has advanced rapidly during the last decade. Merging these two technologies has immense potential in several industries, including agriculture.

We have identified several research gaps in utilizing IoT technology in agriculture. One problem was the digital divide between rural, unconnected, or limited connected areas and urban areas for utilizing images for decision-making, which has advanced with the growth of AI. Another area for improvement was the farmers' demotivation to use in-situ soil moisture sensors for irrigation decision-making due to inherited installation difficulties. As Nebraska …

The Kruger Collection Reimagined: A Case Study In 3d Scanning And Interactive Exhibit Design, Annissa Davis

Department of Anthropology: Theses and Student Research

This thesis examines the use of 3D modeling in museum exhibition to create exploratory exhibits that facilitate unique relationships between the visitors and the collection beyond what is provided by the collection’s in person counterparts. Typical use of 3D modeling in museums is currently often representative rather than exploratory. By employing a Digital Humanities lens to approach the development of a digital exhibition utilizing 3D technology and interactive elements created in a video game engine (Unity), this thesis project evaluates these potential new relationships. Using the Eloise Kruger Collection of Miniatures as a case study, the following text details the …

Vr Circuit Simulation With Advanced Visualization For Enhancing Comprehension In Electrical Engineering, Elliott Wolbach

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

As technology advances, the field of electrical and computer engineering continuously demands innovative tools and methodologies to facilitate effective learning and comprehension of fundamental concepts. Through a comprehensive literature review, it was discovered that there was a gap in the current research on using VR technology to effectively visualize and comprehend non-observable electrical characteristics of electronic circuits. This thesis explores the integration of Virtual Reality (VR) technology and real-time electronic circuit simulation with enhanced visualization of non-observable concepts such as voltage distribution and current flow within these circuits. The primary objective is to develop an immersive educational platform that makes …