Graphics and Human Computer Interfaces Commons^™

Learning Topological Representations With Bidirectional Graph Attention Network For Solving Job Shop Scheduling Problem, Cong Zhang, Zhiguang Cao, Yaoxin Wu, Wen Song, Jing Sun

Research Collection School Of Computing and Information Systems

Existing learning-based methods for solving job shop scheduling problems (JSSP) usually use off-the-shelf GNN models tailored to undirected graphs and neglect the rich and meaningful topological structures of disjunctive graphs (DGs). This paper proposes the topology-aware bidirectional graph attention network (TBGAT), a novel GNN architecture based on the attention mechanism, to embed the DG for solving JSSP in a local search framework. Specifically, TBGAT embeds the DG from a forward and a backward view, respectively, where the messages are propagated by following the different topologies of the views and aggregated via graph attention. Then, we propose a novel operator based …

How People Prompt Generative Ai To Create Interactive Vr Scenes, Setareh Aghel Manesh, Tianyi Zhang, Yuki Onishi, Kotaro Hara, Scott Bateman, Jiannan Li, Anthony Tang

Research Collection School Of Computing and Information Systems

Generative AI tools can provide people with the ability to create virtual environments and scenes with natural language prompts. Yet, how people will formulate such prompts is unclear---particularly when they inhabit the environment that they are designing. For instance, it is likely that a person might say, "Put a chair here,'' while pointing at a location. If such linguistic and embodied features are common to people's prompts, we need to tune models to accommodate them. In this work, we present a Wizard of Oz elicitation study with 22 participants, where we studied people's implicit expectations when verbally prompting such programming …

My Ai Companion: An Examination Of The Removal Of Erotic Role Play From Replika Through User Discussion On Reddit, Chelsee M. Allen

Department of Sociology: Dissertations, Theses, and Student Research

The development of artificial intelligence (AI) software has expanded rapidly in recent years, and thus has emerged the importance of exploring human relationships with AI chatbots. Replika, an app which uses AI to mimic human conversation, removed a function called Erotic Role Play (ERP) that allowed for sexual conversation with users’ customizable chatbots in February of 2023. This exploratory qualitative study examines the aftermath of ERP’s removal through an analysis of user interactions on Reddit. Five overarching themes emerged through the analysis of top posts to a Replika-specific subreddit, encompassing topics around mental health, stigma, coping, sex work and gendered …

Jigsaw: Edge-Based Streaming Perception Over Spatially Overlapped Multi-Camera Deployments, Ila Gokarn, Yigong Hu, Tarek Abdelzaher, Archan Misra

Research Collection School Of Computing and Information Systems

We present JIGSAW, a novel system that performs edge-based streaming perception over multiple video streams, while additionally factoring in the redundancy offered by the spatial overlap often exhibited in urban, multi-camera deployments. To assure high streaming throughput, JIGSAW extracts and spatially multiplexes multiple regions-of-interest from different camera frames into a smaller canvas frame. Moreover, to ensure that perception stays abreast of evolving object kinematics, JIGSAW includes a utility-based weighted scheduler to preferentially prioritize and even skip object-specific tiles extracted from an incoming stream of camera frames. Using the CityflowV2 traffic surveillance dataset, we show that JIGSAW can simultaneously process 25 …

Violet: Visual Analytics For Explainable Quantum Neural Networks, Shaolun Ruan, Zhiding Liang, Qiang Guan, Paul Robert Griffin, Xiaolin Wen, Yanna Lin, Yong Wang

Research Collection School Of Computing and Information Systems

With the rapid development of Quantum Machine Learning, quantum neural networks (QNN) have experienced great advancement in the past few years, harnessing the advantages of quantum computing to significantly speed up classical machine learning tasks. Despite their increasing popularity, the quantum neural network is quite counter-intuitive and difficult to understand, due to their unique quantum-specific layers (e.g., data encoding and measurement) in their architecture. It prevents QNN users and researchers from effectively understanding its inner workings and exploring the model training status. To fill the research gap, we propose VIOLET , a novel visual analytics approach to improve the explainability …

Posmlp-Video: Spatial And Temporal Relative Position Encoding For Efficient Video Recognition, Yanbin Hao, Diansong Zhou, Zhicai Wang, Chong-Wah Ngo, Xiangnan He, Meng Wang

Research Collection School Of Computing and Information Systems

In recent years, vision Transformers and MLPs have demonstrated remarkable performance in image understanding tasks. However, their inherently dense computational operators, such as self-attention and token-mixing layers, pose significant challenges when applied to spatio-temporal video data. To address this gap, we propose PosMLP-Video, a lightweight yet powerful MLP-like backbone for video recognition. Instead of dense operators, we use efficient relative positional encoding (RPE) to build pairwise token relations, leveraging small-sized parameterized relative position biases to obtain each relation score. Specifically, to enable spatio-temporal modeling, we extend the image PosMLP’s positional gating unit to temporal, spatial, and spatio-temporal variants, namely PoTGU, …

More Human-Likeness, Less Self-Disclosure? Avatars' Form Realism And Job Applicants' Self-Disclosure In Ai Interviews, Yamin Xu, Keng Siau, Fiona Fui-Hoon Nah

Research Collection School Of Computing and Information Systems

The rise of AI in recruitment promises to revolutionize how organizations evaluate job candidates. The quality of AI evaluations is determined by the input data, which depends on job applicants' self-disclosure. However, little is known about how the design elements of AI interview systems, particularly avatar interviewers, influence job applicants' self-disclosure during these interactions. This study aims to address this gap by specifically focusing on how the form realism of avatar interviewers affects job applicants' self-disclosure through their perceptions. In addition, the study will examine the effects of job type as a moderator. Drawing on the Stimulus-Organism-Response (S-O-R) model, this …

Combinatorial Creativity: Knowledge Graphs And Idea Generation In Crowdsourcing Innovation, Zhi Wei Vincent Mack

Dissertations and Theses Collection (Open Access)

This dissertation explores the dynamic interplay between combinatorial creativity and technology-driven innovation within various knowledge-intensive fields. It critically examines the role of combinatorial creativity in generating groundbreaking innovations by amalgamating existing ideas and technologies. This research incorporates a detailed examination of how knowledge, whether tacit or explicit, can be transformed into actionable data to foster innovation in crowdsourcing contexts. Chapter 2 provides an overview of the relevant literature on how Artificial Intelligence and Knowledge Management Systems can support combinatorial creativity. The study further delves into the transformative impact of knowledge management systems, particularly focusing on crowdsourcing platforms that leverage collective …

Community Discovery Over Attributed Graphs, Yudong Niu

Dissertations and Theses Collection (Open Access)

Community discovery, as a fundamental problem in graph mining, finds applications in various domains such as biological analysis, system optimization and fraud detection. Although many efforts have been made to address community discovery based on graph topology, few works have been devoted to community discovery over attributed graphs, where graphs are equipped with attribute information such as node and edge types. Thus, this thesis is devoted to designing innovative solutions that can utilize the attribute information together with graph topology for community discovery. In particular, we study novel problems with efficient algorithms for both homogeneous and heterogeneous attributed graphs and …

Open-Vocabulary Video Anomaly Detection, Peng Wu, Xuerong Zhou, Guansong Pang, Yujia Sun, Jing Liu, Peng Wang, Yanning Zhang

Research Collection School Of Computing and Information Systems

Current video anomaly detection (VAD) approaches with weak supervisions are inherently limited to a closed-set setting and may struggle in open-world applications where there can be anomaly categories in the test data unseen during training. A few recent studies attempt to tackle a more realistic setting, open-set VAD, which aims to de-tect unseen anomalies given seen anomalies and normal videos. However, such a setting focuses on predicting frame anomaly scores, having no ability to recognize the specific categories of anomalies, despite the fact that this ability is essential for building more informed video surveillance systems. This paper takes a step …

Drag Your Noise: Interactive Point-Based Editing Via Diffusion Semantic Propagation, Haofeng Liu, Chenshu Xu, Yifei Yang, Lihua Zeng, Shengfeng He

Research Collection School Of Computing and Information Systems

Point-based interactive editing serves as an essential tool to complement the controllability of existing generative models. A concurrent work, DragDiffusion, updates the diffusion latent map in response to user inputs, causing global latent map alterations. This results in imprecise preservation of the original content and unsuccessful editing due to gradient vanishing. In contrast, we present DragNoise, offering robust and accelerated editing without retracing the latent map. The core rationale of DragNoise lies in utilizing the predicted noise output of each U-Net as a semantic editor. This approach is grounded in two critical observations: firstly, the bottleneck features of U-Net inherently …

Beyond Textual Constraints : Learning Novel Diffusion Conditions With Fewer Examples, Yuyang Yu, Bangzhen Liu, Chenxi Zheng, Xuemiao Xu, Huaidong Zhang, Shengfeng He

Research Collection School Of Computing and Information Systems

In this paper, we delve into a novel aspect of learning novel diffusion conditions with datasets an order of magnitude smaller. The rationale behind our approach is the elimination of textual constraints during the few-shot learning process. To that end, we implement two optimization strategies. The first, prompt-free conditional learning, utilizes a prompt-free encoder derived from a pre-trained Stable Diffusion model. This strategy is designed to adapt new conditions to the diffusion process by minimizing the textual-visual cor-relation, thereby ensuring a more precise alignment between the generated content and the specified conditions. The second strategy entails condition-specific negative rectification, which …

Learning With Unreliability : Fast Few-Shot Voxel Radiance Fields With Relative Geometric Consistency, Yingjie Xu, Bangzhen Liu, Hao Tang, Bailin Deng, Shengfeng He

Research Collection School Of Computing and Information Systems

We propose a voxel-based optimization framework, Re VoRF, for few-shot radiance fields that strategically ad-dress the unreliability in pseudo novel view synthesis. Our method pivots on the insight that relative depth relationships within neighboring regions are more reliable than the ab-solute color values in disoccluded areas. Consequently, we devise a bilateral geometric consistency loss that carefully navigates the trade-off between color fidelity and geometric accuracy in the context of depth consistency for uncertain regions. Moreover, we present a reliability-guided learning strategy to discern and utilize the variable quality across syn-thesized views, complemented by a reliability-aware voxel smoothing algorithm that smoothens …

D3still : Decoupled Differential Distillation For Asymmetric Image Retrieval, Yi Xie, Yihong Lin, Wenjie Cai, Xuemiao Xu, Huaidong Zhang, Yong Du, Shengfeng He

Research Collection School Of Computing and Information Systems

Existing methods for asymmetric image retrieval employ a rigid pairwise similarity constraint between the query network and the larger gallery network. However, these oneto-one constraint approaches often fail to maintain retrieval order consistency, especially when the query network has limited representational capacity. To overcome this problem, we introduce the Decoupled Differential Distillation (D3still) framework. This framework shifts from absolute one-to-one supervision to optimizing the relational differences in pairwise similarities produced by the query and gallery networks, thereby preserving a consistent retrieval order across both networks. Our method involves computing a pairwise similarity differential matrix within the gallery domain, which is …

Rethinking Multi-View Representation Learning Via Distilled Disentangling, Guanzhou Ke, Bo Wang, Xiaoli Wang, Shengfeng He

Research Collection School Of Computing and Information Systems

Multi-view representation learning aims to derive robust representations that are both view-consistent and view-specific from diverse data sources. This paper presents an in-depth analysis of existing approaches in this domain, highlighting a commonly overlooked aspect: the redundancy between view-consistent and view-specific representations. To this end, we propose an innovative framework for multi-view representation learning, which incorporates a technique we term 'distilled disentangling'. Our method introduces the concept of masked cross-view prediction, enabling the extraction of compact, high-quality view-consistent representations from various sources without incurring extra computational overhead. Additionally, we develop a distilled disentangling module that efficiently filters out consistency-related information …

Impact Of Similarities In Gender And Physical Appearance Between User And Embodied Conversational Agents On Trustworthiness, Empathy, And Service Evaluation, Sookyoung Park

Dartmouth College Master’s Theses

Embodied conversational agents (ECAs) have significantly enhanced human-machine interactions and show considerable potential in various industries such as customer service, education, healthcare, entertainment, and finance [1, 2]. This study explores the impact of similarities in gender and physical appearance between ECAs and users on the perceptions of trustworthiness, empathy, and service evaluation within the context of counselor ECAs. We conducted a within-subject experiment (n=50), using a 2x2 factorial arrangement, that varied the gender and the physical appearance of four distinct AI avatars. Participants interacted with each avatar, completing a post-experiment survey and participating in semi-structured interviews. Our findings indicate that …

Companionship, Romance, And Self-Perception With Conversational Chatbots, Jonathan Windsor

Departmental Honors & Graduate Capstone Projects

Serving as a metaphorical gateway transcending the communicative barriers of physical relationships in interpersonal dialogues, artificial imators of human behavior and speech, also known as conversational chatbots; a simulation of human knowledge and existence in a bi-directional conversation, functions as a rhetor of expression. Spanning from contexts of professional to romantic, I serve to dissect and critically analyze the nuances of human-machine relationships based on pre-established literature, inviting ethical considerations and biases in their design and marketing. Corporate influences spark pre-established servitude-esque relationships with conversational agents. Professional applications, both task-oriented and emotionally based alike, paint a mixed picture of …

Diffusion-Based Negative Sampling On Graphs For Link Prediction, Yuan Fang, Yuan Fang

Research Collection School Of Computing and Information Systems

Link prediction is a fundamental task for graph analysis with important applications on the Web, such as social network analysis and recommendation systems, etc. Modern graph link prediction methods often employ a contrastive approach to learn robust node representations, where negative sampling is pivotal. Typical negative sampling methods aim to retrieve hard examples based on either predefined heuristics or automatic adversarial approaches, which might be inflexible or difficult to control. Furthermore, in the context of link prediction, most previous methods sample negative nodes from existing substructures of the graph, missing out on potentially more optimal samples in the latent space. …

Automated Cinematographer For Vr Viewing Experiences, Zihan Wu

Dartmouth College Master’s Theses

As the virtual reality (VR) industry continues to evolve, the question of how to effectively capture VR experiences for an audience remains a challenge. The predominant method of showcasing VR applications through first-person recordings lacks cinematic interest, failing to capture other viewpoints and the essence of the moment. Meanwhile, manually setting up cameras and editing videos requires technical expertise on behalf of the user. In this paper, we propose the use of machine learning (ML) to automatically select the most compelling predefined viewpoint in a VR environment, at any given moment. Our models, trained on actor motion and voice volume, …

Privacy Protection In Mobile Photography With Face Cloaking, Rithyka Heng

Computer Science and Computer Engineering Undergraduate Honors Theses

In a world of increasing connectivity, privacy is becoming ever-more difficult to maintain. People have little control over the capture of their image while in public and have even less control over the online sharing or posting of their image. This leaves many people vulnerable to being tracked or profiled via their image’s presence in other people’s photos. This thesis implements and evaluates an approach to privacy protection that involves the photographers protecting the privacy of bystanders. Because most photographs are now being taken by smartphones, a mobile application is decidedly the technology that would best achieve widespread adoption and …

3-D Reconstruction For Underwater Robots With A Monocular Camera And Lights, Monika Roznere

Dartmouth College Ph.D Dissertations

Before a robot can act, it must perceive its environment. Though, this is not a simple task when considering the challenges in underwater domains -- poor visibility conditions, limited sensor configurations, and lack of readily accessible localization. Underwater robots have, nevertheless, improved dramatically with more extensive sensor and navigation equipment. Robot and sensor use have enabled us to explore all reaches of our oceans. On the other hand, these same robots are not easily accessible or transferable to many practical tasks, including fishery management, infrastructure maintenance, disaster response, site conservation, and ecological surveys. There is a growing need for robots …

An Empirical Study On The Efficacy Of Llm-Powered Chatbots In Basic Information Retrieval Tasks, Naja Faysal

Electronic Theses, Projects, and Dissertations

The rise of conversational user interfaces (CUIs) powered by large language models (LLMs) is transforming human-computer interaction. This study evaluates the efficacy of LLM-powered chatbots, trained on website data, compared to browsing websites for finding information about organizations across diverse sectors. A within-subjects experiment with 165 participants was conducted, involving similar information retrieval (IR) tasks using both websites (GUIs) and chatbots (CUIs). The research questions are: (Q1) Which interface helps users find information faster: LLM chatbots or websites? (Q2) Which interface helps users find more accurate information: LLM chatbots or websites?. The findings are: (Q1) Participants found information significantly faster …

Next-Generation Crop Monitoring Technologies: Case Studies About Edge Image Processing For Crop Monitoring And Soil Water Property Modeling Via Above-Ground Sensors, Nipuna Chamara

Dissertations and Doctoral Documents, University of Nebraska-Lincoln, 2023–

Artificial Intelligence (AI) has advanced rapidly in the past two decades. Internet of Things (IoT) technology has advanced rapidly during the last decade. Merging these two technologies has immense potential in several industries, including agriculture.

We have identified several research gaps in utilizing IoT technology in agriculture. One problem was the digital divide between rural, unconnected, or limited connected areas and urban areas for utilizing images for decision-making, which has advanced with the growth of AI. Another area for improvement was the farmers' demotivation to use in-situ soil moisture sensors for irrigation decision-making due to inherited installation difficulties. As Nebraska …

Vr Circuit Simulation With Advanced Visualization For Enhancing Comprehension In Electrical Engineering, Elliott Wolbach

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

As technology advances, the field of electrical and computer engineering continuously demands innovative tools and methodologies to facilitate effective learning and comprehension of fundamental concepts. Through a comprehensive literature review, it was discovered that there was a gap in the current research on using VR technology to effectively visualize and comprehend non-observable electrical characteristics of electronic circuits. This thesis explores the integration of Virtual Reality (VR) technology and real-time electronic circuit simulation with enhanced visualization of non-observable concepts such as voltage distribution and current flow within these circuits. The primary objective is to develop an immersive educational platform that makes …

Learning Nighttime Semantic Segmentation The Hard Way, Wenxi Liu, Jiaxin Cai, Qi Li, Chenyang Liao, Jingjing Cao, Shengfeng He, Yuanlong Yu

Research Collection School Of Computing and Information Systems

Nighttime semantic segmentation is an important but challenging research problem for autonomous driving. The major challenges lie in the small objects or regions from the under-/over-exposed areas or suffer from motion blur caused by the camera deployed on moving vehicles. To resolve this, we propose a novel hard- class-aware module that bridges the main network for full-class segmentation and the hard-class network for segmenting aforementioned hard-class objects. In specific, it exploits the shared focus of hard-class objects from the dual-stream network, enabling the contextual information flow to guide the model to concentrate on the pixels that are hard to classify. …

Fashionregen: Llm‑Empowered Fashion Report Generation, Yujuan Ding, Yunshan Ma, Wenqi Fan, Yige Yao, Tat‑Seng Chua, Qing Li

Research Collection School Of Computing and Information Systems

Fashion analysis refers to the process of examining and evaluating trends, styles, and elements within the fashion industry to understand and interpret its current state, generating fashion reports. It is traditionally performed by fashion professionals based on their expertise and experience, which requires high labour cost and may also produce biased results for relying heavily on a small group of people. In this paper, to tackle the Fashion Report Generation (FashionReGen) task, we propose an intelligent Fashion Analyzing and Reporting system based the advanced Large Language Models (LLMs), debbed as GPT-FAR. Specifically, it tries to deliver FashionReGen based on effective …

Immersive Japanese Language Learning Web Application Using Spaced Repetition, Active Recall, And An Artificial Intelligent Conversational Chat Agent Both In Voice And In Text, Marc Butler

MS in Computer Science Project Reports

In the last two decades various human language learning applications, spaced repetition software, online dictionaries, and artificial intelligent chat agents have been developed. However, there is no solution to cohesively combine these technologies into a comprehensive language learning application including skills such as speaking, typing, listening, and reading. Our contribution is to provide an immersive language learning web application to the end user which combines spaced repetition, a study technique used to review information at systematic intervals, and active recall, the process of purposely retrieving information from memory during a review session, with an artificial intelligent conversational chat agent both …

Visualizing Routes With Ai-Discovered Street-View Patterns, Tsung Heng Wu, Md Amiruzzaman, Ye Zhao, Deepshikha Bhati, Jing Yang

Computer Science Faculty Publications

Street-level visual appearances play an important role in studying social systems, such as understanding the built environment, driving routes, and associated social and economic factors. It has not been integrated into a typical geographical visualization interface (e.g., map services) for planning driving routes. In this article, we study this new visualization task with several new contributions. First, we experiment with a set of AI techniques and propose a solution of using semantic latent vectors for quantifying visual appearance features. Second, we calculate image similarities among a large set of street-view images and then discover spatial imagery patterns. Third, we integrate …

Hop‑Based Heterogeneous Graph Transformer, Zixuan Yang, Xiao Wang, Yanhua Yu, Yuling Wang, Kangkang Lu, Zirui Guo, Xiting Qin, Yunshan Ma, Tat‑Seng Chua

Research Collection School Of Computing and Information Systems

The Graph Transformer (GT) has shown significant ability in processing graph-structured data, addressing limitations in graph neural networks, such as over-smoothing and over-squashing. However, the implementation of GT in real-world heterogeneous graphs (HGs) with complex topology continues to present numerous challenges. Firstly, a challenge arises in designing a tokenizer that is compatible with heterogeneity. Secondly, the complexity of the transformer hampers the acquisition of high-order neighbor information in HGs. In this paper, we propose a novel Hop-basedHeterogeneous Graph Transformer (H2Gormer) framework, paving a promising path for HGs to benefit from the capabilities of Transformers. We propose a Heterogeneous Hop-based Token …

Test-Time Augmentation For 3d Point Cloud Classification And Segmentation, Tuan-Anh Vu, Srinjay Sarkar, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung

Research Collection School Of Computing and Information Systems

Data augmentation is a powerful technique to enhance the performance of a deep learning task but has received less attention in 3D deep learning. It is well known that when 3D shapes are sparsely represented with low point density, the performance of the downstream tasks drops significantly. This work explores test-time augmentation (TTA) for 3D point clouds. We are inspired by the recent revolution of learning implicit representation and point cloud upsampling, which can produce high-quality 3D surface reconstruction and proximity-to-surface, respectively. Our idea is to leverage the implicit field reconstruction or point cloud upsampling techniques as a systematic way …