Graphics and Human Computer Interfaces Commons^™

Leveraging Artificial Intelligence For Team Cognition In Human-Ai Teams, Beau Schelble

All Dissertations

Advances in artificial intelligence (AI) technologies have enabled AI to be applied across a wide variety of new fields like cryptography, art, and data analysis. Several of these fields are social in nature, including decision-making and teaming, which introduces a new set of challenges for AI research. While each of these fields has its unique challenges, the area of human-AI teaming is beset with many that center around the expectations and abilities of AI teammates. One such challenge is understanding team cognition in these human-AI teams and AI teammates' ability to contribute towards, support, and encourage it. Team cognition is …

Data-Centric Image Super-Resolution In Magnetic Resonance Imaging: Challenges And Opportunities, Mamata Shrestha

Graduate Theses and Dissertations

Super-resolution has emerged as a crucial research topic in the field of Magnetic Resonance Imaging (MRI) where it plays an important role in understanding and analysis of complex, qualitative, and quantitative characteristics of tissues at high resolutions. Deep learning techniques have been successful in achieving state-of-the-art results for super-resolution. These deep learning-based methods heavily rely on a substantial amount of data. Additionally, they require a pair of low-resolution and high-resolution images for supervised training which is often unavailable. Particularly in MRI super-resolution, it is often impossible to have low-resolution and high-resolution training image pairs. To overcome this, existing methods for …

The Propagation And Execution Of Malware In Images, Piper Hall

Cybersecurity Undergraduate Research Showcase

Malware has become increasingly prolific and severe in its consequences as information systems mature and users become more reliant on computing in their daily lives. As cybercrime becomes more complex in its strategies, an often-overlooked manner of propagation is through images. In recent years, several high-profile vulnerabilities in image libraries have opened the door for threat actors to steal money and information from unsuspecting users. This paper will explore the mechanisms by which these exploits function and how they can be avoided.

Performative Mixing For Immersive Audio, Brian A. Elizondo

LSU Doctoral Dissertations

Immersive multichannel audio can be produced with specialized setups of loudspeakers, often surrounding the audience. These setups can feature as few as four loudspeakers or more than 300. Performative mixing in these environments requires a bespoke solution offering intuitive gestural control. Beyond the usual faders for gain control, advancements in multichannel sound demand interfaces capable of quickly positioning sounds between channels. The Quad Cartesian Positioner is such a solution in the form of a Eurorack module for surround mixing for use in live or studio performances.

Diffusion/mixing methods for live multichannel immersive music often rely on the repurposing of hardware …

User Feedback On Celebratory Technology Model For Reducing Stigma, Evelyn Lawrie, Daniel Dinh, Sav Avalos, Jack De Bruyn, Spencer Au, Christian Lopez, Ray Tan, Cyrus Fa'amafoe

Student Scholar Symposium Abstracts and Posters

Social stigma is a complex manifestation that affects humanity, particularly individuals with disabilities and other marginalized groups, including those with physical, cognitive, and emotional conditions. Society often judges these individuals' interactions with the world, and many technologies designed to assist those with disabilities attempt to change their daily interactions and behaviors. Nonetheless, when the emphasis is placed on validating disabled identities, there is a potential for it to be seen as "inspiration porn." This approach might inadvertently reduce inclusivity and do little to challenge negative stereotypes; it can also lead to the objectification of individuals with disabilities. Therefore, this project …

Vrmovian - An Immersive Data Annotation Tool For Visual Analysis Of Human Interactions In Vr, Isaac Browen

Student Scholar Symposium Abstracts and Posters

Understanding human behavior in virtual reality (VR) is a key component for developing intelligent systems to enhance human focused VR experiences. The ability to annotate human motion data proves to be a very useful way to analyze and understand human behavior. However, due to the complexity and multi-dimensionality of human activity data, it is necessary to develop software that can display the data in a comprehensible way and can support intuitive data annotation for developing machine learning models able recognize and assist human motions in VR (e.g., remote physical therapy). Although past research has been done to improve VR data …

Bridging Domain Gaps For Cross-Spectrum And Long-Range Face Recognition Using Domain Adaptive Machine Learning, Cedric Armel Nimpa Fondje

Dissertations and Doctoral Documents, University of Nebraska-Lincoln, 2023–

Face recognition technology has witnessed significant advancements in recent decades, enabling its widespread adoption in various applications such as security, surveillance, and biometrics applications. However, one of the primary challenges faced by existing face recognition systems is their limited performance when presented with images from different modalities or domains( such as infrared to visible, long range to close range, nighttime to daytime, profile to f rontal, etc.) Additionally, advancements in camera sensors, analytics beyond the visible spectrum, and the increasing size of cross-modal datasets have led to a particular interest in cross-modal learning for face recognition in the biometrics and …

Enhancing Search Engine Results: A Comparative Study Of Graph And Timeline Visualizations For Semantic And Temporal Relationship Discovery, Muhammad Shahiq Qureshi

Electronic Theses and Dissertations

In today’s digital age, search engines have become indispensable tools for finding information among the corpus of billions of webpages. The standard that most search engines follow is to display search results in a list-based format arranged according to a ranking algorithm. Although this format is good for presenting the most relevant results to users, it fails to represent the underlying relations between different results. These relations, among others, can generally be of either a temporal or semantic nature. A user who wants to explore the results that are connected by those relations would have to make a manual effort …

Designing Secure Mental Healthcare Chatbots For Older Adults, Aishwarya Surani

Electronic Theses and Dissertations

The landscape of mental health support has evolved as a result of the rising demand for digital mental healthcare services. Users now have an opportunity to seek mental health support online due to the growth of digital platforms. For those looking for mental health treatments, chatbots have evolved as user-friendly, accessible platforms that provide remote access and convenience. However, for chatbots to be effective, users must divulge personal and sensitive information, such as demographics, insurance information, and a history of mental illness. While chatbots offer services to a variety of demographic users, older adults face unique challenges related to usability, …

Optimizing E-Payment Applications For Older Adults: User-Centered Solutions To Improve Security, Privacy, Usability, And Accessibility, Urvashi Kishnani

Electronic Theses and Dissertations

In an increasingly digital world, older adults are rapidly becoming a vital demographic in the realm of electronic financial transactions. It is imperative to address their unique needs and challenges to ensure their financial well-being. Older adults can be more vulnerable to various online threats, making security and privacy paramount. As they adapt to the digital age, understanding their specific privacy concerns and preferences is crucial for creating trustworthy e-payment systems. Moreover, enhancing the usability of e-payment applications for older adults promotes financial independence and inclusion, contributing to their overall quality of life. By focusing on these critical dimensions, we …

Cgt-Gan: Clip-Guided Text Gan For Image Captioning, Jiarui Yu, Haoran Li, Yanbin Hao, Bin Zhu, Tong Xu, Xiangnan He

Research Collection School Of Computing and Information Systems

The large-scale visual-language pre-trained model, Contrastive Language-Image Pre-training (CLIP), has significantly improved image captioning for scenarios without human-annotated image-caption pairs. Recent advanced CLIP-based image captioning without human annotations follows a text-only training paradigm, i.e., reconstructing text from shared embedding space. Nevertheless, these approaches are limited by the training/inference gap or huge storage requirements for text embeddings. Given that it is trivial to obtain images in the real world, we propose CLIP-guided text GAN (CgT-GAN), which incorporates images into the training process to enable the model to "see" real visual modality. Particularly, we use adversarial training to teach CgT-GAN to mimic …

Editanything: Empowering Unparalleled Flexibility In Image Editing And Generation, Shanghua Gao, Zhijie Lin, Xingyu Xie, Pan Zhou, Ming-Ming Cheng, Shuicheng Yan

Research Collection School Of Computing and Information Systems

Image editing plays a vital role in computer vision field, aiming to realistically manipulate images while ensuring seamless integration. It finds numerous applications across various fields. In this work, we present EditAnything, a novel approach that empowers users with unparalleled flexibility in editing and generating image content. EditAnything introduces an array of advanced features, including crossimage dragging (e.g., try-on), region-interactive editing, controllable layout generation, and virtual character replacement. By harnessing these capabilities, users can engage in interactive and flexible editing, giving captivating outcomes that uphold the integrity of the original image. With its diverse range of tools, EditAnything caters to …

Metaformer Baselines For Vision, Weihao Yu, Chenyang Si, Pan Zhou, Mi Luo, Yichen Zhou, Jiashi Feng, Shuicheng Yan, Xinchao Wang

Research Collection School Of Computing and Information Systems

Abstract—MetaFormer, the abstracted architecture of Transformer, has been found to play a significant role in achieving competitive performance. In this paper, we further explore the capacity of MetaFormer, again, by migrating our focus away from the token mixer design: we introduce several baseline models under MetaFormer using the most basic or common mixers, and demonstrate their gratifying performance. We summarize our observations as follows: (1) MetaFormer ensures solid lower bound of performance. By merely adopting identity mapping as the token mixer, the MetaFormer model, termed IdentityFormer, achieves >80% accuracy on ImageNet-1K. (2) MetaFormer works well with arbitrary token mixers. When …

Demo Abstract: Vgglass - Demonstrating Visual Grounding And Localization Synergy With A Lidar-Enabled Smart-Glass, Darshana Rathnayake, Dulanga Weerakoon, Meeralakshmi Radhakrishnan, Vigneshwaran Subbaraju, Inseok Hwang, Archan Misra

Research Collection School Of Computing and Information Systems

This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by the user. Key to our system is the union of LiDAR-based localization termed LiLOC and a multi-modal visual grounding approach termed RealG(2)In-Lite. To demonstrate the system, we use Intel RealSense L515 cameras and a Microsoft HoloLens 2, as the user devices. VGGlass is able to: a) track the user in real-time in a global …

Turn-It-Up: Rendering Resistance For Knobs In Virtual Reality Through Undetectable Pseudo-Haptics, Martin Feick, Andre Zenner, Oscar Ariza, Anthony Tang, Cihan Biyikli, Antonio Kruger

Research Collection School Of Computing and Information Systems

Rendering haptic feedback for interactions with virtual objects is an essential part of effective virtual reality experiences. In this work, we explore providing haptic feedback for rotational manipulations, e.g., through knobs. We propose the use of a Pseudo-Haptic technique alongside a physical proxy knob to simulate various physical resistances. In a psychophysical experiment with 20 participants, we found that designers can introduce unnoticeable offsets between real and virtual rotations of the knob, and we report the corresponding detection thresholds. Based on these, we present the Pseudo-Haptic Resistance technique to convey physical resistance while applying only unnoticeable pseudo-haptic manipulation. Additionally, we …

Constructing Holistic Spatio-Temporal Scene Graph For Video Semantic Role Labeling, Yu Zhao, Hao Fei, Yixin Cao, Bobo Li, Meishan Zhang, Jianguo Wei, Min Zhang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

As one of the core video semantic understanding tasks, Video Semantic Role Labeling (VidSRL) aims to detect the salient events from given videos, by recognizing the predict-argument event structures and the interrelationships between events. While recent endeavors have put forth methods for VidSRL, they can be mostly subject to two key drawbacks, including the lack of fine-grained spatial scene perception and the insufficiently modeling of video temporality. Towards this end, this work explores a novel holistic spatio-temporal scene graph (namely HostSG) representation based on the existing dynamic scene graph structures, which well model both the fine-grained spatial semantics and temporal …

Voxelhap: A Toolkit For Constructing Proxies Providing Tactile And Kinesthetic Haptic Feedback In Virtual Reality, M. Feick, C. Biyikli, K. Gani, A. Wittig, Anthony Tang, A. Krüger

Research Collection School Of Computing and Information Systems

Experiencing virtual environments is often limited to abstract interactions with objects. Physical proxies allow users to feel virtual objects, but are often inaccessible. We present the VoxelHap toolkit which enables users to construct highly functional proxy objects using Voxels and Plates. Voxels are blocks with special functionalities that form the core of each physical proxy. Plates increase a proxy’s haptic resolution, such as its shape, texture or weight. Beyond providing physical capabilities to realize haptic sensations, VoxelHap utilizes VR illusion techniques to expand its haptic resolution. We evaluated the capabilities of the VoxelHap toolkit through the construction of a range …

Npf-200: A Multi-Modal Eye Fixation Dataset And Method For Non-Photorealistic Videos, Ziyu Yang, Sucheng Ren, Zongwei Wu, Nanxuan Zhao, Junle Wang, Jing Qin, Shengfeng He

Research Collection School Of Computing and Information Systems

Non-photorealistic videos are in demand with the wave of the metaverse, but lack of sufficient research studies. This work aims to take a step forward to understand how humans perceive nonphotorealistic videos with eye fixation (i.e., saliency detection), which is critical for enhancing media production, artistic design, and game user experience. To fill in the gap of missing a suitable dataset for this research line, we present NPF-200, the first largescale multi-modal dataset of purely non-photorealistic videos with eye fixations. Our dataset has three characteristics: 1) it contains soundtracks that are essential according to vision and psychological studies; 2) it …

Disentangling Multi-View Representations Beyond Inductive Bias, Guanzhou Ke, Yang Yu, Guoqing Chao, Xiaoli Wang, Chenyang Xu, Shengfeng He

Research Collection School Of Computing and Information Systems

Multi-view (or -modality) representation learning aims to understand the relationships between different view representations. Existing methods disentangle multi-view representations into consistent and view-specific representations by introducing strong inductive biases, which can limit their generalization ability. In this paper, we propose a novel multi-view representation disentangling method that aims to go beyond inductive biases, ensuring both interpretability and generalizability of the resulting representations. Our method is based on the observation that discovering multi-view consistency in advance can determine the disentangling information boundary, leading to a decoupled learning objective. We also found that the consistency can be easily extracted by maximizing the …

Pro-Cap: Leveraging A Frozen Vision-Language Model For Hateful Meme Detection, Rui Cao, Ming Shan Hee, Adriel Kuek, Wen Haw Chong, Roy Ka-Wei Lee, Jing Jiang

Research Collection School Of Computing and Information Systems

Hateful meme detection is a challenging multimodal task that requires comprehension of both vision and language, as well as cross-modal interactions. Recent studies have tried to fine-tune pre-trained vision-language models (PVLMs) for this task. However, with increasing model sizes, it becomes important to leverage powerful PVLMs more efficiently, rather than simply fine-tuning them. Recently, researchers have attempted to convert meme images into textual captions and prompt language models for predictions. This approach has shown good performance but suffers from non-informative image captions. Considering the two factors mentioned above, we propose a probing-based captioning approach to leverage PVLMs in a zero-shot …