Graphics and Human Computer Interfaces Commons^™

Scanet: Self-Paced Semi-Curricular Attention Network For Non-Homogeneous Image Dehazing, Yu Guo, Yuan Gao, Ryan Wen Liu, Yuxu Lu, Jingxiang Qu, Shengfeng He, Ren Wenqi

Research Collection School Of Computing and Information Systems

The presence of non-homogeneous haze can cause scene blurring, color distortion, low contrast, and other degradations that obscure texture details. Existing homogeneous dehazing methods struggle to handle the non-uniform distribution of haze in a robust manner. The crucial challenge of non-homogeneous dehazing is to effectively extract the non-uniform distribution features and reconstruct the details of hazy areas with high quality. In this paper, we propose a novel self-paced semi-curricular attention network, called SCANet, for non-homogeneous image dehazing that focuses on enhancing haze-occluded regions. Our approach consists of an attention generator network and a scene re-construction network. We use the luminance …

Towards A Smaller Student: Capacity Dynamic Distillation For Efficient Image Retrieval, Yi Xie, Huaidong Zhang, Xuemiao Xu, Jianqing Zhu, Shengfeng He

Research Collection School Of Computing and Information Systems

Previous Knowledge Distillation based efficient image retrieval methods employ a lightweight network as the student model for fast inference. However, the lightweight student model lacks adequate representation capacity for effective knowledge imitation during the most critical early training period, causing final performance degeneration. To tackle this issue, we propose a Capacity Dynamic Distillation framework, which constructs a student model with editable representation capacity. Specifically, the employed student model is initially a heavy model to fruitfully learn distilled knowledge in the early training epochs, and the student model is gradually compressed during the training. To dynamically adjust the model capacity, our …

Position-Guided Text Prompt For Vision-Language Pre-Training, Alex Jinpeng Wang, Pan Zhou, Mike Zheng Shou, Yan Shuicheng

Research Collection School Of Computing and Information Systems

Vision-Language Pre-Training (VLP) has shown promising capabilities to align image and text pairs, facilitating a broad variety of cross-modal learning tasks. However, we observe that VLP models often lack the visual grounding/localization capability which is critical for many downstream tasks such as visual reasoning. In this work, we propose a novel Position-guided Text Prompt (PTP) paradigm to enhance the visual grounding ability of cross-modal models trained with VLP. Specifically, in the VLP phase, PTP divides the image into N x N blocks, and identifies the objects in each block through the widely used object detector in VLP. It then reformulates …

Class-Incremental Exemplar Compression For Class-Incremental Learning, Zilin Luo, Yaoyao Liu, Bernt Schiele, Qianru Sun

Research Collection School Of Computing and Information Systems

Exemplar-based class-incremental learning (CIL) finetunes the model with all samples of new classes but few-shot exemplars of old classes in each incremental phase, where the "few-shot" abides by the limited memory budget. In this paper, we break this "few-shot" limit based on a simple yet surprisingly effective idea: compressing exemplars by downsampling non-discriminative pixels and saving "many-shot" compressed exemplars in the memory. Without needing any manual annotation, we achieve this compression by generating 0-1 masks on discriminative pixels from class activation maps (CAM). We propose an adaptive mask generation model called class-incremental masking (CIM) to explicitly resolve two difficulties of …

Freestyle Layout-To-Image Synthesis, Han Xue, Zhiwu Huang, Qianru Sun, Li Song, Wenjun Zhang

Research Collection School Of Computing and Information Systems

Typical layout-to-image synthesis (LIS) models generate images for a close set of semantic classes, e.g., 182 common objects in COCO-Stuff. In this work, we explore the freestyle capability of the model, i.e., how far can it generate unseen semantics (e.g., classes, attributes, and styles) onto a given layout, and call the task Freestyle LIS (FLIS). Thanks to the development of large-scale pre-trained language-image models, a number of discriminative models (e.g., image classification and object detection) trained on limited base classes are empowered with the ability of unseen class prediction. Inspired by this, we opt to leverage large-scale pre-trained text-to-image diffusion …

Curricular Contrastive Regularization For Physics-Aware Single Image Dehazing, Yu Zheng, Jiahui Zhan, Shengfeng He, Yong Du

Research Collection School Of Computing and Information Systems

Considering the ill-posed nature, contrastive regularization has been developed for single image dehazing, introducing the information from negative images as a lower bound. However, the contrastive samples are non-consensual, as the negatives are usually represented distantly from the clear (i.e., positive) image, leaving the solution space still under-constricted. Moreover, the interpretability of deep dehazing models is underexplored towards the physics of the hazing process. In this paper, we propose a novel curricular contrastive regularization targeted at a consensual contrastive space as opposed to a non-consensual one. Our negatives, which provide better lower-bound constraints, can be assembled from 1) the hazy …

Venus: A Geometrical Representation For Quantum State Visualization, Shaolun Ruan, Ribo Yuan, Qiang Guan, Yanna Lin, Ying Mao, Weiwen Jiang, Zhepeng Wang, Wei Xu, Yong Wang

Research Collection School Of Computing and Information Systems

Visualizations have played a crucial role in helping quantum computing users explore quantum states in various quantum computing applications. Among them, Bloch Sphere is the widely-used visualization for showing quantum states, which leverages angles to represent quantum amplitudes. However, it cannot support the visualization of quantum entanglement and superposition, the two essential properties of quantum computing. To address this issue, we propose VENUS, a novel visualization for quantum state representation. By explicitly correlating 2D geometric shapes based on the math foundation of quantum computing characteristics, VENUS effectively represents quantum amplitudes of both the single qubit and two qubits for quantum …

Ifundit: Visual Profiling Of Fund Investment Styles, Rong Zhang, Bon Kyung Ku, Yong Wang, Xuanwu Yue, Siyuan Liu, Ke Li, Huamin Qu

Research Collection School Of Computing and Information Systems

Mutual funds are becoming increasingly popular with the emergence of Internet finance. Clear profiling of a fund's investment style is crucial for fund managers to evaluate their investment strategies, and for investors to understand their investment. However, it is challenging to profile a fund's investment style as it requires a comprehensive analysis of complex multi-dimensional temporal data. In addition, different fund managers and investors have different focuses when analysing a fund's investment style. To address the issue, we propose iFUNDit, an interactive visual analytic system for fund investment style analysis. The system decomposes a fund's critical features into performance attributes …

Gnnlens: A Visual Analytics Approach For Prediction Error Diagnosis Of Graph Neural Networks., Zhihua Jin, Yong Wang, Qianwen Wang, Yao Ming, Tengfei Ma, Huamin Qu

Research Collection School Of Computing and Information Systems

Graph Neural Networks (GNNs) aim to extend deep learning techniques to graph data and have achieved significant progress in graph analysis tasks (e.g., node classification) in recent years. However, similar to other deep neural networks like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), GNNs behave like a black box with their details hidden from model developers and users. It is therefore difficult to diagnose possible errors of GNNs. Despite many visual analytics studies being done on CNNs and RNNs, little research has addressed the challenges for GNNs. This paper fills the research gap with an interactive visual analysis …

Extracting Class Activation Maps From Non-Discriminative Features As Well, Zhaozheng Chen, Qianru Sun

Research Collection School Of Computing and Information Systems

Extracting class activation maps (CAM) from a classification model often results in poor coverage on foreground objects, i.e., only the discriminative region (e.g., the “head” of “sheep”) is recognized and the rest (e.g., the “leg” of “sheep”) mistakenly as background. The crux behind is that the weight of the classifier (used to compute CAM) captures only the discriminative features of objects. We tackle this by introducing a new computation method for CAM that explicitly captures non-discriminative features as well, thereby expanding CAM to cover whole objects. Specifically, we omit the last pooling layer of the classification model, and perform clustering …

Semantic Scene Completion With Cleaner Self, Fengyun Wang, Dong Zhang, Hanwang Zhang, Jinhui Tang, Qianru Sun

Research Collection School Of Computing and Information Systems

Semantic Scene Completion (SSC) transforms an image of single-view depth and/or RGB 2D pixels into 3D voxels, each of whose semantic labels are predicted. SSC is a well-known ill-posed problem as the prediction model has to “imagine” what is behind the visible surface, which is usually represented by Truncated Signed Distance Function (TSDF). Due to the sensory imperfection of the depth camera, most existing methods based on the noisy TSDF estimated from depth values suffer from 1) incomplete volumetric predictions and 2) confused semantic labels. To this end, we use the ground-truth 3D voxels to generate a perfect visible surface, …

Unbiased Multiple Instance Learning For Weakly Supervised Video Anomaly Detection, Hui Lyu, Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Weakly Supervised Video Anomaly Detection (WSVAD) is challenging because the binary anomaly label is only given on the video level, but the output requires snippet-level predictions. So, Multiple Instance Learning (MIL) is prevailing in WSVAD. However, MIL is notoriously known to suffer from many false alarms because the snippet-level detector is easily biased towards the abnormal snippets with simple context, confused by the normality with the same bias, and missing the anomaly with a different pattern. To this end, we propose a new MIL framework: Unbiased MIL (UMIL), to learn unbiased anomaly features that improve WSVAD. At each MIL training …

Evading Deepfake Detectors Via Adversarial Statistical Consistency, Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Jianjun Zhao

Research Collection School Of Computing and Information Systems

In recent years, as various realistic face forgery techniques known as DeepFake improves by leaps and bounds, more and more DeepFake detection techniques have been proposed. These methods typically rely on detecting statistical differences between natural (i.e., real) and DeepFake-generated images in both spatial and frequency domains. In this work, we propose to explicitly minimize the statistical differences to evade state-of-the-art DeepFake detectors. To this end, we propose a statistical consistency attack (StatAttack) against DeepFake detectors, which contains two main parts. First, we select several statistical-sensitive natural degradations (i.e., exposure, blur, and noise) and add them to the fake images …

Human And Technical Factors In The Adoption Of Quantum Cryptographic Algorithms, Alyssa Pinkston

Mathematical Sciences Technical Reports (MSTR)

The purpose of this research is to understand what factors would cause users to choose quantum key distribution (QKD) over other methods of cryptography. An Advanced Encryption Standard (AES) key can be exchanged through communication using the Rivest, Shamir, Adleman (RSA) cryptographic algorithm, QKD, or post-quantum cryptography (PQC). QKD relies on quantum physics where RSA and PQC use complex mathematics to encrypt data. The BB84 quantum cryptographic protocol involves communication over a quantum channel and a public channel. The quantum channel can be technically attacked by beamsplitting or intercept/resend. QKD, like other forms of cryptography, is vulnerable to social attacks …

An Assistive Interface For Displaying Novice's Code History, Ruiwei Xiao

McKelvey School of Engineering Graduate Student Theses & Dissertations

As Teaching Assistant (TA) programs grow in number and size in introductory CS courses, TAs play a significant role in novice programmers' experience and contribute to their success. However, many TAs are also relative beginners themselves and thus have limited experience in programming and teaching. Thus the effectiveness and consistency of their guidance can vary significantly. To improve interaction quality and assist TAs in providing better support, we examine the difficulties encountered by inexperienced TAs in previous literature and then identify the potential for the high cognitive load as an unaddressed difficulty that may prevent new TAs from initiating effective …

Expressive Marks: Art In The Age Of Augmented Reality, Carson G. Levine

Dartmouth College Master’s Theses

Augmented reality (AR) and non-fungible tokens (NFTs) introduce new considerations for the long-standing debate of what it means for digital art to be “real.” However, the ability to create AR experiences is limited to those who are technically skilled or who can afford to consult someone else. This paper addresses the need for an accessible tool that enables artists of all technical backgrounds to expressively create marks in AR. The solution includes a mobile application called CrayonAR. The system was designed to be modular, minimal, and physically engaging, and was developed in Unity using ARFoundation and Firebase Storage and Realtime …

Visualized Algorithm Engineering On Two Graph Partitioning Problems, Zizhen Chen

Computer Science and Engineering Theses and Dissertations

Concepts of graph theory are frequently used by computer scientists as abstractions when modeling a problem. Partitioning a graph (or a network) into smaller parts is one of the fundamental algorithmic operations that plays a key role in classifying and clustering. Since the early 1970s, graph partitioning rapidly expanded for applications in wide areas. It applies in both engineering applications, as well as research. Current technology generates massive data (“Big Data”) from business interactions and social exchanges, so high-performance algorithms of partitioning graphs are a critical need.

This dissertation presents engineering models for two graph partitioning problems arising from completely …

Beyond Algorithms: A User-Centered Evaluation Of A Feature Recommender System In Requirements Engineering, Oluwatobi Lasisi

Theses and Dissertations

Several studies have applied recommender technologies to support requirements engineering activities. As in other application areas of recommender systems (RS), many studies have focused on the algorithms’ prediction accuracy, while there have been limited discussions around users’ interactions with the systems. Since recommender systems are designed to aid users in information retrieval, they should be assessed not just as recommendation algorithms but also from the users’ perspective. In contrast to accuracy measures, user-related issues can only be effectively investigated via empirical studies involving real users. Furthermore, researchers are becoming increasingly aware that the effectiveness of the systems goes beyond recommendation …

Augmented & Virtual Reality: Advancement Of Technology And Its Impacts On Medicine, Education, And Other Industries, Yassine Chahid

Publications and Research

Throughout the early 2000s, the ways in which the World Wide Web was used would undergo major changes. The introduction of these changes around this time period would be collectively known as Web 2.0. With Web 2.0, accessibility and distribution of applications became more simplified. During the 2000s, much has evolved from hard capabilities to the internet and its widespread usage amongst companies and general consumers. In contemporary times, multiple technologies, both hardware and digital are becoming more advanced, with general consumers either rejecting or accepting these gradual shifts in what may become everyday technology. Web 3.0, the theoretical advancement …

Developing A Multi-Platform Application To Facilitate Internal Campus Hiring, Carissa Patton

Computer Science and Computer Engineering Undergraduate Honors Theses

Undergraduate research has proven to be highly beneficial to students, yet there are many students who do not know how to get involved or who are too timid to approach professors to inquire about potential research opportunities. Our hypothesis is that a cross-platform application has the potential to bridge the gap and help more students get involved in undergraduate research by providing them information about open positions and the faculty or staff members who are mentoring the projects. The key focus of this thesis is to develop an application that provides details about participating faculty or staff including their research …

Culture In Computing: The Importance Of Developing Gender-Inclusive Software, Creighton France

Computer Science and Computer Engineering Undergraduate Honors Theses

The field of computing as we know it today exists because of the contributions of numerous female mathematicians, computer scientists, and programmers. While working with hardware was viewed as “a man’s job” during the mid-20th century, computing and programming was viewed as a noble and high-paying field for women to occupy. However, as time has progressed, the U.S. has seen a decrease in the number of women pursuing computer science. The idea that computing is a masculine discipline is common in the U.S. today for reasons such as male-centered marketing of electronics and gadgets, an inaccurate representation of what it …

Document Graph Representation Learning, Ce Zhang

Dissertations and Theses Collection (Open Access)

Much of the data on the Web can be represented in a graph structure, ranging from social and biological to academic and Web page graphs, etc. Graph analysis recently attracts escalating research attention due to its importance and wide applicability. Diverse problems could be formulated as graph tasks, such as text classification and information retrieval. As the primary information is the inherent structure of the graph itself, one promising direction known as the graph representation learning problem is to learn the representation of each node, which could in turn fuel tasks such as node classification, node clustering, and link prediction. …

Sprout: Using A Garden Metaphor To Visualize And Support Customizable And Collaborative Health Tracking, Pape Sow Traoré

Dartmouth College Master’s Theses

Self-tracking tools have become increasingly popular, especially with the advent of wearable technology and smartphone applications. However, traditional tracking tools often display data in a quantitative format that can be overwhelming and cause users to abandon their tracking efforts. Additionally, these tools typically provide a generic user experience and are designed from a single-user perspective, lacking external support. To overcome these limitations, we develop Sprout, a mobile data-tracking application that offers a more qualitative, customizable, and collaborative experience for health monitoring and management. Sprout uses a garden metaphor to visually represent health information and allows users to tailor their …

Using Immersive Technology To Improve Mechanical Design: Use Cases, A Review Of Current Technology, And An Experiment In Requirement Elicitation In Virtual Reality, William Hawthorne

All Theses

There has been a growing trend in the use of immersive technologies within engineering design. This research is focused on understanding how Virtual Reality (VR) technologies support design reviews. Modern tools including virtual reality hardware and software were tested for current capabilities and challenges. In a review of literature, two key gaps are identified: the rapid advancements of VR and AR technology and limited formal studies to determine the costs and benefits of immersive technology within engineering design reviews. This research has resulted in three key outcomes. First, use cases of immersive reality technologies are identified, through the lens of …

Artificial: A Study On The Use Of Artificial Intelligence In Art, Hayden Ernst

Theses/Capstones/Creative Projects

In the past three to five years there have been significant improvements made in AI due to improvements in computing capacity, the collection and use of big data, and an increase in public interest and funding for research. Programs such as ChatGPT, DALL•E, and Midjourney have also gained tremendous popularity in a relatively short amount of time. This led me to this project in which I aimed to gain a deeper understanding of these art generator AI and where they fit into art as a whole. My goal was to give recommendations to museums and exhibits in Omaha on what …

Iot Health Devices: Exploring Security Risks In The Connected Landscape, Abasi-Amefon Obot Affia, Hilary Finch, Woosub Jung, Issah Abubakari Samori, Lucas Potter, Xavier-Lewis Palmer

School of Cybersecurity Faculty Publications

The concept of the Internet of Things (IoT) spans decades, and the same can be said for its inclusion in healthcare. The IoT is an attractive target in medicine; it offers considerable potential in expanding care. However, the application of the IoT in healthcare is fraught with an array of challenges, and also, through it, numerous vulnerabilities that translate to wider attack surfaces and deeper degrees of damage possible to both consumers and their confidence within health systems, as a result of patient-specific data being available to access. Further, when IoT health devices (IoTHDs) are developed, a diverse range of …

Lpt: Long-Tailed Prompt Tuning For Image Classification, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo

Research Collection School Of Computing and Information Systems

For long-tailed classification tasks, most works often pretrain a big model on a large-scale (unlabeled) dataset, and then fine-tune the whole pretrained model for adapting to long-tailed data. Though promising, fine-tuning the whole pretrained model tends to suffer from high cost in computation and deployment of different models for different tasks, as well as weakened generalization capability for overfitting to certain features of long-tailed data. To alleviate these issues, we propose an effective Long-tailed Prompt Tuning (LPT) method for long-tailed classification tasks. LPT introduces several trainable prompts into a frozen pretrained model to adapt it to long-tailed data. For better …

Liloc: Enabling Precise 3d Localization In Dynamic Indoor Environments Using Lidars, Darshana Rathnayake, Meera Radhakrishnan, Inseok Hwang, Archan Misra

Research Collection School Of Computing and Information Systems

We present LiLoc, a system for precise 3D localization and tracking of mobile IoT devices (e.g., robots) in indoor environments using multi-perspective LiDAR sensing. The key differentiators in our work are: (a) First, unlike traditional localization approaches, our approach is robust to dynamically changing environmental conditions (e.g., varying crowd levels, object placement/layout changes); (b) Second, unlike prior work on visual and 3D SLAM, LiLoc is not dependent on a pre-built static map of the environment and instead works by utilizing dynamically updated point clouds captured from both infrastructural-mounted LiDARs and LiDARs equipped on individual mobile IoT devices. To achieve fine-grained, …

Compositional Prompt Tuning With Motion Cues For Open-Vocabulary Video Relation Detection, Kaifeng Gao, Long Chen, Hanwang Zhang, Jun Xiao, Qianru Sun

Research Collection School Of Computing and Information Systems

Prompt tuning with large-scale pretrained vision-language models empowers open-vocabulary prediction trained on limited base categories, e.g., object classification and detection. In this paper, we propose compositional prompt tuning with motion cues: an extended prompt tuning paradigm for compositional predictions of video data. In particular, we present Relation Prompt (RePro) for Open-vocabulary Video Visual Relation Detection (Open-VidVRD), where conventional prompt tuning is easily biased to certain subject-object combinations and motion patterns. To this end, RePro addresses the two technical challenges of Open-VidVRD: 1) the prompt tokens should respect the two different semantic roles of subject and object, and 2) the tuning …

Procedural City Generation With Combined Architectures For Real-Time Visualization, Griffin Poyck

All Theses

The work and research of this paper sought to build upon traditional city generation and simulation in creating a tool that both realistically simulates cities and their prominent features and also creates aesthetic and artistically rich cities using assets that combine several contemporary or near contemporary architectural styles. The major city features simulated are the surrounding terrain, road networks, individual buildings, and building placement. The tools used to both create and integrate these features were created in Houdini with Unreal Engine 5 as the intended final destination. This research was influenced by the city, town, and road networking of Ghost …