Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons

Open Access. Powered by Scholars. Published by Universities.®

1,937 Full-Text Articles 3,051 Authors 590,007 Downloads 154 Institutions

All Articles in Graphics and Human Computer Interfaces

Faceted Search

1,937 full-text articles. Page 1 of 78.

Learning Comprehensive Global Features In Person Re-Identification: Ensuring Discriminativeness Of More Local Regions, Jiali XIA, Jianqiang HUANG, Shibao ZHENG, Qin ZHOU, Bernt SCHIELE, Xian-Sheng HUA, Qianru SUN 2023 Singapore Management University

Learning Comprehensive Global Features In Person Re-Identification: Ensuring Discriminativeness Of More Local Regions, Jiali Xia, Jianqiang Huang, Shibao Zheng, Qin Zhou, Bernt Schiele, Xian-Sheng Hua, Qianru Sun

Research Collection School Of Computing and Information Systems

Person re-identification (Re-ID) aims to retrieve person images from a large gallery given a query image of a person of interest. Global information and fine-grained local features are both essential for the representation. However, global embedding learned by naive classification model tends to be trapped in the most discriminative local region, leading to poor evaluation performance. To address the issue, we propose a novel baseline network that learns strong global feature termed as Comprehensive Global Embedding (CGE), ensuring more local regions of global feature maps to be discriminative. In this work, two key modules are proposed including Non-parameterized Local Classifier …


Hybrid Life Cycles In Software Development, Eric Vincent Schoenborn 2022 Grand Valley State University

Hybrid Life Cycles In Software Development, Eric Vincent Schoenborn

Culminating Experience Projects

This project applied software specification gathering, architecture, work planning, and development to a real-world development effort for a local business. This project began with a feasibility meeting with the owner of Zeal Aerial Fitness. After feasibility was assessed the intended users, needed functionality, and expected user restrictions were identified with the stakeholders. A hybrid software lifecycle was selected to allow a focus on base functionality up front followed by an iterative development of expectations of the stakeholders. I was able to create various specification diagrams that express the end projects goals to both developers and non-tech individuals using a standard …


Augmented Reality Fonts With Enhanced Out-Of-Focus Text Legibility, Mohammed Safayet Arefin 2022 Mississippi State University

Augmented Reality Fonts With Enhanced Out-Of-Focus Text Legibility, Mohammed Safayet Arefin

Theses and Dissertations

In augmented reality, information is often distributed between real and virtual contexts, and often appears at different distances from the viewer. This raises the issues of (1) context switching, when attention is switched between real and virtual contexts, (2) focal distance switching, when the eye accommodates to see information in sharp focus at a new distance, and (3) transient focal blur, when information is seen out of focus, during the time interval of focal distance switching. This dissertation research has quantified the impact of context switching, focal distance switching, and transient focal blur on human performance and eye fatigue in …


Segment-Wise Time-Varying Dynamic Bayesian Network With Graph Regularization, Xing YANG, Chen ZHANG, Baihua ZHENG 2022 Singapore Management University

Segment-Wise Time-Varying Dynamic Bayesian Network With Graph Regularization, Xing Yang, Chen Zhang, Baihua Zheng

Research Collection School Of Computing and Information Systems

Time-varying dynamic Bayesian network (TVDBN) is essential for describing time-evolving directed conditional dependence structures in complex multivariate systems. In this article, we construct a TVDBN model, together with a score-based method for its structure learning. The model adopts a vector autoregressive (VAR) model to describe inter-slice and intra-slice relations between variables. By allowing VAR parameters to change segment-wisely over time, the time-varying dynamics of the network structure can be described. Furthermore, considering some external information can provide additional similarity information of variables. Graph Laplacian is further imposed to regularize similar nodes to have similar network structures. The regularized maximum a …


Vr Computing Lab: An Immersive Classroom For Computing Learning, Huan Shan Shawn PANG, Kyong Jin SHIM, Yi Meng LAU, GOTTIPATI Swapna 2022 Singapore Management University

Vr Computing Lab: An Immersive Classroom For Computing Learning, Huan Shan Shawn Pang, Kyong Jin Shim, Yi Meng Lau, Gottipati Swapna

Research Collection School Of Computing and Information Systems

In recent years, virtual reality (VR) is gaining popularity amongst educators and learners. If a picture is worth a thousand words, a VR session is worth a trillion words. VR technology completely immerses users with an experience that transports them into a simulated world. Universities across the United States, United Kingdom, and other countries have already started using VR for higher education in areas such as medicine, business, architecture, vocational training, social work, virtual field trips, virtual campuses, helping students with special needs, and many more. In this paper, we propose a novel VR platform learning framework which maps elements …


Prompting For Multimodal Hateful Meme Classification, Rui CAO, Roy Ka-Wei LEE, Wen-Haw CHONG, Jing JIANG 2022 Singapore Management University

Prompting For Multimodal Hateful Meme Classification, Rui Cao, Roy Ka-Wei Lee, Wen-Haw Chong, Jing Jiang

Research Collection School Of Computing and Information Systems

Hateful meme classification is a challenging multimodal task that requires complex reasoning and contextual background knowledge. Ideally, we could leverage an explicit external knowledge base to supplement contextual and cultural information in hateful memes. However, there is no known explicit external knowledge base that could provide such hate speech contextual information. To address this gap, we propose PromptHate, a simple yet effective prompt-based model that prompts pre-trained language models (PLMs) for hateful meme classification. Specifically, we construct simple prompts and provide a few in-context examples to exploit the implicit knowledge in the pretrained RoBERTa language model for hateful meme classification. …


Controllable Neural Synthesis For Natural Images And Vector Art, Difan Liu 2022 University of Massachusetts Amherst

Controllable Neural Synthesis For Natural Images And Vector Art, Difan Liu

Doctoral Dissertations

Neural image synthesis approaches have become increasingly popular over the last years due to their ability to generate photorealistic images useful for several applications, such as digital entertainment, mixed reality, synthetic dataset creation, computer art, to name a few. Despite the progress over the last years, current approaches lack two important aspects: (a) they often fail to capture long-range interactions in the image, and as a result, they fail to generate scenes with complex dependencies between their different objects or parts. (b) they often ignore the underlying 3D geometry of the shape/scene in the image, and as a result, they …


Ideating Xai: An Exploration Of User’S Mental Models Of An Ai-Driven Recruitment System Using A Design Thinking Approach, Helen Sheridan, Dympna O'Sullivan Dr., Emma Murphy 2022 Technological University Dublin

Ideating Xai: An Exploration Of User’S Mental Models Of An Ai-Driven Recruitment System Using A Design Thinking Approach, Helen Sheridan, Dympna O'Sullivan Dr., Emma Murphy

Conference Papers

Artificial Intelligence (AI) is playing an important role in society including how vital, often life changing decisions are made. For this reason, interest in Explainable Artificial Intelligence (XAI) has grown in recent years as a means of revealing the processes and operations contained within what is often described as a black box, an often-opaque system whose decisions are difficult to understand by the end user. This paper presents the results of a design thinking workshop with 20 participants (computer science and graphic design students) where we sought to investigate users' mental models when interacting with AI systems. Using two personas, …


Wave-Vit: Unifying Wavelet And Transformers For Visual Representation Learning, Ting YAO, Yingwei PAN, Yehao LI, Chong-wah NGO, Tao MEI 2022 Singapore Management University

Wave-Vit: Unifying Wavelet And Transformers For Visual Representation Learning, Ting Yao, Yingwei Pan, Yehao Li, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Multi-scale Vision Transformer (ViT) has emerged as a powerful backbone for computer vision tasks, while the self-attention computation in Transformer scales quadratically w.r.t. the input patch number. Thus, existing solutions commonly employ down-sampling operations (e.g., average pooling) over keys/values to dramatically reduce the computational cost. In this work, we argue that such over-aggressive down-sampling design is not invertible and inevitably causes information dropping especially for high-frequency components in objects (e.g., texture details). Motivated by the wavelet theory, we construct a new Wavelet Vision Transformer (Wave-ViT) that formulates the invertible down-sampling with wavelet transforms and self-attention learning in a unified way. …


Locally Varying Distance Transform For Unsupervised Visual Anomaly Detection, Wen-yan LIN, Zhonghang LIU, Siying LIU 2022 Singapore Management University

Locally Varying Distance Transform For Unsupervised Visual Anomaly Detection, Wen-Yan Lin, Zhonghang Liu, Siying Liu

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection on image data is notoriously unstable. We believe this is because many classical anomaly detectors implicitly assume data is low dimensional. However, image data is always high dimensional. Images can be projected to a low dimensional embedding but such projections rely on global transformations that truncate minor variations. As anomalies are rare, the final embedding often lacks the key variations needed to distinguish anomalies from normal instances. This paper proposes a new embedding using a set of locally varying data projections, with each projection responsible for persevering the variations that distinguish a local cluster of instances from …


Interactive Contrastive Learning For Self-Supervised Entity Alignment, Kaisheng ZENG, Zhenhao DONG, Lei HOU, Yixin CAO, Minghao HU, Jifan YU, Xin LV, Lei CAO, Xin WANG, Haozhuang LIU, Yi HUANG, Jing WAN, Juanzi LI 2022 Singapore Management University

Interactive Contrastive Learning For Self-Supervised Entity Alignment, Kaisheng Zeng, Zhenhao Dong, Lei Hou, Yixin Cao, Minghao Hu, Jifan Yu, Xin Lv, Lei Cao, Xin Wang, Haozhuang Liu, Yi Huang, Jing Wan, Juanzi Li

Research Collection School Of Computing and Information Systems

Self-supervised entity alignment (EA) aims to link equivalent entities across different knowledge graphs (KGs) without the use of pre-aligned entity pairs. The current state-of-the-art (SOTA) selfsupervised EA approach draws inspiration from contrastive learning, originally designed in computer vision based on instance discrimination and contrastive loss, and suffers from two shortcomings. Firstly, it puts unidirectional emphasis on pushing sampled negative entities far away rather than pulling positively aligned pairs close, as is done in the well-established supervised EA. Secondly, it advocates the minimum information requirement for self-supervised EA, while we argue that self-described KG’s side information (e.g., entity name, relation name, …


Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur RYSKELDIEV, Kotaro HARA, Mariko KOBAYASHI, Koki KUSANO 2022 University of Tsukuba

Investigating Accessibility Challenges And Opportunities For Users With Low Vision Disabilities In Customer-To-Customer (C2c) Marketplaces, Bektur Ryskeldiev, Kotaro Hara, Mariko Kobayashi, Koki Kusano

Research Collection School Of Computing and Information Systems

Inaccessible e-commerce websites and mobile applications exclude people with visual impairments (PVI) from online shopping. Customer-to-customer (C2C) marketplaces, a form of e-commerce where trading happens not between businesses and customers but between customers, could pose a unique set of challenges in the interactions that the platform brings about. Through online questionnaire and remote interviews, we investigate problems experienced by people with low vision disabilities in common C2C scenarios. Our study with low vision participants (N = 12) reveal both previously known general accessibility issues (e.g., web and mobile interface accessibility) and C2C specific accessibility issues (e.g., inability to confirm item …


Dynamic Temporal Filtering In Video Models, Fuchen LONG, Zhaofan QIU, Yingwei PAN, Ting YAO, Chong-wah NGO, Tao MEI 2022 Singapore Management University

Dynamic Temporal Filtering In Video Models, Fuchen Long, Zhaofan Qiu, Yingwei Pan, Ting Yao, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Video temporal dynamics is conventionally modeled with 3D spatial-temporal kernel or its factorized version comprised of 2D spatial kernel and 1D temporal kernel. The modeling power, nevertheless, is limited by the fixed window size and static weights of a kernel along the temporal dimension. The pre-determined kernel size severely limits the temporal receptive fields and the fixed weights treat each spatial location across frames equally, resulting in sub-optimal solution for longrange temporal modeling in natural scenes. In this paper, we present a new recipe of temporal feature learning, namely Dynamic Temporal Filter (DTF), that novelly performs spatial-aware temporal modeling in …


On Mitigating Hard Clusters For Face Clustering, Yingjie CHEN, Huasong ZHONG, Chong CHEN, Chen SHEN, Jianqiang HUANG, Tao WANG, Yun LIANG, Qianru SUN 2022 Singapore Management University

On Mitigating Hard Clusters For Face Clustering, Yingjie Chen, Huasong Zhong, Chong Chen, Chen Shen, Jianqiang Huang, Tao Wang, Yun Liang, Qianru Sun

Research Collection School Of Computing and Information Systems

Face clustering is a promising way to scale up face recognition systems using large-scale unlabeled face images. It remains challenging to identify small or sparse face image clusters that we call hard clusters, which is caused by the heterogeneity, i.e., high variations in size and sparsity, of the clusters. Consequently, the conventional way of using a uniform threshold (to identify clusters) often leads to a terrible misclassification for the samples that should belong to hard clusters. We tackle this problem by leveraging the neighborhood information of samples and inferring the cluster memberships (of samples) in a probabilistic way. We introduce …


Equivariance And Invariance Inductive Bias For Learning From Insufficient Data, Tan WANG, Qianru SUN, Sugiri PRANATA, Karlekar JAYASHREE, Hanwang ZHANG 2022 Singapore Management University

Equivariance And Invariance Inductive Bias For Learning From Insufficient Data, Tan Wang, Qianru Sun, Sugiri Pranata, Karlekar Jayashree, Hanwang Zhang

Research Collection School Of Computing and Information Systems

We are interested in learning robust models from insufficient data, without the need for any externally pre-trained model checkpoints. First, compared to sufficient data, we show why insufficient data renders the model more easily biased to the limited training environments that are usually different from testing. For example, if all the training "swan" samples are "white", the model may wrongly use the "white" environment to represent the intrinsic class "swan". Then, we justify that equivariance inductive bias can retain the class feature while invariance inductive bias can remove the environmental feature, leaving only the class feature that generalizes to any …


Ergo: Event Relational Graph Transformer For Document-Level Event Causality Identification, Meiqi CHEN, Yixin CAO, Kunquan DENG, Mukai LI, Kun WANG, Jing SHAO, Yan ZHANG 2022 Singapore Management University

Ergo: Event Relational Graph Transformer For Document-Level Event Causality Identification, Meiqi Chen, Yixin Cao, Kunquan Deng, Mukai Li, Kun Wang, Jing Shao, Yan Zhang

Research Collection School Of Computing and Information Systems

Document-level Event Causality Identification (DECI) aims to identify event-event causal relations in a document. Existing works usually build an event graph for global reasoning across multiple sentences. However, the edges between events have to be carefully designed through heuristic rules or external tools. In this paper, we propose a novel Event Relational Graph TransfOrmer (ERGO) framework1 for DECI, to ease the graph construction and improve it over the noisy edge issue. Different from conventional event graphs, we define a pair of events as a node and build a complete event relational graph without any prior knowledge or tools. This naturally …


Tgdm: Target Guided Dynamic Mixup For Cross-Domain Few-Shot Learning, Linhai ZHUO, Yuqian FU, Jingjing CHEN, Yixin CAO, Yu-Gang JIANG 2022 Singapore Management University

Tgdm: Target Guided Dynamic Mixup For Cross-Domain Few-Shot Learning, Linhai Zhuo, Yuqian Fu, Jingjing Chen, Yixin Cao, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Given sufficient training data on the source domain, cross-domain few-shot learning (CD-FSL) aims at recognizing new classes with a small number of labeled examples on the target domain. The key to addressing CD-FSL is to narrow the domain gap and transferring knowledge of a network trained on the source domain to the target domain. To help knowledge transfer, this paper introduces an intermediate domain generated by mixing images in the source and the target domain. Specifically, to generate the optimal intermediate domain for different target data, we propose a novel target guided dynamic mixup (TGDM) framework that leverages the target …


Interactive Video Corpus Moment Retrieval Using Reinforcement Learning, Zhixin MA, Chong-wah NGO 2022 Singapore Management University

Interactive Video Corpus Moment Retrieval Using Reinforcement Learning, Zhixin Ma, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Known-item video search is effective with human-in-the-loop to interactively investigate the search result and refine the initial query. Nevertheless, when the first few pages of results are swamped with visually similar items, or the search target is hidden deep in the ranked list, finding the know-item target usually requires a long duration of browsing and result inspection. This paper tackles the problem by reinforcement learning, aiming to reach a search target within a few rounds of interaction by long-term learning from user feedbacks. Specifically, the system interactively plans for navigation path based on feedback and recommends a potential target that …


Long-Term Leap Attention, Short-Term Periodic Shift For Video Classification, Hao ZHANG, Lechao CHENG, Yanbin HAO, Chong-wah NGO 2022 Singapore Management University

Long-Term Leap Attention, Short-Term Periodic Shift For Video Classification, Hao Zhang, Lechao Cheng, Yanbin Hao, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Video transformer naturally incurs a heavier computation burden than a static vision transformer, as the former processes �� times longer sequence than the latter under the current attention of quadratic complexity (�� 2�� 2 ). The existing works treat the temporal axis as a simple extension of spatial axes, focusing on shortening the spatio-temporal sequence by either generic pooling or local windowing without utilizing temporal redundancy. However, videos naturally contain redundant information between neighboring frames; thereby, we could potentially suppress attention on visually similar frames in a dilated manner. Based on this hypothesis, we propose the LAPS, a long-term “Leap …


Class Is Invariant To Context And Vice Versa: On Learning Invariance For Out-Of-Distribution Generalization, Jiaxin QI, Kaihua TANG, Qianru SUN, Xian-Sheng HUA, Hanwang ZHANG 2022 Singapore Management University

Class Is Invariant To Context And Vice Versa: On Learning Invariance For Out-Of-Distribution Generalization, Jiaxin Qi, Kaihua Tang, Qianru Sun, Xian-Sheng Hua, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Out-Of-Distribution generalization (OOD) is all about learning invariance against environmental changes. If the context in every class is evenly distributed, OOD would be trivial because the context can be easily removed due to an underlying principle: class is invariant to context. However, collecting such a balanced dataset is impractical. Learning on imbalanced data makes the model bias to context and thus hurts OOD. Therefore, the key to OOD is context balance.We argue that the widely adopted assumption in prior work—the context bias can be directly annotated or estimated from biased class prediction—renders the context incomplete or even incorrect. In contrast, …


Digital Commons powered by bepress