Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons™

Open Access. Powered by Scholars. Published by Universities.®

Discipline
Institution
Keyword
Publication Year
Publication
Publication Type
File Type

Articles 871 - 900 of 2362

Full-Text Articles in Graphics and Human Computer Interfaces

Evaluation Of Gpu Acceleration For Wrf–Sfire, Joshua Benz Dec 2021

Evaluation Of Gpu Acceleration For Wrf–Sfire, Joshua Benz

Master's Projects

WRF–SFIRE is an open source, atmospheric–wildfire model that couples the WRF model with the level set fire spread model to simulate wildfires in real time. This model has many applications and more scientific questions can be asked and answered if the model can be run faster. Nvidia has put a lot of effort into easing the barrier of entry for accelerating applications with their tools to be run on GPUs. Various physical simulations have been successfully ported to utilize GPUs and have benefited from the speed increase. In this research, we take a look at WRF-SFIRE and try to use …

Go to article

Winter 2021 Dec 2021

Winter 2021

In The Loop

2021 Emmy Nominees; Animator Tapped by Cartoon Network; IndieCade Horizons 2021; Hack4Space; Security Daemons Prevail; Role Models: DePaul Originals Game Studio students build industry-level skills that benefit themselves and others; Frames and Fortune: Eugene Bush programmed his indie video studio with patience and planning; Reality Check: Heather Snyder Quinn augments reality to question systems of unchecked power

Go to article

Comparative Analysis Of Rgb-Based Eye-Tracking For Large-Scale Human-Machine Applications, Brett Thaman, Trung Cao Dec 2021

Comparative Analysis Of Rgb-Based Eye-Tracking For Large-Scale Human-Machine Applications, Brett Thaman, Trung Cao

Posters-at-the-Capitol

Gaze tracking has become an established technology that enables using an individual’s gaze as an input signal to support a variety of applications in the context of Human-Computer Interaction. Gaze tracking primarily relies on sensing devices such as infrared (IR) cameras. Nevertheless, in the recent years, several attempts have been realized at detecting gaze by acquiring and processing images acquired from standard RGB cameras. Nowadays, there are only a few publicly available open-source libraries and they have not been tested extensively. In this paper, we present the result of a comparative analysis that studied a commercial eye-tracking device using IR …

Go to article

Markdown To Question & Test Interoperability, Su Kim Dec 2021

Markdown To Question & Test Interoperability, Su Kim

Master's Projects

As the classroom setting shifted to a virtual one as a result of Covid-19, numerous software are readily available to accommodate for the change, including Canvas, the online course management system. Canvas has a core feature that allows teachers to generate and administer quizzes for students through their interface, but it does not fully utilize the potential with online exams. The first step to exploring this potential is this project, known as Markdown to Question & Test Interoperability (M2QTI). Based on the QTI specifications, this tool lets users to plan and write quizzes in Markdown format. Combined with Canvas’s ability …

Go to article

Automated Discovery And Interpretation Of Ada-Compliant Door Placards, John J. Feilmeier Dec 2021

Automated Discovery And Interpretation Of Ada-Compliant Door Placards, John J. Feilmeier

Computer Science and Information Technology Faculty

A familiar difficulty to any new student on campus is making one’s way from classroom A to classroom B. Facilities with different wings, multiple floors, and irregular floorplans can magnify this challenge, while students with vision impairments are impacted even more by the challenge of identifying the destination. This thesis explored different methods of discovering Americans with Disabilities Act (ADA)- compliant room identifying placards (“plaques”) and identifying the text on the sign. The plaque detection was accomplished with both standard image manipulation techniques and a Histogram of Oriented Gradients (HOG) (Dalal & Triggs, 2005) object detector. The text reading utilized …

Go to article

Contrastive Learning For Unsupervised Auditory Texture Models, Christina Trexler Dec 2021

Contrastive Learning For Unsupervised Auditory Texture Models, Christina Trexler

Computer Science and Computer Engineering Undergraduate Honors Theses

Sounds with a high level of stationarity, also known as sound textures, have perceptually relevant features which can be captured by stimulus-computable models. This makes texture-like sounds, such as those made by rain, wind, and fire, an appealing test case for understanding the underlying mechanisms of auditory recognition. Previous auditory texture models typically measured statistics from auditory filter bank representations, and the statistics they used were somewhat ad-hoc, hand-engineered through a process of trial and error. Here, we investigate whether a better auditory texture representation can be obtained via contrastive learning, taking advantage of the stationarity of auditory textures to …

Go to article

Component Damage Source Identification For Critical Infrastructure Systems, Nathan Davis Dec 2021

Component Damage Source Identification For Critical Infrastructure Systems, Nathan Davis

Graduate Theses and Dissertations

Cyber-Physical Systems (CPS) are becoming increasingly prevalent for both Critical Infrastructure and the Industry 4.0 initiative. Bad values within components of the software portion of CPS, or the computer systems, have the potential to cause major damage if left unchecked, and so detection and locating of where these occur is vital. We further define features of these computer systems and create a use-based system topology. We then introduce a function to monitor system integrity and the presence of bad values as well as an algorithm to locate them. We then show an improved version, taking advantage of several system properties …

Go to article

Self-Supervised Learning Disentangled Group Representation As Feature, Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, Hanwang Zhang Dec 2021

Self-Supervised Learning Disentangled Group Representation As Feature, Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, Hanwang Zhang

Research Collection School Of Computing and Information Systems

A good visual representation is an inference map from observations (images) to features (vectors) that faithfully reflects the hidden modularized generative factors (semantics). In this paper, we formulate the notion of “good” representation from a group-theoretic view using Higgins’ definition of disentangled representation [38], and show that existing Self-Supervised Learning (SSL) only disentangles simple augmentation features such as rotation and colorization, thus unable to modularize the remaining semantics. To break the limitation, we propose an iterative SSL algorithm: Iterative Partition-based Invariant Risk Minimization (IP-IRM), which successfully grounds the abstract semantics and the group acting on them into concrete contrastive learning. …

Go to article

A Theory-Driven Self-Labeling Refinement Method For Contrastive Representation Learning, Pan Zhou, Caiming Xiong, Xiao-Tong Yuan Dec 2021

A Theory-Driven Self-Labeling Refinement Method For Contrastive Representation Learning, Pan Zhou, Caiming Xiong, Xiao-Tong Yuan

Research Collection School Of Computing and Information Systems

For an image query, unsupervised contrastive learning labels crops of the same image as positives, and other image crops as negatives. Although intuitive, such a native label assignment strategy cannot reveal the underlying semantic similarity between a query and its positives and negatives, and impairs performance, since some negatives are semantically similar to the query or even share the same semantic class as the query. In this work, we first prove that for contrastive learning, inaccurate label assignment heavily impairs its generalization for semantic instance discrimination, while accurate labels benefit its generalization. Inspired by this theory, we propose a novel …

Go to article

Acceleration Skinning: Kinematics-Driven Cartoon Effects For Articulated Characters, Niranjan Kalyanasundaram Dec 2021

Acceleration Skinning: Kinematics-Driven Cartoon Effects For Articulated Characters, Niranjan Kalyanasundaram

All Theses

Secondary effects are key to adding fluidity and style to animation. This thesis introduces the idea of “Acceleration Skinning” following a recent well-received technique, Velocity Skinning, to automatically create secondary motion in character animation by modifying the standard pipeline for skeletal rig skinning. These effects, which animators may refer to as squash and stretch or drag, attempt to create an illusion of inertia. In this thesis, I extend the Velocity Skinning technique to include acceleration for creating a wider gamut of cartoon effects. I explore three new deformers that make use of this Acceleration Skinning framework: followthrough, centripetal stretch, and …

Go to article

Fine-Grained Generalization Analysis Of Inductive Matrix Completion, Antoine Ledent, Rodrigo Alves, Yunwen Lei, Marius Kloft Dec 2021

Fine-Grained Generalization Analysis Of Inductive Matrix Completion, Antoine Ledent, Rodrigo Alves, Yunwen Lei, Marius Kloft

Research Collection School Of Computing and Information Systems

In this paper, we bridge the gap between the state-of-the-art theoretical results for matrix completion with the nuclear norm and their equivalent in \textit{inductive matrix completion}: (1) In the distribution-free setting, we prove bounds improving the previously best scaling of \widetilde{O}(rd2) to \widetilde{O}(d3/2√r), where d is the dimension of the side information and rr is the rank. (2) We introduce the (smoothed) \textit{adjusted trace-norm minimization} strategy, an inductive analogue of the weighted trace norm, for which we show guarantees of the order \widetilde{O}(dr) under arbitrary sampling. In the inductive case, a similar rate was previously achieved only under uniform sampling …

Go to article

Vireo @ Trecvid 2021 Ad-Hoc Video Search, Jiaxin Wu, Phuong Anh Nguyen, Chong-Wah Ngo Dec 2021

Vireo @ Trecvid 2021 Ad-Hoc Video Search, Jiaxin Wu, Phuong Anh Nguyen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

In this paper, we summarize our submitted runs and results for Ad-hoc Video Search (AVS) task at TRECVid 2020

Go to article

Let's Read: Designing A Smart Display Application To Support Codas When Learning Spoken Language, Katie Rodeghiero, Yingying Yuki Chen, Annika M. Hettmann, Franceli L. Cibrian Nov 2021

Let's Read: Designing A Smart Display Application To Support Codas When Learning Spoken Language, Katie Rodeghiero, Yingying Yuki Chen, Annika M. Hettmann, Franceli L. Cibrian

Engineering Faculty Articles and Research

Hearing children of Deaf adults (CODAs) face many challenges including having difficulty learning spoken languages, experiencing social judgment, and encountering greater responsibilities at home. In this paper, we present a proposal for a smart display application called Let's Read that aims to support CODAs when learning spoken language. We conducted a qualitative analysis using online community content in English to develop the first version of the prototype. Then, we conducted a heuristic evaluation to improve the proposed prototype. As future work, we plan to use this prototype to conduct participatory design sessions with Deaf adults and CODAs to evaluate the …

Go to article

Feel And Touch: A Haptic Mobile Game To Assess Tactile Processing, Ivonne Monarca, Monica Tentori, Franceli L. Cibrian Nov 2021

Feel And Touch: A Haptic Mobile Game To Assess Tactile Processing, Ivonne Monarca, Monica Tentori, Franceli L. Cibrian

Engineering Faculty Articles and Research

Haptic interfaces have great potential for assessing the tactile processing of children with Autism Spectrum Disorder (ASD), an area that has been under-explored due to the lack of tools to assess it. Until now, haptic interfaces for children have mostly been used as a teaching or therapeutic tool, so there are still open questions about how they could be used to assess tactile processing of children with ASD. This article presents the design process that led to the development of Feel and Touch, a mobile game augmented with vibrotactile stimuli to assess tactile processing. Our feasibility evaluation, with 5 children …

Go to article

Facilitating Heuristic Evaluation For Novice Evaluators, Anas Abulfaraj Nov 2021

Facilitating Heuristic Evaluation For Novice Evaluators, Anas Abulfaraj

College of Computing and Digital Media Dissertations

Heuristic evaluation (HE) is one of the most widely used usability evaluation methods. The reason for its popularity is that it is a discount method, meaning that it does not require substantial time or resources, and it is simple, as evaluators can evaluate a system guided by a set of usability heuristics. Despite its simplicity, a major problem with HE is that there is a significant gap in the quality of results produced by expert and novice evaluators. This gap has made some scholars question the usefulness of the method as they claim that the evaluation results are a product …

Go to article

Mapping E-Commerce Locally And Beyond: Citt K12 Special Investigation Project, Thomas O’Brien, Deanna Matsumoto Nov 2021

Mapping E-Commerce Locally And Beyond: Citt K12 Special Investigation Project, Thomas O’Brien, Deanna Matsumoto

Mineta Transportation Institute

As all aspects of the American workplace become automated or digitally enhanced to some degree, K12 educators have an increasing responsibility to help their students acquire the technical skills necessary to organize and interpret information. Increasingly, this is done through Geographic Information Systems (GIS), especially in careers related to transportation and logistics. The Center for International Trade & Transportation (CITT) at CSU Long Beach has developed this K12 Special Investigation Project to introduce ArcGIS StoryMaps, an engaging, accessible and sophisticated web-based GIS application. The lessons center on e-commerce and its accompanying environmental and economic impact. Still, the activities can be …

Go to article

Self-Supervised Multi-Class Pre-Training For Unsupervised Anomaly Detection And Segmentation In Medical Images, Yu Tian, Fengbei Liu, Guansong Pang, Yuanhong Chen, Yuyuan Liu, Johan W. Verjans, Rajvinder Singh Nov 2021

Self-Supervised Multi-Class Pre-Training For Unsupervised Anomaly Detection And Segmentation In Medical Images, Yu Tian, Fengbei Liu, Guansong Pang, Yuanhong Chen, Yuyuan Liu, Johan W. Verjans, Rajvinder Singh

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection (UAD) that requires only normal (healthy) training images is an important tool for enabling the development of medical image analysis (MIA) applications, such as disease screening, since it is often difficult to collect and annotate abnormal (or disease) images in MIA. However, heavily relying on the normal images may cause the model training to overfit the normal class. Self-supervised pre-training is an effective solution to this problem. Unfortunately, current self-supervision methods adapted from computer vision are sub-optimal for MIA applications because they do not explore MIA domain knowledge for designing the pretext tasks or the training process. …

Go to article

Wav-Bert: Cooperative Acoustic And Linguistic Representation Learning For Low-Resource Speech Recognition, Guolin Zheng, Yubei Xiao, Ke Gong, Pan Zhou, Xiaodan Liang, Liang Lin Nov 2021

Wav-Bert: Cooperative Acoustic And Linguistic Representation Learning For Low-Resource Speech Recognition, Guolin Zheng, Yubei Xiao, Ke Gong, Pan Zhou, Xiaodan Liang, Liang Lin

Research Collection School Of Computing and Information Systems

Unifying acoustic and linguistic representation learning has become increasingly crucial to transfer the knowledge learned on the abundance of high-resource language data for low-resource speech recognition. Existing approaches simply cascade pre-trained acoustic and language models to learn the transfer from speech to text. However, how to solve the representation discrepancy of speech and text is unexplored, which hinders the utilization of acoustic and linguistic information. Moreover, previous works simply replace the embedding layer of the pre-trained language model with the acoustic features, which may cause the catastrophic forgetting problem. In this work, we introduce Wav-BERT, a cooperative acoustic and linguistic …

Go to article

Exploiting Reasoning Chains For Multi-Hop Science Question Answering, Weiwen Xu, Yang Deng, Huihui Zhang, Deng Cai, Wai Lam Nov 2021

Exploiting Reasoning Chains For Multi-Hop Science Question Answering, Weiwen Xu, Yang Deng, Huihui Zhang, Deng Cai, Wai Lam

Research Collection School Of Computing and Information Systems

We propose a novel Chain Guided Retriever reader (CGR) framework to model the reasoning chain for multi-hop Science Question Answering. Our framework is capable of performing explainable reasoning without the need of any corpus-specific annotations, such as the ground-truth reasoning chain, or human annotated entity mentions. Specifically, we first generate reasoning chains from a semantic graph constructed by Abstract Meaning Representation of retrieved evidence facts. A Chain-aware loss, concerning both local and global chain information, is also designed to enable the generated chains to serve as distant supervision signals for training the retriever, where reinforcement learning is also adopted to …

Go to article

Towards Balancing Vr Immersion And Bystander Awareness, Yoshiki Kudo, Anthony Tang, Kazuyuki Fujita, Isamu Endo, Kazuki Takashima, Yoshifumi Kitamura Nov 2021

Towards Balancing Vr Immersion And Bystander Awareness, Yoshiki Kudo, Anthony Tang, Kazuyuki Fujita, Isamu Endo, Kazuki Takashima, Yoshifumi Kitamura

Research Collection School Of Computing and Information Systems

Head-mounted displays (HMDs) increase immersion into virtual worlds. The problem is that this limits headset users' awareness of bystanders: headset users cannot attend to bystanders' presence and activities. We call this the HMD boundary. We explore how to make the HMD boundary permeable by comparing different ways of providing informal awareness cues to the headset user about bystanders. We adapted and implemented three visualization techniques (Avatar View, Radar and Presence++) that share bystanders' location and orientation with headset users. We conducted a hybrid user and simulation study with three different types of VR content (high, medium, low interactivity) with twenty …

Go to article

Learning To Teach And Learn For Semi-Supervised Few-Shot Image Classification, Xinzhe Li, Jianqiang Huang, Yaoyao Liu, Qin Zhou, Shibao Zheng, Bernt Schiele, Qianru Sun Nov 2021

Learning To Teach And Learn For Semi-Supervised Few-Shot Image Classification, Xinzhe Li, Jianqiang Huang, Yaoyao Liu, Qin Zhou, Shibao Zheng, Bernt Schiele, Qianru Sun

Research Collection School Of Computing and Information Systems

This paper presents a novel semi-supervised few-shot image classification method named Learning to Teach and Learn (LTTL) to effectively leverage unlabeled samples in small-data regimes. Our method is based on self-training, which assigns pseudo labels to unlabeled data. However, the conventional pseudo-labeling operation heavily relies on the initial model trained by using a handful of labeled data and may produce many noisy labeled samples. We propose to solve the problem with three steps: firstly, cherry-picking searches valuable samples from pseudo-labeled data by using a soft weighting network; and then, cross-teaching allows the classifiers to teach mutually for rejecting more noisy …

Go to article

Intercept Graph: An Interactive Radial Visualization For Comparison Of State Changes, Shaolun Ruan, Yong Wang, Qiang Guan Nov 2021

Intercept Graph: An Interactive Radial Visualization For Comparison Of State Changes, Shaolun Ruan, Yong Wang, Qiang Guan

Research Collection School Of Computing and Information Systems

State change comparison of multiple data items is often necessary in multiple application domains, such as medical science, financial engineering, sociology, biological science, etc. Slope graphs and grouped bar charts have been widely used to show a “before-and-after” story of different data states and indicate their changes. However, they visualize state changes as either slope or difference of bars, which has been proved less effective for quantitative comparison. Also, both visual designs suffer from visual clutter issues with an increasing number of data items. In this paper, we propose Intercept Graph, a novel visual design to facilitate effective interactive comparison …

Go to article

Acoustic/Gravity Wave Phenomena In Wide-Field Imaging: From Data Analysis To A Modeling Framework For Observability In The Mlt Region And Beyond, Jaime Aguilar Guerrero Nov 2021

Acoustic/Gravity Wave Phenomena In Wide-Field Imaging: From Data Analysis To A Modeling Framework For Observability In The Mlt Region And Beyond, Jaime Aguilar Guerrero

Doctoral Dissertations and Master's Theses

Acoustic waves, gravity waves, and larger-scale tidal and planetary waves are significant drivers of the atmosphere’s dynamics and of the local and global circulation that have direct and indirect impacts on our weather and climate. Their measurements and characterization are fundamental challenges in Aeronomy that require a wide range of instrumentation with distinct operational principles. Most measurements share the common features of integrating optical emissions or effects on radio waves through deep layers of the atmosphere. The geometry of these integrations create line-of-sight effects that must be understood, described, and accounted for to properly present the measured data in traditional …

Go to article

Masked Face Analysis Via Multi-Task Deep Learning, Vatsa S. Patel, Zhongliang Nie, Trung-Nghia Le, Tam Van Nguyen Oct 2021

Masked Face Analysis Via Multi-Task Deep Learning, Vatsa S. Patel, Zhongliang Nie, Trung-Nghia Le, Tam Van Nguyen

Computer Science Faculty Publications

Face recognition with wearable items has been a challenging task in computer vision and involves the problem of identifying humans wearing a face mask. Masked face analysis via multi-task learning could effectively improve performance in many fields of face analysis. In this paper, we propose a unified framework for predicting the age, gender, and emotions of people wearing face masks. We first construct FGNET-MASK, a masked face dataset for the problem. Then, we propose a multi-task deep learning model to tackle the problem. In particular, the multi-task deep learning model takes the data as inputs and shares their weight to …

Go to article

The Efficacy Of Collaborative Authoring Of Video Scene Descriptions, Rosiana Natalie, Jolene Kar Inn Loh, Huei Suen Tan, Joshua Shi-Hao Tseng, Ian Luke Yi-Ren Chan, Ebrima H. Jarjue, Hernisa Kacorri, Kotaro Hara Oct 2021

The Efficacy Of Collaborative Authoring Of Video Scene Descriptions, Rosiana Natalie, Jolene Kar Inn Loh, Huei Suen Tan, Joshua Shi-Hao Tseng, Ian Luke Yi-Ren Chan, Ebrima H. Jarjue, Hernisa Kacorri, Kotaro Hara

Research Collection School Of Computing and Information Systems

The majority of online video contents remain inaccessible to people with visual impairments due to the lack of audio descriptions to depict the video scenes. Content creators have traditionally relied on professionals to author audio descriptions, but their service is costly and not readily-available. We investigate the feasibility of creating more cost-effective audio descriptions that are also of high quality by involving novices. Specifically, we designed, developed, and evaluated ViScene, a web-based collaborative audio description authoring tool that enables a sighted novice author and a reviewer either sighted or blind to interact and contribute to scene descriptions (SDs)—text that can …

Go to article

Visionary Caption: Improving The Accessibility Of Presentation Slides Through Highlighting Visualization, Carmen Ji Yan Yip, Jie Mi Chong, Sin Yee Kwek, Yong Wang, Kotaro Hara Oct 2021

Visionary Caption: Improving The Accessibility Of Presentation Slides Through Highlighting Visualization, Carmen Ji Yan Yip, Jie Mi Chong, Sin Yee Kwek, Yong Wang, Kotaro Hara

Research Collection School Of Computing and Information Systems

Presentation slides are widely used in occasions such as academic talks and business meetings. Captions placed on slides support deaf and hard of hearing (DHH) people to understand spoken contents, but simultaneously comprehending and associating visual contents on slides and caption text could be challenging. In this paper, we design and develop a visualization technique to highlight and associate chart on a slide and numerical data in caption. We first conduct a small formative study with people with and without hearing impairments to assess the value of the visualization technique using a lo-fidelity video prototype. We then develop Visionary Caption, …

Go to article

Conquer: Contextual Query-Aware Ranking For Video Corpus Moment Retrieval, Zhijian Hou, Chong-Wah Ngo, W. K. Chan Oct 2021

Conquer: Contextual Query-Aware Ranking For Video Corpus Moment Retrieval, Zhijian Hou, Chong-Wah Ngo, W. K. Chan

Research Collection School Of Computing and Information Systems

This paper tackles a recently proposed Video Corpus Moment Retrieval task. This task is essential because advanced video retrieval applications should enable users to retrieve a precise moment from a large video corpus. We propose a novel CONtextual QUery-awarE Ranking (CONQUER) model for effective moment localization and ranking. CONQUER explores query context for multi-modal fusion and representation learning in two different steps. The first step derives fusion weights for the adaptive combination of multi-modal video content. The second step performs bi-directional attention to tightly couple video and query as a single joint representation for moment localization. As query context is …

Go to article

Constrained Contrastive Distribution Learning For Unsupervised Anomaly Detection And Localisation In Medical Images, Yu Tian, Guansong Pang, Fengbei Liu, Yuanhong Chen, Seon Ho Shin, Johan W. Verjans, Rajvinder Singh Oct 2021

Constrained Contrastive Distribution Learning For Unsupervised Anomaly Detection And Localisation In Medical Images, Yu Tian, Guansong Pang, Fengbei Liu, Yuanhong Chen, Seon Ho Shin, Johan W. Verjans, Rajvinder Singh

Research Collection School Of Computing and Information Systems

Unsupervised anomaly detection (UAD) learns one-class classifiers exclusively with normal (i.e., healthy) images to detect any abnormal (i.e., unhealthy) samples that do not conform to the expected normal patterns. UAD has two main advantages over its fully supervised counterpart. Firstly, it is able to directly leverage large datasets available from health screening programs that contain mostly normal image samples, avoiding the costly manual labelling of abnormal samples and the subsequent issues involved in training with extremely class-imbalanced data. Further, UAD approaches can potentially detect and localise any type of lesions that deviate from the normal patterns. One significant challenge faced …

Go to article

Differentiated Learning For Multi-Modal Domain Adaptation, Jianming Lv, Kaijie Liu, Shengfeng He Oct 2021

Differentiated Learning For Multi-Modal Domain Adaptation, Jianming Lv, Kaijie Liu, Shengfeng He

Research Collection School Of Computing and Information Systems

Directly deploying a trained multi-modal classifier to a new environment usually leads to poor performance due to the well-known domain shift problem. Existing multi-modal domain adaptation methods treated each modality equally and optimize the sub-models of different modalities synchronously. However, as observed in this paper, the degrees of domain shift in different modalities are usually diverse. We propose a novel Differentiated Learning framework to make use of the diversity between multiple modalities for more effective domain adaptation. Specifically, we model the classifiers of different modalities as a group of teacher/student sub-models, and a novel Prototype based Reliability Measurement is presented …

Go to article

Condensing A Sequence To One Informative Frame For Video Recognition, Qiu. Zhaofan, Ting Yao, Yan Shu, Chong-Wah Ngo, Tao Mei Oct 2021

Condensing A Sequence To One Informative Frame For Video Recognition, Qiu. Zhaofan, Ting Yao, Yan Shu, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Video is complex due to large variations in motion and rich content in fine-grained visual details. Abstracting useful information from such information-intensive media requires exhaustive computing resources. This paper studies a two-step alternative that first condenses the video sequence to an informative" frame" and then exploits off-the-shelf image recognition system on the synthetic frame. A valid question is how to define" useful information" and then distill it from a video sequence down to one synthetic frame. This paper presents a novel Informative Frame Synthesis (IFS) architecture that incorporates three objective tasks, ie, appearance reconstruction, video categorization, motion estimation, and two …

Go to article

First Prev Next Last