Graphics and Human Computer Interfaces Commons^™

Sail: Saliency-Driven Injection Of Aria Landmarks, Ali Selman Aydin, Shirin Feiz, Vikas Ashok, Iv Ramakrishnan

Computer Science Faculty Publications

Navigating webpages with screen readers is a challenge even with recent improvements in screen reader technologies and the increased adoption of web standards for accessibility, namely ARIA. ARIA landmarks, an important aspect of ARIA, lets screen reader users access different sections of the webpage quickly, by enabling them to skip over blocks of irrelevant or redundant content. However, these landmarks are sporadically and inconsistently used by web developers, and in many cases, even absent in numerous web pages. Therefore, we propose SaIL, a scalable approach that automatically detects the important sections of a web page, and then injects ARIA landmarks …

Neighbourhood Structure Preserving Cross-Modal Embedding For Video Hyperlinking, Yanbin Hao, Chong-Wah Ngo, Benoit Huet

Research Collection School Of Computing and Information Systems

Video hyperlinking is a task aiming to enhance the accessibility of large archives, by establishing links between fragments of videos. The links model the aboutness between fragments for efficient traversal of video content. This paper addresses the problem of link construction from the perspective of cross-modal embedding. To this end, a generalized multi-modal auto-encoder is proposed.& x00A0;The encoder learns two embeddings from visual and speech modalities, respectively, whereas each of the embeddings performs self-modal and cross-modal translation of modalities. Furthermore, to preserve the neighbourhood structure of fragments, which is important for video hyperlinking, the auto-encoder is devised to model data …

Vireo @ Video Browser Showdown 2020, Phuong Anh Nguyen, Jiaxin Wu, Chong-Wah Ngo, Danny Francis, Benoit Huet

Research Collection School Of Computing and Information Systems

In this paper, we present the features implemented in the 4th version of the VIREO Video Search System (VIREO-VSS). In this version, we propose a sketch-based retrieval model, which allows the user to specify a video scene with objects and their basic properties, including color, size, and location. We further utilize the temporal relation between video frames to strengthen this retrieval model. For text-based retrieval module, we supply speech and on-screen text for free-text search and upgrade the concept bank for concept search. The search interface is also re-designed targeting the novice user. With the introduced system, we expect that …

Web Content Management System And Accessibility Awareness: A Comparative Study Of Novice Users And Accessibility Outcomes, Fatima Artiba Diaz

Theses: Doctorates and Masters

Since its creation, the Web has progressively developed and become a vital source of information in every domain and for almost all people. It is crucial to guarantee that the information contained on the Web is available for everyone, especially for people with special needs. Removing accessibility barriers is fundamentally based on tools, skills and support of all contributors, particularly the content creators, to ensure information is navigable and usable in the context of the end users experience. Web Content Management Systems play a significant role in structuring, storing and provision content to the Web and have evolved to address …

Feature Extraction And Description For Retinal Fundus Image Registration, Ramli Roziana

Student Works (2020-2029)

Retinal fundus image registration (RIR) is performed to align two or more fundus images. A general framework of a feature-based RIR technique comprises of preprocessing, feature extraction, feature descriptor, matching and estimating geometrical transformation. The RIR is mainly performed for super-resolution, image mosaicking and longitudinal study applications to assist diagnosis and monitoring retinal diseases. Registering image pair from these applications involve a combination of challenges such as overlapping area and rotation between images. The challenges of the overlapping area and rotation can be addressed at feature extraction and feature descriptor stages of the feature-based RIR technique, respectively. To address the …

An Intra-Severity Classification And Adaptation Technique To Improve Dysarthric Speech Recognition Accuracy, Al-Qatab Bassam Ali Qasem

Student Works (2020-2029)

Dysarthria is a motor speech impairment at the neurological and/or muscular levels that caused difficulty in pronouncing words clearly. Automatic speech recognition (ASR) system is increasingly applied as assistive technology to aid an individual with physical disability particularly the speech impaired community such as dysarthria speakers. However, the development of an effective ASR system is hindered by the data sparsity, either in the coverage of the language or the size of the existing speech databases. The speaker adaptation (SA) technique is one of the solutions to overcome the data sparsity issue of ASR for dysarthric speakers. Our proposed method introduces …

A Desire Fulfillment Theory Of Digital Game Enjoyment, Owen M. Schaffer

College of Computing and Digital Media Dissertations

Empirical research on what makes digital games enjoyable is critical for practitioners who want to design for enjoyment, including for Game Design, Gamification, and Serious Games. But existing theories of what leads to digital game enjoyment have been incomplete or lacking in empirical support showing their impact on enjoyment.

Desire Fulfillment Theory is proposed as a new theory of what leads to digital game enjoyment and tested through research with people who have recently played a digital game. This theory builds on three established theories: Expectancy Disconfirmation Theory, Theory of Basic Human Desires, and Flow Theory. These three theories are …

Efficient Meta Learning Via Minibatch Proximal Update, Pan Zhou, Xiao-Tong Yuan, Huan Xu, Shuicheng Yan, Jiashi Feng

Research Collection School Of Computing and Information Systems

We address the problem of meta-learning which learns a prior over hypothesis from a sample of meta-training tasks for fast adaptation on meta-testing tasks. A particularly simple yet successful paradigm for this research is model-agnostic meta-learning (MAML). Implementation and analysis of MAML, however, can be tricky; first-order approximation is usually adopted to avoid directly computing Hessian matrix but as a result the convergence and generalization guarantees remain largely mysterious for MAML. To remedy this deficiency, in this paper we propose a minibatch proximal update based meta-learning approach for learning to efficient hypothesis transfer. The principle is to learn a prior …

Improved Generalisation Bounds For Deep Learning Through L∞ Covering Numbers, Antoine Ledent, Yunwen Lei, Marius Kloft

Research Collection School Of Computing and Information Systems

Using proof techniques involving L∞ covering numbers, we show generalisation error bounds for deep learning with two main improvements over the state of the art. First, our bounds have no explicit dependence on the number of classes except for logarithmic factors. This holds even when formulating the bounds in terms of the L 2 norm of the weight matrices, while previous bounds exhibit at least a square-root dependence on the number of classes in this case. Second, we adapt the Rademacher analysis of DNNs to incorporate weight sharing—a task of fundamental theoretical importance which was previously attempted only under very …

Image Classification Using Fuzzy Fca, Niruktha Roy Gotoor

School of Computing: Dissertations, Theses, and Student Research

Formal concept analysis (FCA) is a mathematical theory based on lattice and order theory used for data analysis and knowledge representation. It has been used in various domains such as data mining, machine learning, semantic web, Sciences, for the purpose of data analysis and Ontology over the last few decades. Various extensions of FCA are being researched to expand it's scope over more departments. In this thesis,we review the theory of Formal Concept Analysis (FCA) and its extension Fuzzy FCA. Many studies to use FCA in data mining and text learning have been pursued. We extend these studies to include …

Improving Medication Information Presentation Through Interactive Visualization In Mobile Apps: Human Factors Design, Don Roosan, Yan Li, Anandi Law, Huy Truong, Mazharul Karim, Jay Chok, Moom Roosan

Pharmacy Faculty Articles and Research

Background: Despite the detailed patient package inserts (PPIs) with prescription drugs that communicate crucial information about safety, there is a critical gap between patient understanding and the knowledge presented. As a result, patients may suffer from adverse events. We propose using human factors design methodologies such as hierarchical task analysis (HTA) and interactive visualization to bridge this gap. We hypothesize that an innovative mobile app employing human factors design with an interactive visualization can deliver PPI information aligned with patients’ information processing heuristics. Such an app may help patients gain an improved overall knowledge of medications.

Objective: The …

Eurecom At Trecvid Avs 2019, Danny Francis, Phuong Anh Nguyen, Benoit Huet, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

This notebook reports the model and results of the EURECOM runs at TRECVID AVS 2019.

Special Issue On Multimedia Recommendation And Multi-Modal Data Analysis, Xiangnan He, Zhenguang Liu, Hanwang Zhang, Chong-Wah Ngo, Svebor Karaman, Yongfeng Zhang

Research Collection School Of Computing and Information Systems

Rich multimedia contents are dominating the Web. In popular social media platforms such as FaceBook, Twitter, and Instagram, there are over millions of multimedia contents being created by users on a daily basis. In the meantime, multimedia data consist of data in multiple modalities, such as text, images, audio, and so on. Users are heavily overloaded by the massive multi-modal data, and it becomes critical to explore advanced techniques for heterogeneous big data analytics and multimedia recommendation. Traditional multimedia recommendation and data analysis technologies cannot well address the problem of understanding users’ preference in the feature-rich multimedia contents, and have …

Semi-Supervised Entity Alignment Via Joint Knowledge Embedding Model And Cross-Graph Model, Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Entity alignment aims at integrating complementary knowledge graphs (KGs) from different sources or languages, which may benefit many knowledge-driven applications. It is challenging due to the heterogeneity of KGs and limited seed alignments. In this paper, we propose a semi-supervised entity alignment method by joint Knowledge Embedding model and Cross-Graph model (KECG). It can make better use of seed alignments to propagate over the entire graphs with KG-based constraints. Specifically, as for the knowledge embedding model, we utilize TransE to implicitly complete two KGs towards consistency and learn relational constraints between entities. As for the cross-graph model, we extend Graph …

Shellnet: Efficient Point Cloud Convolutional Neural Networks Using Concentric Shells Statistics, Zhiyuan Zhang, Binh-Son Hua, Sai-Kit Yeung

Research Collection School Of Computing and Information Systems

Deep learning with 3D data has progressed significantly since the introduction of convolutional neural networks that can handle point order ambiguity in point cloud data. While being able to achieve good accuracies in various scene understanding tasks, previous methods often have low training speed and complex network architecture. In this paper, we address these problems by proposing an efficient end-to-end permutation invariant convolution for point cloud deep learning. Our simple yet effective convolution operator named ShellConv uses statistics from concentric spherical shells to define representative features and resolve the point order ambiguity, allowing traditional convolution to perform on such features. …

Gender And Racial Diversity In Commercial Brands' Advertising Images On Social Media, Jisun An, Haewoon Kwak

Research Collection School Of Computing and Information Systems

Gender and racial diversity in the mediated images from the media shape our perception of different demographic groups. In this work, we investigate gender and racial diversity of 85,957 advertising images shared by the 73 top international brands on Instagram and Facebook. We hope that our analyses give guidelines on how to build a fully automated watchdog for gender and racial diversity in online advertisements.

Revisiting Collaboration Through Mixed Reality: The Evolution Of Groupware, Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, Mark Billinghurst

Research Collection School Of Computing and Information Systems

Collaborative Mixed Reality (MR) systems are at a critical point in time as they are soon to become more commonplace. However, MR technology has only recently matured to the point where researchers can focus deeply on the nuances of supporting collaboration, rather than needing to focus on creating the enabling technology. In parallel, but largely independently, the field of Computer Supported Cooperative Work (CSCW) has focused on the fundamental concerns that underlie human communication and collaboration over the past 30-plus years. Since MR research is now on the brink of moving into the real world, we reflect on three decades …

Vireo-Eurecom @ Trecvid 2019: Ad-Hoc Video Search (Avs), Phuong Anh Nguyen, Jiaxin Wu, Chong-Wah Ngo, Francis Danny, Benoit Huet

Research Collection School Of Computing and Information Systems

In this paper, we describe the systems developed for Ad-hoc Video Search (AVS) task at TRECVID 2019[1] and the achieved results.

Vireojd-Mm @ Trecvid 2019: Activities In Extended Video (Actev), Zhijian Hou, Ying-Wei Pan, Ting Yao, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

In this paper, we describe the system developed for Activities in Extended Video(ActEV) task at TRECVid 2019 [1] and the achieved results. Activities in Extended Video(ActEV): The goal of Activities in Extended Video is to spatially and temporally localize the action instances in a surveillance setting. We have participated in previous ActEV prize challenge. Since the only difference between the two challenges is evaluation metric, we maintain previous pipeline [2] for this challenge. The pipeline has three stages: object detection, tubelet generation and temporal action localization. This time we extend the system for two aspects separately: better object detection and …

Low-Resource Name Tagging Learned With Weakly Labeled Data, Yixin Cao, Zikun Hu, Tat-Seng Chua, Zhiyuan Liu, Heng Ji

Research Collection School Of Computing and Information Systems

Name tagging in low-resource languages or domains suffers from inadequate training data. Existing work heavily relies on additional information, while leaving those noisy annotations unexplored that extensively exist on the web. In this paper, we propose a novel neural model for name tagging solely based on weakly labeled (WL) data, so that it can be applied in any low-resource settings. To take the best advantage of all WL sentences, we split them into high-quality and noisy portions for two modules, respectively: (1) a classification module focusing on the large portion of noisy data can efficiently and robustly pretrain the tag …