Databases and Information Systems Commons^™

The Virtual Reality Questionnaire Toolkit, Martin Feick, Niko Kleer, Anthony Tang, Anthonio Kruger

Research Collection School Of Computing and Information Systems

In this work, we present the VRQuestionnaire Toolkit, which enables the research community to easily collect subjective measures within virtual reality (VR). We contribute a highly customizable and reusable open-source toolkit which can be integrated in existing VR projects rapidly. The toolkit comes with a pre-installed set of standard questionnaires such as NASA TLX, SSQ and SUS Presence questionnaire. Our system aims to lower the entry barrier to use questionnaires in VR and to significantly reduce development time and cost needed to run pre-, in between- and post-study questionnaires.

Multi-Modal Cooking Workflow Construction For Food Recipes, Liangming Pan, Jingjing Chen, Jianlong Wu, Shaoteng Liu, Chong-Wah Ngo, Min-Yen Kan, Yugang Jiang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Understanding food recipe requires anticipating the implicit causal effects of cooking actions, such that the recipe can be converted into a graph describing the temporal workflow of the recipe. This is a non-trivial task that involves common-sense reasoning. However, existing efforts rely on hand-crafted features to extract the workflow graph from recipes due to the lack of large-scale labeled datasets. Moreover, they fail to utilize the cooking images, which constitute an important part of food recipes. In this paper, we build MM-ReS, the first large-scale dataset for cooking workflow construction, consisting of 9,850 recipes with human-labeled workflow graphs. Cooking steps …

Interpretable Embedding For Ad-Hoc Video Search, Jiaxin Wu, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Answering query with semantic concepts has long been the mainstream approach for video search. Until recently, its performance is surpassed by concept-free approach, which embeds queries in a joint space as videos. Nevertheless, the embedded features as well as search results are not interpretable, hindering subsequent steps in video browsing and query reformulation. This paper integrates feature embedding and concept interpretation into a neural network for unified dual-task learning. In this way, an embedding is associated with a list of semantic concepts as an interpretation of video content. This paper empirically demonstrates that, by using either the embedding features or …

Person-Level Action Recognition In Complex Events Via Tsd-Tsm Networks, Yanbin Hao, Zi-Niu Liu, Hao Zhang, Bin Zhu, Jingjing Chen, Yu-Gang Jiang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

The task of person-level action recognition in complex events aims to densely detect pedestrians and individually predict their actions from surveillance videos. In this paper, we present a simple yet efficient pipeline for this task, referred to as TSD-TSM networks. Firstly, we adopt the TSD detector for the pedestrian localization on each single keyframe. Secondly, we generate the sequential ROIs for a person proposal by replicating the adjusted bounding box coordinates around the keyframe. Particularly, we propose to conduct straddling expansion and region squaring on the original bounding box of a person proposal to widen the potential space of motion …

Weakly Paired Multi-Domain Image Translation, M.Y. Zhang, Zhiwu Huang, D.P. Paudel, J. Thoma, Gool L. Van

Research Collection School Of Computing and Information Systems

In this paper, we aim at studying the new problem of weakly paired multi-domain image translation. To this end, we collect a dataset that contains weakly paired images from multiple domains. Two images are considered to be weakly paired if they are captured from nearby locations and share an overlapping field of view. These images are possibly captured by two asynchronous cameras—often resulting in images from separate domains, e.g. summer and winter. Major motivations for using weakly paired images are: (i) performance improvement towards that of paired data; (ii) cheap labels and abundant data availability. For the first time in …

Snow-Albedo Feedback In Northern Alaska: How Vegetation Influences Snowmelt, Lucas C. Reckhaus

Theses and Dissertations

This paper investigates how the snow-albedo feedback mechanism of the arctic is changing in response to rising climate temperatures. Specifically, the interplay of vegetation and snowmelt, and how these two variables can be correlated. This has the potential to refine climate modelling of the spring transition season. Research was conducted at the ecoregion scale in northern Alaska from 2000 to 2020. Each ecoregion is defined by distinct topographic and ecological conditions, allowing for meaningful contrast between the patterns of spring albedo transition across surface conditions and vegetation types. The five most northerly ecoregions of Alaska are chosen as they encompass …

Mining User-Generated Content Of Mobile Patient Portal: Dimensions Of User Experience, Mohammad A. Al-Ramahi, Cherie Noteboom

Computer Information Systems Faculty Publications (Archived)

Patient portals are positioned as a central component of patient engagement through the potential to change the physician-patient relationship and enable chronic disease self-management. The incorporation of patient portals provides the promise to deliver excellent quality, at optimized costs, while improving the health of the population. This study extends the existing literature by extracting dimensions related to the Mobile Patient Portal Use. We use a topic modeling approach to systematically analyze users’ feedback from the actual use of a common mobile patient portal, Epic's MyChart. Comparing results of Latent Dirichlet Allocation analysis with those of human analysis validated the extracted …

Interface Design Of Web-Based Educational Platforms For Young Students, Lurong He, Fiona Fui-Hoon Nah

Research Collection School Of Computing and Information Systems

With the outbreak of the Covid-19 pandemic, almost all the school programs in the United States are closed. Most educational programs have moved online. Both teachers and students are facing challenges adapting to this new teaching and learning mode. This paper will focus on the interface design of webbased educational platforms for students and teachers from elementary and middle schools. The age of students from elementary and middle schools typically ranges from 5 to 13 years old. Their computer literacy level is still preliminary. So traditional online teaching platforms may not be appropriate for them. The design of learning systems …

Tree-Augmented Cross-Modal Encoding For Complex-Query Video Retrieval, Xun Yang, Jianfeng Dong, Yixin Cao, Xun Wang, Meng Wang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

The rapid growth of user-generated videos on the Internet has intensified the need for text-based video retrieval systems. Traditional methods mainly favor the concept-based paradigm on retrieval with simple queries, which are usually ineffective for complex queries that carry far more complex semantics. Recently, embedding-based paradigm has emerged as a popular approach. It aims to map the queries and videos into a shared embedding space where semantically-similar texts and videos are much closer to each other. Despite its simplicity, it forgoes the exploitation of the syntactic structure of text queries, making it suboptimal to model the complex queries. To facilitate …

Improving Event Detection Via Open-Domain Event Trigger Knowledge, Meihan Tong, Bin Xu, Shuai Wang, Yixin Cao, Lei Hou, Juanzi Li, Jun Xie

Research Collection School Of Computing and Information Systems

Event Detection (ED) is a fundamental task in automatically structuring texts. Due to the small scale of training data, previous methods perform poorly on unseen/sparsely labeled trigger words and are prone to overfitting densely labeled trigger words. To address the issue, we propose a novel Enrichment Knowledge Distillation (EKD) model to leverage external open-domain trigger knowledge to reduce the in-built biases to frequent trigger words in annotations. Experiments on benchmark ACE2005 show that our model outperforms nine strong baselines, is especially effective for unseen/sparsely labeled trigger words. The source code is released on https://github.com/shuaiwa16/ekd.git.

A Review On Eye-Tracking Metrics For Sleepiness, Debasis Roy, Fiona Fui-Hoon Nah

Research Collection School Of Computing and Information Systems

Sleepiness that can arise from sleep deprivation can increase human errors in task performance and create workplace hazards and accidents. Hence, it is critical to detect sleepiness to minimize hazards and human errors. This paper provides a review of the literature on eye tracking metrics that can be used to detect sleepiness. These metrics include blink duration, blink frequency, saccade latency, saccade peak velocity, saccade accuracy, smooth pursuit velocity gain, fixation rate, pupil size, and latency to pupil constriction.

Expertise Style Transfer: A New Task Towards Better Communication Between Experts And Laymen, Yixin Cao, Ruihao Shui, Liangming Pan, Min-Yen Kan, Zhiyuan Lu, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

The curse of knowledge can impede communication between experts and laymen. We propose a new task of expertise style transfer and contribute a manually annotated dataset with the goal of alleviating such cognitive biases. Solving this task not only simplifies the professional language, but also improves the accuracy and expertise level of laymen descriptions using simple words. This is a challenging task, unaddressed in previous work, as it requires the models to have expert intelligence in order to modify text with a deep understanding of domain knowledge and structures. We establish the benchmark performance of five state-of-the-art models for style …

Deep Learning Of Facial Embeddings And Facial Landmark Points For The Detection Of Academic Emotions, Hua Leong Fwa

Research Collection School Of Computing and Information Systems

Automatic emotion recognition is an actively researched area as emotion plays a pivotal role in effective human communications. Equipping a computer to understand and respond to human emotions has potential applications in many fields including education, medicine, transport and hospitality. In a classroom or online learning context, the basic emotions do not occur frequently and do not influence the learning process itself. The academic emotions such as engagement, frustration, confusion and boredom are the ones which are pivotal to sustaining the motivation of learners. In this study, we evaluated the use of deep learning on FaceNet embeddings and facial landmark …

Visual Commonsense Representation Learning Via Causal Inference, Tan Wang, Jianqiang Huang, Hanwang Zhang, Qianru Sun

Research Collection School Of Computing and Information Systems

We present a novel unsupervised feature representation learning method, Visual Commonsense Region-based Convolutional Neural Network (VC R-CNN), to serve as an improved visual region encoder for high-level tasks such as captioning and VQA. Given a set of detected object regions in an image (e.g., using Faster R-CNN), like any other unsupervised feature learning methods (e.g., word2vec), the proxy training objective of VC R-CNN is to predict the con-textual objects of a region. However, they are fundamentally different: the prediction of VC R-CNN is by using causal intervention: P(Y|do(X)), while others are by using the conventional likelihood: P(Y|X). We extensively apply …

Hyperbolic Visual Embedding Learning For Zero-Shot Recognition, Shaoteng Liu, Jingjing Chen, Liangming Pan, Chong-Wah Ngo, Tat-Seng Chua, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

This paper proposes a Hyperbolic Visual Embedding Learning Network for zero-shot recognition. The network learns image embeddings in hyperbolic space, which is capable of preserving the hierarchical structure of semantic classes in low dimensions. Comparing with existing zeroshot learning approaches, the network is more robust because the embedding feature in hyperbolic space better represents class hierarchy and thereby avoid misleading resulted from unrelated siblings. Our network outperforms exiting baselines under hierarchical evaluation with an extremely challenging setting, i.e., learning only from 1,000 categories to recognize 20,841 unseen categories. While under flat evaluation, it has competitive performance as state-of-the-art methods but …

Mnemonics Training: Multi-Class Incremental Learning Without Forgetting, Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, Qianru Sun

Research Collection School Of Computing and Information Systems

Multi-Class Incremental Learning (MCIL) aims to learn new concepts by incrementally updating a model trained on previous concepts. However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones. To alleviate this issue, it has been proposed to keep around a few examples of the previous concepts but the effectiveness of this approach heavily depends on the representativeness of these examples. This paper proposes a novel and automatic framework we call mnemonics, where we parameterize exemplars and make them optimizable in an end-to-end manner. We train the framework through bilevel optimizations, i.e., model-level and …

Ntire 2020 Challenge On Video Quality Mapping: Methods And Results, D. Fuoli, Zhiwu Huang, M. Danelljan, R. Timofte, H. Wang, L. Jin, D. Su, J. Liu, J. Lee, M. Kudelski, L. Bala, D. Hryboy, M. Mozejko, M. Li, S. Li, B. Pang, C. Lu, Li C., He D., Li F.

Research Collection School Of Computing and Information Systems

This paper reviews the NTIRE 2020 challenge on video quality mapping (VQM), which addresses the issues of quality mapping from source video domain to target video domain. The challenge includes both a supervised track (track 1) and a weakly-supervised track (track 2) for two benchmark datasets. In particular, track 1 offers a new Internet video benchmark, requiring algorithms to learn the map from more compressed videos to less compressed videos in a supervised training manner. In track 2, algorithms are required to learn the quality mapping from one device to another when their quality varies substantially and weaklyaligned video pairs …

Server Score, Zachary Buresh

Student Academic Conference

This presentation is in regards to the Android mobile application that I developed in the Kotlin programming language named "Server Score". The app helps waiters/waitresses calculate, track, and predict performance related statistics on the job.

Use Of Eye-Tracking To Identify Psychological Indicators Of Sleepiness, Debasis Roy, Fiona Fui-Hoon Nah, Matthew Thimgan

Research Collection School Of Computing and Information Systems

Sleepiness or sleep deprivation creates a serious hazard or obstacle to task execution and performance. Sleep deprivation can be life-threating (e.g., when driving or executing attention-critical tasks). We are interested to examine if eye-tracking technology can be used to assess and detect sleepiness in an online environment. In this research proposal, we will focus on examining the relationships between sleepiness and parameters involving pupil size, blinks, and saccades.

Relationships Between Willingness To Share Information For Benefits And Trust, Gaurav Bansal, Fiona Fui-Hoon Nah

Research Collection School Of Computing and Information Systems

This research examines the role of willingness to share one’s information for three benefits as tradeoffs – monetary gains, personalization, and national security – and their effects on trust in online businesses. Data were gathered from MTurk and the results indicate that willingness to share information for monetary gains and personalization is marginally associated with trust in online businesses, but willingness to share information for national security has no association with trust in online businesses. The paper also discusses implications, limitations, and future research directions.

Storage Management Strategy In Mobile Phones For Photo Crowdsensing, En Wang, Zhengdao Qu, Xinyao Liang, Xiangyu Meng, Yongjian Yang, Dawei Li, Weibin Meng

Department of Computer Science Faculty Scholarship and Creative Works

In mobile crowdsensing, some users jointly finish a sensing task through the sensors equipped in their intelligent terminals. In particular, the photo crowdsensing based on Mobile Edge Computing (MEC) collects pictures for some specific targets or events and uploads them to nearby edge servers, which leads to richer data content and more efficient data storage compared with the common mobile crowdsensing; hence, it has attracted an important amount of attention recently. However, the mobile users prefer uploading the photos through Wifi APs (PoIs) rather than cellular networks. Therefore, photos stored in mobile phones are exchanged among users, in order to …

Poster Abstract: Data Communication Using Switchable Privacy Glass, Changshuo Hu, Dong Ma, Mahbub Hassan, Wen Hu

Research Collection School Of Computing and Information Systems

Switchable privacy glass can electronically change its state between opaque and transparent. In this work, we propose to exploit the electronic configurability of switchable glass to modulate natural light, which can be demodulated by a nearby receiver with light sensing capability to realise data communication over natural light. A key advantage is that no energy is used to generate light, as it simply modulates the existing light in the nature. We demonstrate that the proposed data communication using switchable glass modulation can achieve 33.33 bits per second communication with a bit rate below 1% under a wide range of ambient …

Reinforced Negative Sampling Over Knowledge Graph For Recommendation, Xiang Wang, Yaokun Xu, Xiangnan He, Yixin Cao, Meng Wang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Properly handling missing data is a fundamental challenge in recommendation. Most present works perform negative sampling from unobserved data to supply the training of recommender models with negative signals. Nevertheless, existing negative sampling strategies, either static or adaptive ones, are insufficient to yield high-quality negative samples — both informative to model training and reflective of user real needs. In this work, we hypothesize that item knowledge graph (KG), which provides rich relations among items and KG entities, could be useful to infer informative and factual negative samples. Towards this end, we develop a new negative sampling model, Knowledge Graph Policy …

Image Enhanced Event Detection In News Articles, Meihan Tong, Shuai Wang, Yixin Cao, Bin Xu, Juaizi Li, Lei Hou, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Event detection is a crucial and challenging sub-task of event extraction, which suffers from a severe ambiguity issue of trigger words. Existing works mainly focus on using textual context information, while there naturally exist many images accompanied by news articles that are yet to be explored. We believe that images not only reflect the core events of the text, but are also helpful for the disambiguation of trigger words. In this paper, we first contribute an image dataset supplement to ED benchmarks (i.e., ACE2005) for training and evaluation. We then propose a novel Dual Recurrent Multimodal Model, DRMM, to conduct …

Building Something With The Raspberry Pi, Richard Kordel

Harrisburg University Presidential Research Grants

In 2017 Ryan Korn and I submitted a grant proposal in the annual Harrisburg University President’s Grant process. Our proposal was to partner with a local high school to install a classroom of 20 Raspberry Pi’s, along with the requisite peripherals. In that classroom students would be challenged to design something that combined programming with physical computing. In our presentation to the school we suggested that this project would give students the opportunity to be “amazing.”

As part of the grant, the top three students would be given scholarships to HU and the top five finalists would all be permitted …

Improved Generalisation Bounds For Deep Learning Through L∞ Covering Numbers, Antoine Ledent, Yunwen Lei, Marius Kloft

Research Collection School Of Computing and Information Systems

Using proof techniques involving L∞ covering numbers, we show generalisation error bounds for deep learning with two main improvements over the state of the art. First, our bounds have no explicit dependence on the number of classes except for logarithmic factors. This holds even when formulating the bounds in terms of the L 2 norm of the weight matrices, while previous bounds exhibit at least a square-root dependence on the number of classes in this case. Second, we adapt the Rademacher analysis of DNNs to incorporate weight sharing—a task of fundamental theoretical importance which was previously attempted only under very …

Special Issue On Multimedia Recommendation And Multi-Modal Data Analysis, Xiangnan He, Zhenguang Liu, Hanwang Zhang, Chong-Wah Ngo, Svebor Karaman, Yongfeng Zhang

Research Collection School Of Computing and Information Systems

Rich multimedia contents are dominating the Web. In popular social media platforms such as FaceBook, Twitter, and Instagram, there are over millions of multimedia contents being created by users on a daily basis. In the meantime, multimedia data consist of data in multiple modalities, such as text, images, audio, and so on. Users are heavily overloaded by the massive multi-modal data, and it becomes critical to explore advanced techniques for heterogeneous big data analytics and multimedia recommendation. Traditional multimedia recommendation and data analysis technologies cannot well address the problem of understanding users’ preference in the feature-rich multimedia contents, and have …

Semi-Supervised Entity Alignment Via Joint Knowledge Embedding Model And Cross-Graph Model, Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Entity alignment aims at integrating complementary knowledge graphs (KGs) from different sources or languages, which may benefit many knowledge-driven applications. It is challenging due to the heterogeneity of KGs and limited seed alignments. In this paper, we propose a semi-supervised entity alignment method by joint Knowledge Embedding model and Cross-Graph model (KECG). It can make better use of seed alignments to propagate over the entire graphs with KG-based constraints. Specifically, as for the knowledge embedding model, we utilize TransE to implicitly complete two KGs towards consistency and learn relational constraints between entities. As for the cross-graph model, we extend Graph …

Low-Resource Name Tagging Learned With Weakly Labeled Data, Yixin Cao, Zikun Hu, Tat-Seng Chua, Zhiyuan Liu, Heng Ji

Research Collection School Of Computing and Information Systems

Name tagging in low-resource languages or domains suffers from inadequate training data. Existing work heavily relies on additional information, while leaving those noisy annotations unexplored that extensively exist on the web. In this paper, we propose a novel neural model for name tagging solely based on weakly labeled (WL) data, so that it can be applied in any low-resource settings. To take the best advantage of all WL sentences, we split them into high-quality and noisy portions for two modules, respectively: (1) a classification module focusing on the large portion of noisy data can efficiently and robustly pretrain the tag …

Mixed-Dish Recognition With Contextual Relation Networks, Lixi Deng, Jingjing Chen, Qianru Sun, Xiangnan He, Sheng Tang, Zhaoyan Ming, Yongdong Zhang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Mixed dish is a food category that contains different dishes mixed in one plate, and is popular in Eastern and Southeast Asia. Recognizing individual dishes in a mixed dish image is important for health related applications, e.g. calculating the nutrition values. However, most existing methods that focus on single dish classification are not applicable to mixed-dish recognition. The new challenge in recognizing mixed-dish images are the complex ingredient combination and severe overlap among different dishes. In order to tackle these problems, we propose a novel approach called contextual relation networks (CR-Nets) that encodes the implicit and explicit contextual relations among …