Databases and Information Systems Commons^™

Mmekg: Multi-Modal Event Knowledge Graph Towards Universal Representation Across Modalities, Yubo Ma, Zehao Wang, Mukai Li, Yixin Cao, Meiqi Chen, Xinze Li, Wenqi Sun, Kunquan Deng, Kun Wang, Aixin Sun, Jing Shao

Research Collection School Of Computing and Information Systems

Events are fundamental building blocks of realworld happenings. In this paper, we present a large-scale, multi-modal event knowledge graph named MMEKG. MMEKG unifies different modalities of knowledge via events, which complement and disambiguate each other. Specifically, MMEKG incorporates (i) over 990 thousand concept events with 644 relation types to cover most types of happenings, and (ii) over 863 million instance events connected through 934 million relations, which provide rich contextual information in texts and/or images. To collect billion-scale instance events and relations among them, we additionally develop an efficient yet effective pipeline for textual/visual knowledge extraction system. We also develop …

Prompt For Extraction? Paie: Prompting Argument Interaction For Event Argument Extraction, Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, Jing Shao

Research Collection School Of Computing and Information Systems

In this paper, we propose an effective yet efficient model PAIE for both sentence-level and document-level Event Argument Extraction (EAE), which also generalizes well when there is a lack of training data. On the one hand, PAIE utilizes prompt tuning for extractive objectives to take the best advantages of Pre-trained Language Models (PLMs). It introduces two span selectors based on the prompt to select start/end tokens among input texts for each role. On the other hand, it captures argument interactions via multi-role prompts and conducts joint optimization with optimal span assignments via a bipartite matching loss. Also, with a flexible …

Database Query Execution Through Virtual Reality, Logan Bateman, Marc Butler

Campus Research Month

Building database queries often requires technical knowledge of a query language. However, company employees, such as executives, managers, and others (outside of software research and development, generally) may not have the pre-required knowledge to accurately construct and execute database queries. This paper proposes an approach to constructing database queries using virtual reality. This approach utilizes natural hand or controller gestures which map to various components of building and visualizing database queries.

Novel 360-Degree Camera, Ian Gauger, Andrew Kurtz, Zakariya Niazi

Frameless

Circle Optics is developing novel technology for low-parallax, real time, panoramic image capture using an integrated array of multiple adjacent polygonal-edged cameras. This technology can be optimized and deployed for a variety of markets, including cinematic VR. Circle Optics’ existing prototype, Hydra Alpha, will be demonstrated.

Trend: Temporal Event And Node Dynamics For Graph Representation Learning, Zhihao Wen, Yuan Fang

Research Collection School Of Computing and Information Systems

Temporal graph representation learning has drawn significant attention for the prevalence of temporal graphs in the real world. However, most existing works resort to taking discrete snapshots of the temporal graph, or are not inductive to deal with new nodes, or do not model the exciting effects which is the ability of events to influence the occurrence of another event. In this work, We propose TREND, a novel framework for temporal graph representation learning, driven by TempoRal Event and Node Dynamics and built upon a Hawkes process-based graph neural network (GNN). TREND presents a few major advantages: (1) it is …

Gesturelens: Visual Analysis Of Gestures In Presentation Videos, Haipeng Zeng, Xingbo Wang, Yong Wang, Aoyu Wu, Ting Chuen Pong, Huamin Qu

Research Collection School Of Computing and Information Systems

Appropriate gestures can enhance message delivery and audience engagement in both daily communication and public presentations. In this paper, we contribute a visual analytic approach that assists professional public speaking coaches in improving their practice of gesture training through analyzing presentation videos. Manually checking and exploring gesture usage in the presentation videos is often tedious and time-consuming. There lacks an efficient method to help users conduct gesture exploration, which is challenging due to the intrinsically temporal evolution of gestures and their complex correlation to speech content. In this paper, we propose GestureLens, a visual analytics system to facilitate gesture-based and …

Modeling Sentiments And Preferences From Multimodal Data, Quoc Tuan Truong

Dissertations and Theses Collection (Open Access)

Online reviews are prevalent in many modern Web applications, such as e-commerce, crowd-sourced location and check-in platforms. Fueled by the rise of mobile phones that are often the only cameras on hand, reviews are increasingly multimodal, with photos in addition to textual content. In this thesis, we focus on modeling the subjectivity carried in this form of data, with two research objectives.

In the first part, we tackle the problem of detecting sentiment expressed by a review. This is a key unlocking many applications, e.g., analyzing opinions, monitoring consumer satisfaction, assessing product quality.
Traditionally, the task of sentiment analysis primarily …

Usability Of Health-Related Websites By Filipino-American Adults And Nursing Informatics Experts, Kathleen Begonia

Dissertations, Theses, and Capstone Projects

Filipino-Americans are an understudied minority group with high prevalence and mortality from chronic conditions, such as cardiovascular disease and diabetes. Facing barriers to care and lack of culturally appropriate health resources, they frequently use the internet to obtain health information. It is unknown whether they perceive health-related websites to be useful or easy to use because there are no published usability studies involving this population. Using the Technology Acceptance Model as a theoretical framework, this study investigated the difference between website design ratings by experts and the perceptions of Filipino-American users to determine if usability guidelines influenced the perceived ease …

Developing An Ios Game Application: Magnet Hockey, Trevor D. Wysong

The Graduate Review

Mobile application development requires mindful and meticulous planning. Application design should be responsive and intuitive so that navigation feels natural for the user. All targeted devices should be offered a relatively consistent experience. To ensure this, app performance needs to be closely monitored and different screen sizes and aspect ratios need to be considered when scaling. For an app to become attractive to many people, it should either be competitive with similar apps or be unique and interesting enough to entice people to download it. A unique app should couple familiar elements with new components or twists [8]. People are …

Deep Learning For Video-Grounded Dialogue Systems, Hung Le

Dissertations and Theses Collection (Open Access)

In recent years, we have witnessed significant progress in building systems with artificial intelligence. However, despite advancements in machine learning and deep learning, we are still far from achieving autonomous agents that can perceive multi-dimensional information from the surrounding world and converse with humans in natural language. Towards this goal, this thesis is dedicated to building intelligent systems in the task of video-grounded dialogues. Specifically, in a video-grounded dialogue, a system is required to hold a multi-turn conversation with humans about the content of a video. Given an input video, a dialogue history, and a question about the video, the …

Kg4vis: A Knowledge Graph-Based Approach For Visualization Recommendation, Haotian Li, Yong Wang, Songheng Zhang, Yangqiu Song, Huamin. Qu

Research Collection School Of Computing and Information Systems

Visualization recommendation or automatic visualization generation can significantly lower the barriers for general users to rapidly create effective data visualizations, especially for those users without a background in data visualizations. However, existing rule-based approaches require tedious manual specifications of visualization rules by visualization experts. Other machine learning-based approaches often work like black-box and are difficult to understand why a specific visualization is recommended, limiting the wider adoption of these approaches. This paper fills the gap by presenting KG4Vis, a knowledge graph (KG)-based approach for visualization recommendation. It does not require manual specifications of visualization rules and can also guarantee good …

Cross-Modal Food Retrieval: Learning A Joint Embedding Of Food Images And Recipes With Semantic Consistency And Attention Mechanism, Hao Wang, Doyen Sahoo, Chenghao Liu, Ke Shu, Palakorn Achananuparp, Ee-Peng Lim, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Food retrieval is an important task to perform analysis of food-related information, where we are interested in retrieving relevant information about the queried food item such as ingredients, cooking instructions, etc. In this paper, we investigate cross-modal retrieval between food images and cooking recipes. The goal is to learn an embedding of images and recipes in a common feature space, such that the corresponding image-recipe embeddings lie close to one another. Two major challenges in addressing this problem are 1) large intra-variance and small inter-variance across cross-modal food data; and 2) difficulties in obtaining discriminative recipe representations. To address these …

Evaluating Technology-Mediated Collaborative Workflows For Telehealth, Christopher Bondy Ph.D., Pengcheng Shi, Pamela Grover Md, Vicki Hanson, Linlin Chen, Rui Li

Articles

Goals: This paper discusses the need for a predictable method to evaluate gains and gaps of collaborative technology-mediated workflows and introduces an evaluation framework to address this need. Methods: The Collaborative Space Analysis Framework (CS-AF), introduced in this research, is a cross-disciplinary evaluation method designed to evaluate technology-mediated collaborative workflows. The 5-step CS-AF approach includes: (1) current-state workflow definition, (2) current-state (baseline) workflow assessment, (3) technology-mediated workflow development and deployment, (4) technology-mediated workflow assessment, (5) analysis, and conclusions. For this research, a comprehensive, empirical study of hypertension exam workflow for telehealth was conducted using the CS-AF approach. Results: The CS-AF …

Fine-Grained Generalization Analysis Of Inductive Matrix Completion, Antoine Ledent, Rodrigo Alves, Yunwen Lei, Marius Kloft

Research Collection School Of Computing and Information Systems

In this paper, we bridge the gap between the state-of-the-art theoretical results for matrix completion with the nuclear norm and their equivalent in \textit{inductive matrix completion}: (1) In the distribution-free setting, we prove bounds improving the previously best scaling of \widetilde{O}(rd2) to \widetilde{O}(d3/2√r), where d is the dimension of the side information and rr is the rank. (2) We introduce the (smoothed) \textit{adjusted trace-norm minimization} strategy, an inductive analogue of the weighted trace norm, for which we show guarantees of the order \widetilde{O}(dr) under arbitrary sampling. In the inductive case, a similar rate was previously achieved only under uniform sampling …

Vireo @ Trecvid 2021 Ad-Hoc Video Search, Jiaxin Wu, Phuong Anh Nguyen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

In this paper, we summarize our submitted runs and results for Ad-hoc Video Search (AVS) task at TRECVid 2020

Self-Supervised Learning Disentangled Group Representation As Feature, Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, Hanwang Zhang

Research Collection School Of Computing and Information Systems

A good visual representation is an inference map from observations (images) to features (vectors) that faithfully reflects the hidden modularized generative factors (semantics). In this paper, we formulate the notion of “good” representation from a group-theoretic view using Higgins’ definition of disentangled representation [38], and show that existing Self-Supervised Learning (SSL) only disentangles simple augmentation features such as rotation and colorization, thus unable to modularize the remaining semantics. To break the limitation, we propose an iterative SSL algorithm: Iterative Partition-based Invariant Risk Minimization (IP-IRM), which successfully grounds the abstract semantics and the group acting on them into concrete contrastive learning. …

Mapping E-Commerce Locally And Beyond: Citt K12 Special Investigation Project, Thomas O’Brien, Deanna Matsumoto

Mineta Transportation Institute

As all aspects of the American workplace become automated or digitally enhanced to some degree, K12 educators have an increasing responsibility to help their students acquire the technical skills necessary to organize and interpret information. Increasingly, this is done through Geographic Information Systems (GIS), especially in careers related to transportation and logistics. The Center for International Trade & Transportation (CITT) at CSU Long Beach has developed this K12 Special Investigation Project to introduce ArcGIS StoryMaps, an engaging, accessible and sophisticated web-based GIS application. The lessons center on e-commerce and its accompanying environmental and economic impact. Still, the activities can be …

Learning To Teach And Learn For Semi-Supervised Few-Shot Image Classification, Xinzhe Li, Jianqiang Huang, Yaoyao Liu, Qin Zhou, Shibao Zheng, Bernt Schiele, Qianru Sun

Research Collection School Of Computing and Information Systems

This paper presents a novel semi-supervised few-shot image classification method named Learning to Teach and Learn (LTTL) to effectively leverage unlabeled samples in small-data regimes. Our method is based on self-training, which assigns pseudo labels to unlabeled data. However, the conventional pseudo-labeling operation heavily relies on the initial model trained by using a handful of labeled data and may produce many noisy labeled samples. We propose to solve the problem with three steps: firstly, cherry-picking searches valuable samples from pseudo-labeled data by using a soft weighting network; and then, cross-teaching allows the classifiers to teach mutually for rejecting more noisy …

Exploiting Reasoning Chains For Multi-Hop Science Question Answering, Weiwen Xu, Yang Deng, Huihui Zhang, Deng Cai, Wai Lam

Research Collection School Of Computing and Information Systems

We propose a novel Chain Guided Retriever reader (CGR) framework to model the reasoning chain for multi-hop Science Question Answering. Our framework is capable of performing explainable reasoning without the need of any corpus-specific annotations, such as the ground-truth reasoning chain, or human annotated entity mentions. Specifically, we first generate reasoning chains from a semantic graph constructed by Abstract Meaning Representation of retrieved evidence facts. A Chain-aware loss, concerning both local and global chain information, is also designed to enable the generated chains to serve as distant supervision signals for training the retriever, where reinforcement learning is also adopted to …

Aixfood'21: 3rd Workshop On Aixfood, Ricardo Guerrero, Michael Spranger, Shuqiang Jiang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Food and cooking analysis present exciting research and application challenges for modern AI systems, particularly in the context of multimodal data such as images or video. A meal that appears in a food image is a product of a complex progression of cooking stages, often described in the accompanying textual recipe form. In the cooking process, individual ingredients change their physical properties, become combined with other food components, all to produce a final, yet highly variable, appearance of the meal. Recognizing food items or meals on a plate from images or videos, their physical properties such as the amount, nutritional …

Differentiated Learning For Multi-Modal Domain Adaptation, Jianming Lv, Kaijie Liu, Shengfeng He

Research Collection School Of Computing and Information Systems

Directly deploying a trained multi-modal classifier to a new environment usually leads to poor performance due to the well-known domain shift problem. Existing multi-modal domain adaptation methods treated each modality equally and optimize the sub-models of different modalities synchronously. However, as observed in this paper, the degrees of domain shift in different modalities are usually diverse. We propose a novel Differentiated Learning framework to make use of the diversity between multiple modalities for more effective domain adaptation. Specifically, we model the classifiers of different modalities as a group of teacher/student sub-models, and a novel Prototype based Reliability Measurement is presented …

Prediction Of Synthetic Lethal Interactions In Human Cancers Using Multi-View Graph Auto-Encoder, Zhifeng Hao, Di Wu, Yuan Fang, Min Wu, Ruichu Cai, Xiaoli Li

Research Collection School Of Computing and Information Systems

Synthetic lethality (SL) is a very important concept for the development of targeted anticancer drugs. However, experimental methods for SL detection often suffer from various issues like high cost and low consistency across cell lines. Hence, computational methods for predicting novel SLs have recently emerged as complements for wet-lab experiments. In addition, SL data can be represented as a graph where nodes are genes and edges are the SL interactions. It is thus motivated to design advanced graph-based machine learning algorithms for SL prediction. In this paper, we propose a novel SL prediction method using Multi-view Graph Auto-Encoder (SLMGAE). We …

A Large-Scale Benchmark For Food Image Segmentation, Xiongwei Wu, Xin Fu, Ying Liu, Ee-Peng Lim, Steven C. H. Hoi, Qianru Sun

Research Collection School Of Computing and Information Systems

Food image segmentation is a critical and indispensible task for developing health-related applications such as estimating food calories and nutrients. Existing food image segmentation models are underperforming due to two reasons: (1) there is a lack of high quality food image datasets with fine-grained ingredient labels and pixel-wise location masks—the existing datasets either carry coarse ingredient labels or are small in size; and (2) the complex appearance of food makes it difficult to localize and recognize ingredients in food images, e.g., the ingredients may overlap one another in the same image, and the identical ingredient may appear distinctly in different …

Visilence: An Interactive Visualization Tool For Error Resilience Analysis, Shaolun Ruan, Yong Wang, Qiang Guan

Research Collection School Of Computing and Information Systems

Soft errors have become one of the major concerns for HPC applications, as those errors can result in seriously corrupted outcomes, such as silent data corruptions (SDCs). Prior studies on error resilience have studied the robustness of HPC applications. However, it is still difficult for program developers to identify potential vulnerability to soft errors. In this paper, we present Visilence, a novel visualization tool to visually analyze error vulnerability based on the control-flow graph generated from HPC applications. Visilence efficiently visualizes the affected program states under injected errors and presents the visual analysis of the most vulnerable parts of an …

Condensing A Sequence To One Informative Frame For Video Recognition, Qiu. Zhaofan, Ting Yao, Yan Shu, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Video is complex due to large variations in motion and rich content in fine-grained visual details. Abstracting useful information from such information-intensive media requires exhaustive computing resources. This paper studies a two-step alternative that first condenses the video sequence to an informative" frame" and then exploits off-the-shelf image recognition system on the synthetic frame. A valid question is how to define" useful information" and then distill it from a video sequence down to one synthetic frame. This paper presents a novel Informative Frame Synthesis (IFS) architecture that incorporates three objective tasks, ie, appearance reconstruction, video categorization, motion estimation, and two …

Semi-Supervised Semantic Visualization For Networked Documents, Delvin Ce Zhang, Hady W. Lauw

Research Collection School Of Computing and Information Systems

Semantic interpretability and visual expressivity are important objectives in exploratory analysis of text. On the one hand, while some documents may have explicit categories, we could develop a better understanding of a corpus by studying its finer-grained structures, which may be latent. By inferring latent topics and discovering keywords associated with each topic, one obtains a semantic interpretation of the corpus. One the other hand, by visualizing documents, latent topics, and category labels on the same plot, one gains a bird’s eye view of the relationships among documents, topics, and various categories. Semantic visualization is a class of methods that …

An Automated Method To Enrich Consumer Health Vocabularies Using Glove Word Embeddings And An Auxiliary Lexical Resource, Mohammed Ibrahim, Susan Gauch, Omar Salman, Mohammed Alqahtani

Computer Science and Computer Engineering Faculty Publications and Presentations

Background

Clear language makes communication easier between any two parties. A layman may have difficulty communicating with a professional due to not understanding the specialized terms common to the domain. In healthcare, it is rare to find a layman knowledgeable in medical terminology which can lead to poor understanding of their condition and/or treatment. To bridge this gap, several professional vocabularies and ontologies have been created to map laymen medical terms to professional medical terms and vice versa.

Objective

Many of the presented vocabularies are built manually or semi-automatically requiring large investments of time and human effort and consequently the …

Transitioning From Vue 2 To Vue 3, Adele Kanley

Theses/Capstones/Creative Projects

Frontend development is a field that is constantly changing because of the vast amounts of tools that are made available each year. One of the most popular frameworks being utilized to create fluid user experience is the Vue framework. Branching from the well-known Angular.js, Vue.js is an independent open-source project that is making its mark in the user interface community.

Regardless of the popularity of a framework, updates are inevitable to keep up with the innovations required by the IT Field. To ensure that UNO IS&T students are being offered opportunities to learn and develop in the most update to …

Forecasting Interaction Order On Temporal Graphs, Wenwen Xia, Yuchen Li, Jianwei Tian, Shenghong Li

Research Collection School Of Computing and Information Systems

Link prediction is a fundamental task for graph analysis and the topic has been studied extensively for static or dynamic graphs. Essentially, the link prediction is formulated as a binary classification problem about two nodes. However, for temporal graphs, links (or interactions) among node sets appear in sequential orders. And the orders may lead to interesting applications. While a binary link prediction formulation fails to handle such an order-sensitive case. In this paper, we focus on such an interaction order prediction (IOP) problem among a given node set on temporal graphs. For the technical aspect, we develop a graph neural …

Learning From Miscellaneous Other-Class Words For Few-Shot Named Entity Recognition, Meihan Tong, Shuai Wang, Bin Xu, Yixin Cao, Minghui Liu, Lei Hou, Juanzi Li

Research Collection School Of Computing and Information Systems

Few-shot Named Entity Recognition (NER) exploits only a handful of annotations to identify and classify named entity mentions. Prototypical network shows superior performance on few-shot NER. However, existing prototypical methods fail to differentiate rich semantics in other-class words, which will aggravate overfitting under few shot scenario. To address the issue, we propose a novel model, Mining Undefined Classes from Other-class (MUCO), that can automatically induce different undefined classes from the other class to improve few-shot NER. With these extra-labeled undefined classes, our method will improve the discriminative ability of NER classifier and enhance the understanding of predefined classes with stand-by …