Graphics and Human Computer Interfaces Commons^™

Advancing Food Nutrition Estimation Via Visual-Ingredient Feature Fusion, Huiyan Qi, Bin Zhu, Chong-Wah Ngo, Jingjing Chen, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

Nutrition estimation is an important component of promoting healthy eating and mitigating diet-related health risks. Despite advances in tasks such as food classification and ingredient recognition, progress in nutrition estimation is limited due to the lack of datasets with nutritional annotations. To address this issue, we introduce FastFood, a dataset with 84,446 images across 908 fast food categories, featuring ingredient and nutritional annotations. In addition, we propose a new model-agnostic Visual-Ingredient Feature Fusion (VIF2 ) method to enhance nutrition estimation by integrating visual and ingredient features. Ingredient robustness is improved through synonym replacement and resampling strategies during training. The ingredient-aware …

Reducing Food Scarcity: The Benefits Of Urban Farming, S.A. Claudell, Emilio Mejia

Journal of Nonprofit Innovation

Urban farming can enhance the lives of communities and help reduce food scarcity. This paper presents a conceptual prototype of an efficient urban farming community that can be scaled for a single apartment building or an entire community across all global geoeconomics regions, including densely populated cities and rural, developing towns and communities. When deployed in coordination with smart crop choices, local farm support, and efficient transportation then the result isn’t just sustainability, but also increasing fresh produce accessibility, optimizing nutritional value, eliminating the use of ‘forever chemicals’, reducing transportation costs, and fostering global environmental benefits.

Imagine Doris, who is …

Iot Health Devices: Exploring Security Risks In The Connected Landscape, Abasi-Amefon Obot Affia, Hilary Finch, Woosub Jung, Issah Abubakari Samori, Lucas Potter, Xavier-Lewis Palmer

School of Cybersecurity Faculty Publications

The concept of the Internet of Things (IoT) spans decades, and the same can be said for its inclusion in healthcare. The IoT is an attractive target in medicine; it offers considerable potential in expanding care. However, the application of the IoT in healthcare is fraught with an array of challenges, and also, through it, numerous vulnerabilities that translate to wider attack surfaces and deeper degrees of damage possible to both consumers and their confidence within health systems, as a result of patient-specific data being available to access. Further, when IoT health devices (IoTHDs) are developed, a diverse range of …

Chatgpt As Metamorphosis Designer For The Future Of Artificial Intelligence (Ai): A Conceptual Investigation, Amarjit Kumar Singh (Library Assistant), Dr. Pankaj Mathur (Deputy Librarian)

Library Philosophy and Practice (e-journal)

Abstract

Purpose: The purpose of this research paper is to explore ChatGPT’s potential as an innovative designer tool for the future development of artificial intelligence. Specifically, this conceptual investigation aims to analyze ChatGPT’s capabilities as a tool for designing and developing near about human intelligent systems for futuristic used and developed in the field of Artificial Intelligence (AI). Also with the helps of this paper, researchers are analyzed the strengths and weaknesses of ChatGPT as a tool, and identify possible areas for improvement in its development and implementation. This investigation focused on the various features and functions of ChatGPT that …

Hybrid Life Cycles In Software Development, Eric Vincent Schoenborn

Masters Projects

This project applied software specification gathering, architecture, work planning, and development to a real-world development effort for a local business. This project began with a feasibility meeting with the owner of Zeal Aerial Fitness. After feasibility was assessed the intended users, needed functionality, and expected user restrictions were identified with the stakeholders. A hybrid software lifecycle was selected to allow a focus on base functionality up front followed by an iterative development of expectations of the stakeholders. I was able to create various specification diagrams that express the end projects goals to both developers and non-tech individuals using a standard …

Spotlight Report #6: Proffering Machine-Readable Personal Privacy Research Agreements: Pilot Project Findings For Ieee P7012 Wg, Noreen Y. Whysel, Lisa Levasseur

Publications and Research

What if people had the ability to assert their own legally binding permissions for data collection, use, sharing, and retention by the technologies they use? The IEEE P7012 has been working on an interoperability specification for machine-readable personal privacy terms to support this ability since 2018. The premise behind the work of IEEE P7012 is that people need technology that works on their behalf—i.e. software agents that assert the individual’s permissions and preferences in a machine-readable format.

Thanks to a grant from the IEEE Technical Activities Board Committee on Standards (TAB CoS), we were able to explore the attitudes of …

Analysis Of Gpu Memory Vulnerabilities, Jarrett Hoover

Computer Science and Computer Engineering Undergraduate Honors Theses

Graphics processing units (GPUs) have become a widely used technology for various purposes. While their intended use is accelerating graphics rendering, their parallel computing capabilities have expanded their use into other areas. They are used in computer gaming, deep learning for artificial intelligence and mining cryptocurrencies. Their rise in popularity led to research involving several security aspects, including this paper’s focus, memory vulnerabilities. Research documented many vulnerabilities, including GPUs not implementing address space layout randomization, not zeroing out memory after deallocation, and not initializing newly allocated memory. These vulnerabilities can lead to a victim’s sensitive data being leaked to an …

Designing Respectful Tech: What Is Your Relationship With Technology?, Noreen Y. Whysel

Publications and Research

According to research at the Me2B Alliance, people feel they have a relationship with technology. It’s emotional. It’s embodied. And it’s very personal. We are studying digital relationships to answer questions like “Do people have a relationship with technology?” “What does that relationship feel like?” And “Do people understand the commitments that they are making when they explore, enter into and dissolve these relationships?” There are parallels between messy human relationships and the kinds of relationships that people develop with technology. As with human relationships, we move through states of discovery, commitment and breakup with digital applications as well. Technology …

An Automated Method To Enrich Consumer Health Vocabularies Using Glove Word Embeddings And An Auxiliary Lexical Resource, Mohammed Ibrahim, Susan Gauch, Omar Salman, Mohammed Alqahtani

Computer Science and Computer Engineering Faculty Publications and Presentations

Background

Clear language makes communication easier between any two parties. A layman may have difficulty communicating with a professional due to not understanding the specialized terms common to the domain. In healthcare, it is rare to find a layman knowledgeable in medical terminology which can lead to poor understanding of their condition and/or treatment. To bridge this gap, several professional vocabularies and ontologies have been created to map laymen medical terms to professional medical terms and vice versa.

Objective

Many of the presented vocabularies are built manually or semi-automatically requiring large investments of time and human effort and consequently the …

Interactive Search Vs. Automatic Search: An Extensive Study On Video Retrieval, Phuong-Anh Nguyen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

This article conducts user evaluation to study the performance difference between interactive and automatic search. Particularly, the study aims to provide empirical insights of how the performance landscape of video search changes, with tens of thousands of concept detectors freely available to exploit for query formulation. We compare three types of search modes: free-to-play (i.e., search from scratch), non-free-to-play (i.e., search by inspecting results provided by automatic search), and automatic search including concept-free and concept-based retrieval paradigms. The study involves a total of 40 participants; each performs interactive search over 15 queries of various difficulty levels using two search modes …

Mapping Renewal: How An Unexpected Interdisciplinary Collaboration Transformed A Digital Humanities Project, Elise Tanner, Geoffrey Joseph

Digital Initiatives Symposium

Funded by a National Endowment for Humanities (NEH) Humanities Collections and Reference Resources Foundations Grant, the UA Little Rock Center for Arkansas History and Culture’s “Mapping Renewal” pilot project focused on creating access to and providing spatial context to archival materials related to racial segregation and urban renewal in the city of Little Rock, Arkansas, from 1954-1989. An unplanned interdisciplinary collaboration with the UA Little Rock Arkansas Economic Development Institute (AEDI) has proven to be an invaluable partnership. One team member from each department will demonstrate the Mapping Renewal website and discuss how the collaborative process has changed and shaped …

Transferring And Regularizing Prediction For Semantic Segmentation, Yiheng Zhang, Zhaofan Qiu, Ting Yao, Chong-Wah Ngo, Dong Liu, Tao Mei

Research Collection School Of Computing and Information Systems

Semantic segmentation often requires a large set of images with pixel-level annotations. In the view of extremely expensive expert labeling, recent research has shown that the models trained on photo-realistic synthetic data (e.g., computer games) with computer-generated annotations can be adapted to real images. Despite this progress, without constraining the prediction on real images, the models will easily overfit on synthetic data due to severe domain mismatch. In this paper, we novelly exploit the intrinsic properties of semantic segmentation to alleviate such problem for model transfer. Specifically, we present a Regularizer of Prediction Transfer (RPT) that imposes the intrinsic properties …

R2gan: Cross-Modal Recipe Retrieval With Generative Adversarial Network, Bin Zhu, Chong-Wah Ngo, Jingjing Chen, Yanbin Hao

Research Collection School Of Computing and Information Systems

Representing procedure text such as recipe for crossmodal retrieval is inherently a difficult problem, not mentioning to generate image from recipe for visualization. This paper studies a new version of GAN, named Recipe Retrieval Generative Adversarial Network (R2GAN), to explore the feasibility of generating image from procedure text for retrieval problem. The motivation of using GAN is twofold: learning compatible cross-modal features in an adversarial way, and explanation of search results by showing the images generated from recipes. The novelty of R2GAN comes from architecture design, specifically a GAN with one generator and dual discriminators is used, which makes the …

Vireo @ Video Browser Showdown 2019, Phuong Anh Nguyen, Chong-Wah Ngo, Danny Francis, Benoit Huet

Research Collection School Of Computing and Information Systems

In this paper, the VIREO team video retrieval tool is described in details. As learned from Video Browser Showdown (VBS) 2018, the visualization of video frames is a critical need to improve the browsing effectiveness. Based on this observation, a hierarchical structure that represents the video frame clusters has been built automatically using k-means and self-organizing-map and used for visualization. Also, the relevance feedback module which relies on real-time supportvector-machine classification becomes unfeasible with the large dataset provided in VBS 2019 and has been replaced by a browsing module with pre-calculated nearest neighbors. The preliminary user study results on IACC.3 …

Semantic Reasoning In Zero Example Video Event Retrieval, M. H. T. De Boer, Yi-Jie Lu, Hao Zhang, Klamer Schutte, Chong-Wah Ngo, Wessel Kraaij

Research Collection School Of Computing and Information Systems

Searching in digital video data for high-level events, such as a parade or a car accident, is challenging when the query is textual and lacks visual example images or videos. Current research in deep neural networks is highly beneficial for the retrieval of high-level events using visual examples, but without examples it is still hard to (1) determine which concepts are useful to pre-train (Vocabulary challenge) and (2) which pre-trained concept detectors are relevant for a certain unseen high-level event (Concept Selection challenge). In our article, we present our Semantic Event Retrieval Systemwhich (1) shows the importance of high-level concepts …

Cross-Modal Recipe Retrieval With Rich Food Attributes, Jingjing Chen, Chong-Wah Ngo, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Food is rich of visible (e.g., colour, shape) and procedural (e.g., cutting, cooking) attributes. Proper leveraging of these attributes, particularly the interplay among ingredients, cutting and cooking methods, for health-related applications has not been previously explored. This paper investigates cross-modal retrieval of recipes, specifically to retrieve a text-based recipe given a food picture as query. As similar ingredient composition can end up with wildly different dishes depending on the cooking and cutting procedures, the difficulty of retrieval originates from fine-grained recognition of rich attributes from pictures. With a multi-task deep learning model, this paper provides insights on the feasibility of …

Learning Query And Image Similarities With Ranking Canonical Correlation Analysis, Ting Yao, Tao Mei, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

One of the fundamental problems in image search is to learn the ranking functions, i.e., similarity between the query and image. The research on this topic has evolved through two paradigms: feature-based vector model and image ranker learning. The former relies on the image surrounding texts, while the latter learns a ranker based on human labeled query-image pairs. Each of the paradigms has its own limitation. The vector model is sensitive to the quality of text descriptions, and the learning paradigm is difficult to be scaled up as human labeling is always too expensive to obtain. We demonstrate in this …

Direct Or Indirect Match? Selecting Right Concepts For Zero-Example Case, Yi-Jie Lu, Maaike De Boer, Hao Zhang, Klamer Schutte, Wessel Kraaij, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

No abstract provided.

Deep Multimodal Learning For Affective Analysis And Retrieval, Lei Pang, Shiai Zhu, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Social media has been a convenient platform for voicing opinions through posting messages, ranging from tweeting a short text to uploading a media file, or any combination of messages. Understanding the perceived emotions inherently underlying these user-generated contents (UGC) could bring light to emerging applications such as advertising and media analytics. Existing research efforts on affective computation are mostly dedicated to single media, either text captions or visual content. Few attempts for combined analysis of multiple media are made, despite that emotion can be viewed as an expression of multimodal experience. In this paper, we explore the learning of highly …

Vireo-Tno @ Trecvid 2015: Multimedia Event Detection, Hao Zhang, Yi-Jie Lu, Maaike De Boer, Frank Ter Haar, Zhaofan Qiu, Klamer Schutte, Wessel Kraaij, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

This paper presents an overview and comparative analysis of our systems designed for the TRECVID 2015 [1] multimedia event detection (MED) task. We submitted 17 runs, of which 5 each for the zeroexample, 10-example and 100-example subtasks for the Pre-Specified (PS) event detection and 2 runs for the 10-example subtask for the Ad-Hoc (AH) event detection. We did not participate in the Interactive Run. This year we focus on three different parts of the MED task: 1) extending the size of our concept bank and combining it with improved dense trajectories; 2) exploring strategies for semantic query generation (SQG); and …

Improving Automatic Name-Face Association Using Celebrity Images On The Web, Zhineng Chen, Bailan Feng, Chong-Wah Ngo, Caiyan Jia, Xiangsheng Huang

Research Collection School Of Computing and Information Systems

This paper investigates the task of automatically associating faces appearing in images (or videos) with their names. Our novelty lies in the use of celebrity Web images to facilitate the task. Specifically, we first propose a method named Image Matching (IM), which uses the faces in images returned from name queries over an image search engine as the gallery set of the names, and a probe face is classified as one of the names, or none of them, according to their matching scores and compatibility characterized by a proposed Assigning-Thresholding (AT) pipeline. Noting IM could provide guidance for association for …

Click-Boosting Multi-Modality Graph-Based Reranking For Image Search, Xiaopeng Yang, Yongdong Zhang, Ting Yao, Chong-Wah Ngo, Tao Mei

Research Collection School Of Computing and Information Systems

Image reranking is an effective way for improving the retrieval performance of keyword-based image search engines. A fundamental issue underlying the success of existing image reranking approaches is the ability in identifying potentially useful recurrent patterns from the initial search results. Ideally, these patterns can be leveraged to upgrade the ranks of visually similar images, which are also likely to be relevant. The challenge, nevertheless, originates from the fact that keyword-based queries are used to be ambiguous, resulting in difficulty in predicting the search intention. Mining useful patterns without understanding query is risky, and may lead to incorrect judgment in …

Click-Through-Based Subspace Learning For Image Search, Yingwei Pan, Ting Yao, Xinmei Tian, Houqiang Li, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

One of the fundamental problems in image search is to rank image documents according to a given textual query. We address two limitations of the existing image search engines in this paper. First, there is no straightforward way of comparing textual keywords with visual image content. Image search engines therefore highly depend on the surrounding texts, which are often noisy or too few to accurately describe the image content. Second, ranking functions are trained on query-image pairs labeled by human labelers, making the annotation intellectually expensive and thus cannot be scaled up. We demonstrate that the above two fundamental challenges …

Vireo @ Trecvid 2014: Instance Search And Semantic Indexing, Wei Zhang, Hao Zhang, Ting Yao, Yijie Lu, Jingjing Chen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

This paper summarizes the following two tasks participated by VIREO group: instance search and semantic indexing. We will present our approaches and analyze the results obtained in TRECVID 2014 benchmark evaluation

Organizing Video Search Results To Adapted Semantic Hierarchies For Topic-Based Browsing, Jiajun Wang, Yu-Gang Jiang, Qiang Wang, Kuiyuan Yang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Organizing video search results into semantically structured hierarchies can greatly improve the efficiency of browsing complex query topics. Traditional hierarchical clustering techniques are inadequate since they lack the ability to generate semantically interpretable structures. In this paper, we introduce an approach to organize video search results to an adapted semantic hierarchy. As many hot search topics such as celebrities and famous cities have Wikipedia pages where hierarchical topic structures are available, we start from the Wikipedia hierarchies and adjust the structures according to the characteristics of the returned videos from a search engine. Ordinary clustering based on textual information of …

Name-Face Association In Web Videos: A Large-Scale Dataset, Baselines, And Open Issues, Zhi-Neng Chen, Chong-Wah Ngo, Wei Zhang, Juan Cao, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Associating faces appearing in Web videos with names presented in the surrounding context is an important task in many applications. However, the problem is not well investigated particularly under large-scale realistic scenario, mainly due to the scarcity of dataset constructed in such circumstance. In this paper, we introduce a Web video dataset of celebrities, named WebV-Cele, for name-face association. The dataset consists of 75 073 Internet videos of over 4 000 hours, covering 2 427 celebrities and 649 001 faces. This is, to our knowledge, the most comprehensive dataset for this problem. We describe the details of dataset construction, discuss …

Click-Through-Based Cross-View Learning For Image Search, Yingwei Pan, Ting Yao, Tao Mei, Houqiang Li, Chong-Wah Ngo, Yong Rui

Research Collection School Of Computing and Information Systems

One of the fundamental problems in image search is to rank image documents according to a given textual query. Existing search engines highly depend on surrounding texts for ranking images, or leverage the query-image pairs annotated by human labelers to train a series of ranking functions. However, there are two major limitations: 1) the surrounding texts are often noisy or too few to accurately describe the image content, and 2) the human annotations are resourcefully expensive and thus cannot be scaled up. We demonstrate in this paper that the above two fundamental challenges can be mitigated by jointly exploring the …

Placing Videos On A Semantic Hierarchy For Search Result Navigation, Song Tan, Yu-Gang Jiang, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Organizing video search results in a list view is widely adopted by current commercial search engines, which cannot support efficient browsing for complex search topics that have multiple semantic facets. In this article, we propose to organize video search results in a highly structured way. Specifically, videos are placed on a semantic hierarchy that accurately organizes various facets of a given search topic. To pick the most suitable videos for each node of the hierarchy, we define and utilize three important criteria: relevance, uniqueness, and diversity. Extensive evaluations on a large YouTube video dataset demonstrate the effectiveness of our approach.

Image Search By Graph-Based Label Propagation With Image Representation From Dnn, Yingwei Pan, Yao Ting, Kuiyuan Yang, Houqiang Li, Chong-Wah Ngo, Jingdong Wang, Tao Mei

Research Collection School Of Computing and Information Systems

Our objective is to estimate the relevance of an image to a query for image search purposes. We address two limitations of the existing image search engines in this paper. First, there is no straightforward way of bridging the gap between semantic textual queries as well as users’ search intents and image visual content. Image search engines therefore primarily rely on static and textual features. Visual features are mainly used to identify potentially useful recurrent patterns or relevant training examples for complementing search by image reranking. Second, image rankers are trained on query-image pairs labeled by human experts, making the …

Error Recovered Hierarchical Classification, Shiai Zhu, Xiao-Yong Wei, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Hierarchical classification (HC) is a popular and efficient way for detecting the semantic concepts from the images. However, the conventional HC, which always selects the branch with the highest classification response to go on, has the risk of propagating serious errors from higher levels of the hierarchy to the lower levels. We argue that the highestresponse-first strategy is too arbitrary, because the candidate nodes are considered individually which ignores the semantic relationship among them. In this paper, we propose a novel method for HC, which is able to utilize the semantic relationship among candidate nodes and their children to recover …