Digital Commons Network^™

Cross-Thought For Sentence Encoder Pre-Training, Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jingjing Liu, Jing Jiang

Research Collection School Of Computing and Information Systems

In this paper, we propose Cross-Thought, a novel approach to pre-training sequence encoder, which is instrumental in building reusable sequence embeddings for large-scale NLP tasks such as question answering. Instead of using the original signals of full sentences, we train a Transformer-based sequence encoder over a large set of short sequences, which allows the model to automatically select the most useful information for predicting masked words. Experiments on question answering and textual entailment tasks demonstrate that our pre-trained encoder can outperform state-of-the-art encoders trained with continuous sentence signals as well as traditional masked language modeling baselines. Our proposed approach also …

Leveraging Profanity For Insincere Content Detection: A Neural Network Approach, Swapna Gottipati, Annabel Tan, David Jing Shan Chow, Joel Wee Kiat Lim

Research Collection School Of Computing and Information Systems

Community driven social media sites are rich sources of knowledge and entertainment and at the same vulnerable to the flames or toxic content that can be dangerous to various users of these platforms as well as to the society. Therefore, it is crucial to identify and remove such content to have a better and safe online experience. Manually eliminating flames is tedious and hence many research works focus on machine learning or deep learning models for automated methods. In this paper, we primarily focus on detecting the insincere content using neural network-based learning methods. We also integrated the profanity features …

Uniconv: A Unified Conversational Neural Architecture For Multi-Domain Task-Oriented Dialogues, Hung Le, Doyen Sahoo, Chenghao Liu, Nancy F. Chen, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Building an end-to-end conversational agent for multi-domain task-oriented dialogues has been an open challenge for two main reasons. First, tracking dialogue states of multiple domains is non-trivial as the dialogue agent must obtain complete states from all relevant domains, some of which might have shared slots among domains as well as unique slots specifically for one domain only. Second, the dialogue agent must also process various types of information across domains, including dialogue context, dialogue states, and database, to generate natural responses to users. Unlike the existing approaches that are often designed to train each module separately, we propose “UniConv" …

Bist: Bi-Directional Spatio-Temporal Reasoning For Video-Grounded Dialogues, Hung Le, Doyen Sahoo, Nancy F. Chen, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Video-grounded dialogues are very challenging due to (i) the complexity of videos which contain both spatial and temporal variations, and (ii) the complexity of user utterances which query different segments and/or different objects in videos over multiple dialogue turns. However, existing approaches to video-grounded dialogues often focus on superficial temporal-level visual cues, but neglect more fine-grained spatial signals from videos. To address this drawback, we propose Bi-directional Spatio-Temporal Learning (BiST), a vision-language neural framework for high-resolution queries in videos based on textual cues. Specifically, our approach not only exploits both spatial and temporal-level information, but also learns dynamic information diffusion …

Deepcommenter: A Deep Code Comment Generation Tool With Hybrid Lexical And Syntactical Information, Boao Li, Meng Yan, Xin Xia, Xing Hu, Ge Li, David Lo

Research Collection School Of Computing and Information Systems

As the scale of software projects increases, the code comments are more and more important for program comprehension. Unfortunately, many code comments are missing, mismatched or outdated due to tight development schedule or other reasons. Automatic code comment generation is of great help for developers to comprehend source code and reduce their workload. Thus, we propose a code comment generation tool (DeepCommenter) to generate descriptive comments for Java methods. DeepCommenter formulates the comment generation task as a machine translation problem and exploits a deep neural network that combines the lexical and structural information of Java methods. We implement DeepCommenter in …

Api Method Recommendation Via Explicit Matching Of Functionality Verb Phrases, Wenkai Xie, Xin Peng, Mingwei Liu, Christoph Treude, Zhenchang Xing, Xiaoxin Zhang, Wenyun Zhao

Research Collection School Of Computing and Information Systems

Due to the lexical gap between functionality descriptions and user queries, documentation-based API retrieval often produces poor results. Verb phrases and their phrase patterns are essential in both describing API functionalities and interpreting user queries. Thus we hypothesize that API retrieval can be facilitated by explicitly recognizing and matching between the fine-grained structures of functionality descriptions and user queries. To verify this hypothesis, we conducted a large-scale empirical study on the functionality descriptions of 14,733 JDK and Android API methods. We identified 356 different functionality verbs from the descriptions, which were grouped into 87 functionality categories, and we extracted 523 …

Beyond Accuracy: Assessing Software Documentation Quality, Christoph Treude, Justin Middleton, Thushari Atapattu

Research Collection School Of Computing and Information Systems

Good software documentation encourages good software engineering, but the meaning of “good” documentation is vaguely defined in the software engineering literature. To clarify this ambiguity, we draw on work from the data and information quality community to propose a framework that decomposes documentation quality into ten dimensions of structure, content, and style. To demonstrate its application, we recruited technical editors to apply the framework when evaluating examples from several genres of software documentation. We summarise their assessments—for example, reference documentation and README files excel in quality whereas blog articles have more problems—and we describe our vision for reasoning about software …

Exploring And Evaluating Attributes, Values, And Structures For Entity Alignment, Zhiyuan Liu, Yixin Cao, Liangming Pan, Juanzi Li, Zhiyuan Liu, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Entity alignment (EA) aims at building a unified Knowledge Graph (KG) of rich content by linking the equivalent entities from various KGs. GNN-based EA methods present promising performance by modeling the KG structure defined by relation triples. However, attribute triples can also provide crucial alignment signal but have not been well explored yet. In this paper, we propose to utilize an attributed value encoder and partition the KG into subgraphs to model the various types of attribute triples efficiently. Besides, the performances of current EA methods are overestimated because of the name-bias of existing EA datasets. To make an objective …

Answerfact: Fact Checking In Product Question Answering, Wenxuan Zhang, Yang Deng, Jing Ma, Wai Lam

Research Collection School Of Computing and Information Systems

Product-related question answering platforms nowadays are widely employed in many E-commerce sites, providing a convenient way for potential customers to address their concerns during online shopping. However, the misinformation in the answers on those platforms poses unprecedented challenges for users to obtain reliable and truthful product information, which may even cause a commercial loss in E-commerce business. To tackle this issue, we investigate to predict the veracity of answers in this paper and introduce AnswerFact, a large scale fact checking dataset from product question answering forums. Each answer is accompanied by its veracity label and associated evidence sentences, providing a …

Multi-Hop Inference For Question-Driven Summarization, Yang Deng, Wenxuan Zhang, Wai Lam

Research Collection School Of Computing and Information Systems

Question-driven summarization has been recently studied as an effective approach to summarizing the source document to produce concise but informative answers for non-factoid questions. In this work, we propose a novel question-driven abstractive summarization method, Multi-hop Selective Generator (MSG), to incorporate multi-hop reasoning into question-driven summarization and, meanwhile, provide justifications for the generated summaries. Specifically, we jointly model the relevance to the question and the interrelation among different sentences via a human-like multi-hop inference module, which captures important sentences for justifying the summarized answer. A gated selective pointer generator network with a multi-view coverage mechanism is designed to integrate diverse …

Developing An Ai-Powered Chatbot To Support The Administration Of Middle And High School Cybersecurity Camps, Jonathan He, Chunsheng Xin

KSU Proceedings on Cybersecurity Education, Research and Practice

Throughout the Internet, many chatbots have been deployed by various organizations to answer questions asked by customers. In recent years, we have been running cybersecurity summer camps for youth. Due to COVID-19, our in-person camp has been changed to virtual camps. As a result, we decided to develop a chatbot to reduce the number of emails, phone calls, as well as the human burden for answering the same or similar questions again and again based on questions we received from previous camps. This paper introduces our practical experience to implement an AI-powered chatbot for middle and high school cybersecurity camps …

Visual Feedback Fuzzy Control For A Robot Manipulator Based On Svr Learning, Xianxia Zhang, Jinqiang Zhang, Zhiyuan Li, Shiwei Ma, Banghua Yang

Journal of System Simulation

Abstract: A fuzzy controller based on SVR learning is proposed for uncalibrated robot visual servoing. In this paper, a fuzzy controller is used to directly construct the nonlinear mapping between image features and robot joint motion. The fuzzy basis function of the fuzzy controller is taken as the kernel function of an SVR and the equivalent relationship between the SVR and the fuzzy controller is established. The learned support vector from the SVR is used as the rule of the fuzzy controller. Since all rules are learned from the data, there is no need to manually design the rules. …

Gru-Based Car-Following Behavior Simulation Model, Fei Rong, Liu Fang, Xie Guo, Hei Xinhong, Shasha Li, Hu Bo

Journal of System Simulation

Abstract: The accuracy of acceleration prediction can be effectively improved by the driver's memory in car-following behavior. A new car-following model based on the General Motors (GM) and the gate control unit (GRU) is proposed. The car-following data between small vehicles with similar driving behavior are obtained by data preprocessing. The established model is calibrated by the car-following data, and the optimal parameters and structure of the model are determined. According to car-following characteristics, the effectiveness of model is verified by simulation. It is confirmed that the model has high robustness and improved simulation accuracy comparing with the traditional models.

Assessing Topical Homogeneity With Word Embedding And Distance Matrices, Jeffrey M. Stanton, Yisi Sang

School of Information Studies - Faculty Scholarship

Researchers from many fields have used statistical tools to make sense of large bodies of text. Many tools support quantitative analysis of documents within a corpus, but relatively few studies have examined statistical characteristics of whole corpora. Statistical summaries of whole corpora and comparisons between corpora have potential application in the analysis of topically organized applications such social media platforms. In this study, we created matrix representations of several corpora and examined several statistical tests to make comparisons between pairs of corpora with respect to the topical homogeneity of documents within each corpus. Results of three experiments suggested that a …

Crossasr: Efficient Differential Testing Of Automatic Speech Recognition Via Text-To-Speech, Muhammad Hilmi Asyrofi, Thung Ferdian, David Lo, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Automatic speech recognition (ASR) systems are ubiquitous parts of modern life. It can be found in our smartphones, desktops, and smart home systems. To ensure its correctness in recognizing speeches, ASR needs to be tested. Testing ASR requires test cases in the form of audio files and their transcribed texts. Building these test cases manually, however, is tedious and time-consuming.To deal with the aforementioned challenge, in this work, we propose CrossASR, an approach that capitalizes the existing Text-To-Speech (TTS) systems to automatically generate test cases for ASR systems. CrossASR is a differential testing solution that compares outputs of multiple ASR …

The Impact Of Dynamics Of Collaborative Software Engineering On Introverts: A Study Protocol, Ingrid Nunes, Christoph Treude, Fabio Calefato

Research Collection School Of Computing and Information Systems

Background: Collaboration among software engineers through face-to-face discussions in teams has been promoted since the adoption of agile methods. However, these discussions might demote the contribution of software engineers who are introverts, possibly leading to sub-optimal solutions and creating work environments that benefit extroverts. Objective: We aim to evaluate whether providing software engineers with time to work individually and reason about a collective problem is a setting that makes introverts more comfortable to interact and contribute more, ultimately leading to better solutions. Method: We plan to conduct a between-subjects study, with teams in a control group that design a software …

Knowledge Enhanced Neural Fashion Trend Forecasting, Yunshan Ma, Yujuan Ding, Xun Yang, Lizi Liao, Wai Keung Wong, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Fashion trend forecasting is a crucial task for both academia and industry. Although some efforts have been devoted to tackling this challenging task, they only studied limited fashion elements with highly seasonal or simple patterns, which could hardly reveal the real fashion trends. Towards insightful fashion trend forecasting, this work focuses on investigating fine-grained fashion element trends for specific user groups. We first contribute a large-scale fashion trend dataset (FIT) collected from Instagram with extracted time series fashion element records and user information. Furthermore, to effectively model the time series data of fashion elements with rather complex patterns, we propose …

Interpretable Embedding For Ad-Hoc Video Search, Jiaxin Wu, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

Answering query with semantic concepts has long been the mainstream approach for video search. Until recently, its performance is surpassed by concept-free approach, which embeds queries in a joint space as videos. Nevertheless, the embedded features as well as search results are not interpretable, hindering subsequent steps in video browsing and query reformulation. This paper integrates feature embedding and concept interpretation into a neural network for unified dual-task learning. In this way, an embedding is associated with a list of semantic concepts as an interpretation of video content. This paper empirically demonstrates that, by using either the embedding features or …

Federated Topic Discovery: A Semantic Consistent Approach, Yexuan Shi, Yongxin Tong, Zhiyang Su, Di Jiang, Zimu Zhou, Wenbin Zhang

Research Collection School Of Computing and Information Systems

General-purpose topic models have widespread industrial applications. Yet high-quality topic modeling is becoming increasingly challenging because accurate models require large amounts of training data typically owned by multiple parties, who are often unwilling to share their sensitive data for collaborative training without guarantees on their data privacy. To enable effective privacy-preserving multiparty topic modeling, we propose a novel federated general-purpose topic model named private and consistent topic discovery (PC-TD). On the one hand, PC-TD seamlessly integrates differential privacy in topic modeling to provide privacy guarantees on sensitive data of different parties. On the other hand, PC-TD exploits multiple sources of …

Multi-Modal Cooking Workflow Construction For Food Recipes, Liangming Pan, Jingjing Chen, Jianlong Wu, Shaoteng Liu, Chong-Wah Ngo, Min-Yen Kan, Yugang Jiang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Understanding food recipe requires anticipating the implicit causal effects of cooking actions, such that the recipe can be converted into a graph describing the temporal workflow of the recipe. This is a non-trivial task that involves common-sense reasoning. However, existing efforts rely on hand-crafted features to extract the workflow graph from recipes due to the lack of large-scale labeled datasets. Moreover, they fail to utilize the cooking images, which constitute an important part of food recipes. In this paper, we build MM-ReS, the first large-scale dataset for cooking workflow construction, consisting of 9,850 recipes with human-labeled workflow graphs. Cooking steps …

Cross-Domain Cross-Modal Food Transfer, Bin Zhu, Chong-Wah Ngo, Jingjing Chen

Research Collection School Of Computing and Information Systems

The recent works in cross-modal image-to-recipe retrieval pave a new way to scale up food recognition. By learning the joint space between food images and recipes, food recognition is boiled down as a retrieval problem by evaluating the similarity of embedded features. The major drawback, nevertheless, is the difficulty in applying an already-trained model to recognize different cuisines of dishes unknown to the model. In general, model updating with new training examples, in the form of image-recipe pairs, is required to adapt a model to new cooking styles in a cuisine. Nevertheless, in practice, acquiring sufficient number of image-recipe pairs …

Deepsonar: Towards Effective And Robust Detection Of Ai-Synthesized Fake Voices, Run Wang, Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu

Research Collection School Of Computing and Information Systems

With the recent advances in voice synthesis, AI-synthesized fake voices are indistinguishable to human ears and widely are applied to produce realistic and natural DeepFakes, exhibiting real threats to our society. However, effective and robust detectors for synthesized fake voices are still in their infancy and are not ready to fully tackle this emerging threat. In this paper, we devise a novel approach, named DeepSonar, based on monitoring neuron behaviors of speaker recognition (SR) system, i.e., a deep neural network (DNN), to discern AI-synthesized fake voices. Layer-wise neuron behaviors provide an important insight to meticulously catch the differences among inputs, …

Sentiment Analysis For Software Engineering: How Far Can Pre-Trained Transformer Models Go?, Ting Zhang, Bowen Xu, Thung Ferdian, Stefanus Agus Haryono, David Lo, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Extensive research has been conducted on sentiment analysis for software engineering (SA4SE). Researchers have invested much effort in developing customized tools (e.g., SentiStrength-SE, SentiCR) to classify the sentiment polarity for Software Engineering (SE) specific contents (e.g., discussions in Stack Overflow and code review comments). Even so, there is still much room for improvement. Recently, pre-trained Transformer-based models (e.g., BERT, XLNet) have brought considerable breakthroughs in the field of natural language processing (NLP). In this work, we conducted a systematic evaluation of five existing SA4SE tools and variants of four state-of-the-art pre-trained Transformer-based models on six SE datasets. Our work is …

Automated Discussion Analysis - Framework For Knowledge Analysis From Class Discussions, Swapna Gottipati, Venky Shankararaman, Mallikan Gokarn Nitin

Research Collection School Of Computing and Information Systems

This research full paper, describes knowledge management of class discussions using an analytics based framework. Discussions, either live classroom or through online forums, when used as a teaching method can help stimulate critical thinking. It allows the teacher to explore in-depth the key concepts covered in the course, motivates students to articulate their ideas clearly and challenge the students to think more deeply. Analysing the discussions helps instructors gain better insights on the personal and collaborative learning behaviour of students. However, knowledge from in-class discussions and online forums is not effectively captured and mined due to lack of appropriate automated …

Tpr: Text-Aware Preference Ranking For Recommender Systems, Yu-Neng Chuang, Chih-Ming Chen, Chuan-Ju Wang, Ming-Feng Tsai, Yuan Fang, Ee-Peng Lim

Research Collection School Of Computing and Information Systems

Textual data is common and informative auxiliary information for recommender systems. Most prior art utilizes text for rating prediction, but rare work connects it to top-N recommendation. Moreover, although advanced recommendation models capable of incorporating auxiliary information have been developed, none of these are specifically designed to model textual information, yielding a limited usage scenario for typical user-to-item recommendation. In this work, we present a framework of text-aware preference ranking (TPR) for top-N recommendation, in which we comprehensively model the joint association of user-item interaction and relations between items and associated text. Using the TPR framework, we construct a joint …

A Critical Genre Analysis Of The User Requirement Specification Report In The Information Technology Industry, Shahizan Shaharuddin

Student Works (2020-2029)

While it has been established that discursive practices involving writing are highly situated and context-bound, few local studies have examined the processes involved in the construction of texts and the shaping of the intended text. Recent perspectives on writing as ways of working and acting, and texts as social action suggest that studying the processes behind the construction of a text could provide a clearer understanding of the discursive practices involved in the shaping of the text. This is seen as relevant to professional writing context in the present study as texts are often shaped to facilitate the social action …

F-Measure Optimisation And Label Regularisation For Energy-Based Neural Dialogue State Tracking Models, Anh Duong Trinh, Robert J. Ross, John D. Kelleher

Conference papers

In recent years many multi-label classification methods have exploited label dependencies to improve performance of classification tasks in various domains, hence casting the tasks to structured prediction problems. We argue that multi-label predictions do not always satisfy domain constraint restrictions. For example when the dialogue state tracking task in task-oriented dialogue domains is solved with multi-label classification approaches, slot-value constraint rules should be enforced following real conversation scenarios.

To address these issues we propose an energy-based neural model to solve the dialogue state tracking task as a structured prediction problem. Furthermore we propose two improvements over previous methods with respect …

Tag: Automated Image Captioning, Nathan Funckes

McNair Scholars Manuscripts

Many websites remain non-ADA compliant, containing images which lack accompanying textual descriptions. This leaves sight-impaired individuals unable to fully enjoy the rich wonders of the web. To address this inequity, our research aims to create an autonomous system capable of generating semantically accurate descriptions of images. This problem involves two tasks: recognizing an image and linguistically describing it. Our solution uses state-of-the-art deep learning: employing a convolutional neural network that "learns" to understand images and extracts their salient features, and a recurrent neural network that learns to generate structured, coherent sentences. These two networks are merged to create a single …

Research Method For Integrating Subjective And Objective Information In Evaluating Quality Level Of Journals, Zhang Quan, Wen Xin, Liping Yu, Jia Wei

Journal of System Simulation

Abstract: The evaluation of the quality level of academic journals was investigated. In order to overcome the drawbacks of current methods which made uses of only the subjective evaluation information or the objective information on the journals, a comprehensive evaluation method was proposed for integrating both the decision makers' subjective information and the objective information. The decision makers' subjective evaluation information was expressed in some grade, grade interval, or a fuzzy set over a grade evaluation set. The objective information of journals was reflected by the numerical attributes, such as, the total citation frequency, impact factor and immediacy …

Research 4.0: Research In The Age Of Automation, Rob Procter, Ben Glover, Elliot Jones

Copyright, Fair Use, Scholarly Communication, etc.

Executive Summary

There is a growing consensus that we are at the start of a fourth industrial revolution, driven by developments in Artificial Intelligence, machine learning, robotics, the Internet of Things, 3-D printing, nanotechnology, biotechnology, 5G, new forms of energy storage and quantum computing. This wave of technical innovations is already having a significant impact on how research is conducted, with dramatic change across research methods in recent years within some disciplines, as this project’s interim report set out.

Whilst there are a wide range of technologies associated with the fourth industrial revolution, this report primarily seeks to understand what …