Digital Commons Network^™

Semantic Reasoning In Zero Example Video Event Retrieval, M. H. T. De Boer, Yi-Jie Lu, Hao Zhang, Klamer Schutte, Chong-Wah Ngo, Wessel Kraaij

Research Collection School Of Computing and Information Systems

Searching in digital video data for high-level events, such as a parade or a car accident, is challenging when the query is textual and lacks visual example images or videos. Current research in deep neural networks is highly beneficial for the retrieval of high-level events using visual examples, but without examples it is still hard to (1) determine which concepts are useful to pre-train (Vocabulary challenge) and (2) which pre-trained concept detectors are relevant for a certain unseen high-level event (Concept Selection challenge). In our article, we present our Semantic Event Retrieval Systemwhich (1) shows the importance of high-level concepts …

Visual Sentiment Analysis For Review Images With Item-Oriented And User-Oriented Cnn, Quoc Tuan Truong, Hady W. Lauw

Research Collection School Of Computing and Information Systems

Online reviews are prevalent. When recounting their experience with a product, service, or venue, in addition to textual narration, a reviewer frequently includes images as photographic record. While textual sentiment analysis has been widely studied, in this paper we are interested in visual sentiment analysis to infer whether a given image included as part of a review expresses the overall positive or negative sentiment of that review. Visual sentiment analysis can be formulated as image classification using deep learning methods such as Convolutional Neural Networks or CNN. However, we observe that the sentiment captured within an image may be affected …

Exploring The Internal Statistics: Single Image Super-Resolution, Completion And Captioning, Yang Xian

Dissertations, Theses, and Capstone Projects

Image enhancement has drawn increasingly attention in improving image quality or interpretability. It aims to modify images to achieve a better perception for human visual system or a more suitable representation for further analysis in a variety of applications such as medical imaging, remote sensing, and video surveillance. Based on different attributes of the given input images, enhancement tasks vary, e.g., noise removal, deblurring, resolution enhancement, prediction of missing pixels, etc. The latter two are usually referred to as image super-resolution and image inpainting (or completion).

Image super-resolution and completion are numerically ill-posed problems. Multi-frame-based approaches make use of the …

Am I Who I Say I Am? Unobtrusive Selfrepresentation And Personality Recognition On Facebook, Margeret A. Hall, Simon Caton

Interdisciplinary Informatics Faculty Publications

Across social media platforms users (sub)consciously represent themselves in a way which is appropriate for their intended audience. This has unknown impacts on studies with unobtrusive designs based on digital (social) platforms, and studies of contemporary social phenomena in online settings. A lack of appropriate methods to identify, control for, and mitigate the effects of self-representation, the propensity to express socially responding characteristics or self-censorship in digital settings, hinders the ability of researchers to confidently interpret and generalize their findings. This article proposes applying boosted regression modelling to fill this research gap. A case study of paid Amazon Mechanical Turk …

Improving Speech Recognition For Interviews With Both Clean And Telephone Speech, Sung Woo Choi

Journal of Undergraduate Research at Minnesota State University, Mankato

High quality automatic speech recognition (ASR) depends on the context of the speech. Cleanly recorded speech has better results than speech recorded over telephone lines. In telephone speech, the signal is band-pass filtered which limits frequencies available for computation. Consequently, the transmitted speech signal may be distorted by noise, causing higher word error rates (WER). The main goal of this research project is to examine approaches to improve recognition of telephone speech while maintaining or improving results for clean speech in mixed telephone-clean speech recordings, by reducing mismatches between the test data and the available models. The test data includes …

Y-Chromosome Markers For The Red Fox, Halie M. Rando, Jeremy T. Stutchman, Estelle R. Bastounes, Jennifer L. Johnson, Carlos A. Driscoll, Christina S. Barr, Lyudmila N. Trut, Benjamin N. Sacks, Anna V. Kukekova

Computer Science: Faculty Publications

The de novo assembly of the red fox (Vulpes vulpes) genome has facilitated the development of genomic tools for the species. Efforts to identify the population history of red foxes in North America have previously been limited by a lack of information about the red fox Y-chromosome sequence. However, a megabase of red fox Y-chromosome sequence was recently identified over 2 scaffolds in the reference genome. Here, these scaffolds were scanned for repeated motifs, revealing 194 likely microsatellites. Twenty-Three of these loci were selected for primer development and, after testing, produced a panel of 11 novel markers that …

Nlp2code: Code Snippet Content Assist Via Natural Language Tasks, Brock A. Campbell, Christoph Treude

Research Collection School Of Computing and Information Systems

Developers increasingly take to the Internet for code snippets to integrate into their programs. To save developers the time required to switch from their development environments to a web browser in the quest for a suitable code snippet, we introduce NLP2Code, a content assist for code snippets. Unlike related tools, NLP2Code integrates directly into the source code editor and provides developers with a content assist feature to close the vocabulary gap between developers’ needs and code snippet meta data. Our preliminary evaluation of NLP2Code shows that the majority of invocations lead to code snippets rated as helpful by users and …

Xsearch: A Domain-Specific Cross-Language Relevant Question Retrieval Tool, Bowen Xu, Zhenchang Xing, Xin Xia, David Lo, Xuan-Bach D. Le

Research Collection School Of Computing and Information Systems

During software development process, Chinese developers often seek solutions to the technical problems they encounter by searching relevant questions on Q&A sites. When developers fail to find solutions on Q&A sites in Chinese, they could translate their query and search on the English Q&A sites. However, Chinese developers who are non-native English speakers often are not comfortable to ask or search questions in English, as they do not know the proper translation of the Chinese technical words into the English technical words. Furthermore, the process of manually formulating cross-language queries and determining the importance of query words is a tedious …

Inferring Spread Of Readers’ Emotion Affected By Online News, Agus Sulistya, Ferdian Thung, David Lo

Research Collection School Of Computing and Information Systems

Depending on the reader, A news article may be viewed from many different perspectives, thus triggering different (and possibly contradicting) emotions. In this paper, we formulate a problem of predicting readers’ emotion distribution affected by a news article. Our approach analyzes affective annotations provided by readers of news articles taken from a non-English online news site. We create a new corpus from the annotated articles, and build a domain-specific emotion lexicon and word embedding features. We finally construct a multi-target regression model from a set of features extracted from online news articles. Our experiments show that by combining lexicon and …

Involute Analysis: Virtual Discourse, Memory Systems And Archive In The Involutes Of Thomas De Quincey, Kimberley A. Garcia

Dissertations, Theses, and Capstone Projects

Thomas De Quincey’s involutes inform metaphysical thought on memory and language, particularly concerning multiplicity and the virtual, repetition and difference. When co-opting the mathematic and mechanic involute in Suspiria de Profundis, De Quincey generates an interdisciplinary matrix for the semiotics underpinning his philosophy of language and theory of memory and experience. Involutes entangle and reproduce. De Quincey’s involute exposes the concrete and actual through which all experience accesses the abstract or virtual. The materiality of their informatics and technics provides a literary model and theoretical precursor to a combination of archive and systems theory. The textuality of involute system(s)—both …

The Human Touch: How Non-Expert Users Perceive, Interpret, And Fix Topic Models, Tak Yeon Lee, Alison Smith, Kevin Seppi, Niklas Elmqvist, Jordan Boyd-Graber, Leah Findlater

Faculty Publications

Topic modeling is a common tool for understanding large bodies of text, but is typically provided as a “take it or leave it” proposition. Incorporating human knowledge in unsupervised learning is a promising approach to create high-quality topic models. Existing interactive systems and modeling algorithms support a wide range of refinement operations to express feedback. However, these systems’ interactions are primarily driven by algorithmic convenience, ignoring users who may lack expertise in topic modeling. To better understand how non-expert users understand, assess, and refine topics, we conducted two user studies—an in-person interview study and an online crowdsourced study. These studies …

Rating By Ranking: An Improved Scale For Judgement-Based Labels, Jack O'Neill, Sarah Jane Delany, Brian Mac Namee

Conference papers

Labels representing value judgements are commonly elicited using an interval scale of absolute values. Data collected in such a manner is not always reliable. Psychologists have long recognized a number of biases to which many human raters are prone, and which result in disagreement among raters as to the true gold standard rating of any particular object. We hypothesize that the issues arising from rater bias may be mitigated by treating the data received as an ordered set of preferences rather than a collection of absolute values. We experiment on real-world and artificially generated data, finding that treating label ratings …

Detect Rumors In Microblog Posts Using Propagation Structure Via Kernel Learning, Jing Ma, Wei Gao, Kam-Fai Wong

Research Collection School Of Computing and Information Systems

How fake news goes viral via social media? How does its propagation pattern differ from real stories? In this paper, we attempt to address the problem of identifying rumors, i.e., fake information, out of microblog posts based on their propagation structure. We firstly model microblog posts diffusion with propagation trees, which provide valuable clues on how an original message is transmitted and developed over time. We then propose a kernel-based method called Propagation Tree Kernel, which captures high-order patterns differentiating different types of rumors by evaluating the similarities between their propagation tree structures. Experimental results on two real-world datasets demonstrate …

An Intelligent Multimodal Upper-Limb Rehabilitation Robotic System, Alexandros Lioulemes

Computer Science and Engineering Dissertations - Archive

A traffic accident, a battlefield injury, or a stroke can lead to brain or musculoskeletal injuries that impact motor and cognitive functions and can drastically change a person's life. In such situations, rehabilitation plays a critical role in the ability of the patient to partially or totally regain motor function, but the optimal training approach remains unclear. Robotic technologies are recognized as powerful tools to promote neuroplasticity and stimulate motor re-learning. Moreover, they deliver high-intensity, repetitive, active and task-oriented training; in addition, they provide objective measurements for patient evaluation. The primary focus of this research is to investigate the development …

Humor Detection, Manan Jain

Dissertations and Theses

Humor is a very complex characteristic concept that defines us as human beings and social entities. Humor is an essential component in personal communication. How to create a method or model to discover the structures behind humor, recognize humor and even extraction of humor remains a challenge because of its subjective nature. Humor also provides valuable information related to linguistic, psychological, neurological and sociological phenomena. However, because of its complexity, humor is still an undefined phenomenon. Because the reaction that make people laugh can hardly be generalized or formalized. For instance, cognitive aspects as well as cultural knowledge, are some …

Semantic Visualization For Short Texts With Word Embeddings, Van Minh Tuan Le, Hady W. Lauw

Research Collection School Of Computing and Information Systems

Semantic visualization integrates topic modeling and visualization, such that every document is associated with a topic distribution as well as visualization coordinates on a low-dimensional Euclidean space. We address the problem of semantic visualization for short texts. Such documents are increasingly common, including tweets, search snippets, news headlines, or status updates. Due to their short lengths, it is difficult to model semantics as the word co-occurrences in such a corpus are very sparse. Our approach is to incorporate auxiliary information, such as word embeddings from a larger corpus, to supplement the lack of co-occurrences. This requires the development of a …

Large-Scale Online Feature Selection For Ultra-High Dimensional Sparse Data, Yue Wu, Steven C. H. Hoi, Tao Mei, Nenghai Yu

Research Collection School Of Computing and Information Systems

Feature selection (FS) is an important technique in machine learning and data mining, especially for large scale high-dimensional data. Most existing studies have been restricted to batch learning, which is often inefficient and poorly scalable when handling big data in real world. As real data may arrive sequentially and continuously, batch learning has to retrain the model for the new coming data, which is very computationally intensive. Online feature selection (OFS) is a promising new paradigm that is more efficient and scalable than batch learning algorithms. However, existing online algorithms usually fall short in their inferior efficacy. In this article, …

Predicting Potential Alzheimer Medical Condition In Elderly Using Iot Sensors - Case Study, Zhi Hao Kevin Chong, Yu Xuan Tee, Ling Jing Toh, Shi Jia Phang, Jie Ying Liew, Bertran Queck, Swapna Gottipati

Research Collection School Of Computing and Information Systems

Ageing population would cause profound problems and the impact is already being felt today in many developed countries such as Singapore. The main concern for the Government is to help the citizens with active ageing through home ownership and good healthcare. With Internet of Things (IoT) gaining traction globally, Singapore is set to take advantage of this technology and leverage it to extend its capabilities towards a graceful Ageing-In-Place for the elderly. This ties in nicely with the expertise of SHINE Seniors project by SMU-iCity Lab, which integrates IT with healthcare in ways that creates innovative IT health solutions that …

Bridge Text And Knowledge By Learning Multi-Prototype Entity Mention Embedding, Yixin Cao, Lifu Huang, Heng Ji, Xu Chen, Juanzi Li

Research Collection School Of Computing and Information Systems

Integrating text and knowledge into a unified semantic space has attracted significant research interests recently. However, the ambiguity in the common space remains a challenge, namely that the same mention phrase usually refers to various entities. In this paper, to deal with the ambiguity of entity mentions, we propose a novel Multi-Prototype Mention Embedding model, which learns multiple sense embeddings for each mention by jointly modeling words from textual contexts and entities derived from a knowledge base. In addition, we further design an efficient language model based approach to disambiguate each mention to a specific sense. In experiments, both qualitative …

Tool Support For Capturing The Essence Of A Concern In Source Code, Chuntao Fu

Student Work

Software evolves constantly to adapt to changing user needs. As it evolves, it becomes progressively harder to understand due to accumulation of code changes, increasing code size, and the introduction of complex code dependencies. As a result, it becomes harder to maintain, exposing the software to potential bugs and degradation of code quality. High maintenance costs and diminished opportunities for software reusability and portability lead to reduced return on investment, increasing the likelihood of the software product being discarded or replaced. Nevertheless, we believe that there is value in legacy software due to the amount of intellectual efforts that have …

Can Syntax Help? Improving An Lstm-Based Sentence Compression Model For New Domains, Liangguo Wang, Jing Jiang, Hai Leong Chieu, Chen Hui Ong, Dandan Song, Lejian Liao

Research Collection School Of Computing and Information Systems

In this paper, we study how to improve thedomain adaptability of a deletion-basedLong Short-Term Memory (LSTM) neuralnetwork model for sentence compression.We hypothesize that syntactic informationhelps in making such modelsmore robust across domains. We proposetwo major changes to the model: usingexplicit syntactic features and introducingsyntactic constraints through Integer LinearProgramming (ILP). Our evaluationshows that the proposed model works betterthan the original model as well as a traditionalnon-neural-network-based modelin a cross-domain setting.

Embedding-Based Representation Of Categorical Data By Hierarchical Value Coupling Learning, Songlei Jian, Longbing Cao, Guansong Pang, Kai Lu, Hang Gao

Research Collection School Of Computing and Information Systems

Learning the representation of categorical data with hierarchical value coupling relationships is very challenging but critical for the effective analysis and learning of such data. This paper proposes a novel coupled unsupervised categorical data representation (CURE) framework and its instantiation, i.e., a coupled data embedding (CDE) method, for representing categorical data by hierarchical value-to-value cluster coupling learning. Unlike existing embedding- and similarity-based representation methods which can capture only a part or none of these complex couplings, CDE explicitly incorporates the hierarchical couplings into its embedding representation. CDE first learns two complementary feature value couplings which are then used to cluster …

Vertex Weighted Spectral Clustering, Mohammad Masum

Electronic Theses and Dissertations

Spectral clustering is often used to partition a data set into a specified number of clusters. Both the unweighted and the vertex-weighted approaches use eigenvectors of the Laplacian matrix of a graph. Our focus is on using vertex-weighted methods to refine clustering of observations. An eigenvector corresponding with the second smallest eigenvalue of the Laplacian matrix of a graph is called a Fiedler vector. Coefficients of a Fiedler vector are used to partition vertices of a given graph into two clusters. A vertex of a graph is classified as unassociated if the Fiedler coefficient of the vertex is close to …

A Research Stream On Sentiment Analysis, B. Yuan, Keng Siau

Research Collection School Of Computing and Information Systems

Sentiment analysis (SA) is a powerful mining technique to study online reviews and comments (Lee & Siau, 2001; Adeborna & Siau, 2014; Zhao & Siau, 2017). It is an advanced text mining technique and the goal of SA is to recognize and extract meaningful information from data using natural language processing (NLP) and computational linguistics.SA has been applied to areas such as marketing, online social media, customer service, education, and even energy fields (Yuan & Siau, 2017). For example, SA can be used to identify the attitude of customers according to polarity of the reviews and comments that they left …

A Pattern Approach To Examine The Design Space Of Spatiotemporal Visualization, Chen Guo

Department of Computer Graphics Technology Degree Theses

Pattern language has been widely used in the development of visualization systems. This dissertation applies a pattern language approach to explore the design space of spatiotemporal visualization. The study provides a framework for both designers and novices to communicate, develop, evaluate, and share spatiotemporal visualization design on an abstract level. The touchstone of the work is a pattern language consisting of fifteen design patterns and four categories. In order to validate the design patterns, the researcher created two visualization systems with this framework in mind. The first system displayed the daily routine of human beings via a polygon-based visualization. The …

Explaining Trained Neural Networks With Semantic Web Technologies: First Steps, Md Kamruzzaman Sarker, Ning Xie, Derek Doran, Michael L. Raymer, Pascal Hitzler

Computer Science and Engineering Faculty Publications

The ever increasing prevalence of publicly available struc-tured data on the World Wide Web enables new applications in a varietyof domains. In this paper, we provide a conceptual approach that lever-ages such data in order to explain the input-output behavior of trainedartificial neural networks. We apply existing Semantic Web technologiesin order to provide an experimental proof of concept.

Detecting User Demographics In Twitter To Inform Health Trends In Social Media, Christopher R. Markson

Dissertations

The widespread and popular use of social media and social networking applications offer a promising opportunity for gaining knowledge and insights regarding population health conditions thanks to the diversity and abundance of online user-generated information (UGHI) relating to healthcare and well-being. However, users on social media and social networking sites often do not supply their complete demographic information, which greatly undermines the value of the aforementioned information for health 2.0 research, e.g., for discerning disparities across population groups in certain health conditions. To recover the missing user demographic information, existing methods observe a limited scope of user behaviors, such as …

Discovering Explanatory Models To Identify Relevant Tweets On Zika, Roopteja Muppalla, Michele Miller, Tanvi Banerjee, William L. Romine

Kno.e.sis Publications

Zika virus has caught the worlds attention, and has led people to share their opinions and concerns on social media like Twitter. Using text-based features, extracted with the help of Parts of Speech (POS) taggers and N-gram, a classifier was built to detect Zika related tweets from Twitter. With a simple logistic classifier, the system was successful in detecting Zika related tweets from Twitter with a 92% accuracy. Moreover, key features were identified that provide deeper insights on the content of tweets relevant to Zika. This system can be leveraged by domain experts to perform sentiment analysis, and understand the …

Truly Multi-Modal Youtube-8m Video Classification With Video, Audio, And Text, Zhe Wang, Kingsley Kuan, Mathieu Ravant, Gaurav Manek, Sibo Song, Yuan Fang, Et Al

Research Collection School Of Computing and Information Systems

The YouTube-8M video classification challenge requires teams to classify 0.7 million videos into one or more of 4,716 classes. In this Kaggle competition, we placed in the top 3% out of 650 participants using released video and audio features. Beyond that, we extend the original competition by including text information in the classification, making this a truly multi-modal approach with vision, audio and text. The newly introduced text data is termed as YouTube-8M-Text. We present a classification framework for the joint use of text, visual and audio features, and conduct an extensive set of experiments to quantify the benefit that …

A Domain Based Approach To Social Relation Recognition, Qianru Sun, Bernt Schiele, Mario Fritz

Research Collection School Of Computing and Information Systems

Social relations are the foundation of human daily life. Developing techniques to analyze such relations from visual data bears great potential to build machines that better understand us and are capable of interacting with us at a social level. Previous investigations have remained partial due to the overwhelming diversity and complexity of the topic and consequently have only focused on a handful of social relations. In this paper, we argue that the domain-based theory from social psychology is a great starting point to systematically approach this problem. The theory provides coverage of all aspects of social relations and equally is …