Digital Commons Network^™

Ordinal Hyperplane Loss, Bob Vanderheyden, Ying Xie

OHPL Publications

The problem of ordinal classification occurs in a large and growing number of areas. Some of the most common source and applications of ordinal data include rating scales, medical classification scales, socio-economic scales, meaningful groupings of continuous data, facial emotional intensity, facial age estimation, etc. The problem of predicting ordinal classes is typically addressed by either performing n-1 binary classification for n ordinal classes or treating ordinal classes as continuous values for regression. However, the first strategy doesn’t fully utilize the ordering information of classes and the second strategy imposes a strong continuous assumption to ordinal classes. In this paper, …

Deep Air Learning: Interpolation, Prediction, And Feature Analysis Of Fine-Grained Air Quality, Zhongang Qi, Tianchun Wang, Guojie Song, Weisong Hu, Xi Li, Zhongfei Mark Zhang

Research Collection School Of Computing and Information Systems

The interpolation, prediction, and feature analysis of fine-gained air quality are three important topics in the area of urban air computing. The solutions to these topics can provide extremely useful information to support air pollution control, and consequently generate great societal and technical impacts. Most of the existing work solves the three problems separately by different models. In this paper, we propose a general and effective approach to solve the three problems in one model called the Deep Air Learning (DAL). The main idea of DAL lies in embedding feature selection and semi-supervised learning in different layers of the deep …

Deep Learning For Learning Representation And Its Application To Natural Language Processing, Shekoofeh Mokhtari

FIU Electronic Theses and Dissertations

As the web evolves even faster than expected, the exponential growth of data becomes overwhelming. Textual data is being generated at an ever-increasing pace via emails, documents on the web, tweets, online user reviews, blogs, and so on. As the amount of unstructured text data grows, so does the need for intelligently processing and understanding it. The focus of this dissertation is on developing learning models that automatically induce representations of human language to solve higher level language tasks.

In contrast to most conventional learning techniques, which employ certain shallow-structured learning architectures, deep learning is a newly developed machine learning …

A Multi-Task Approach To Incremental Dialogue State Tracking, Anh Duong Trinh, Robert J. Ross, John D. Kelleher

Conference papers

Incrementality is a fundamental feature of language in real world use. To this point, however, the vast majority of work in automated dialogue processing has focused on language as turn based. In this paper we explore the challenge of incremental dialogue state tracking through the development and analysis of a multi-task approach to incremental dialogue state tracking. We present the design of our incremental dialogue state tracker in detail and provide evaluation against the well known Dialogue State Tracking Challenge 2 (DSTC2) dataset. In addition to a standard evaluation of the tracker, we also provide an analysis of the Incrementality …

Dsm: A Specification Mining Tool Using Recurrent Neural Network Based Language Model, Tien-Duy B. Le, Lingfeng Bao, David Lo

Research Collection School Of Computing and Information Systems

Formal specifications are important but often unavailable. Furthermore, writing these specifications is time-consuming and requires skills from developers. In this work, we present Deep Specification Miner (DSM), an automated tool that applies deep learning to mine finite-state automaton (FSA) based specifications. DSM accepts as input a set of execution traces to train a Recurrent Neural Network Language Model (RNNLM). From the input traces, DSM creates a Prefix Tree Acceptor (PTA) and leverages the inferred RNNLM to extract many features. These features are then forwarded to clustering algorithms for merging similar automata states in the PTA for assembling a number of …

Prediction Of Relatedness In Stack Overflow: Deep Learning Vs. Svm: A Reproducibility Study, Bowen Xu, Amirreza Shirani, David Lo, Mohammad Amin Alipour

Research Collection School Of Computing and Information Systems

Background Xu et al. used a deep neural network (DNN) technique to classify the degree of relatedness between two knowledge units (question-answer threads) on Stack Overflow. More recently, extending Xu et al.'s work, Fu and Menzies proposed a simpler classification technique based on a fine-tuned support vector machine (SVM) that achieves similar performance but in a much shorter time. Thus, they suggested that researchers need to compare their sophisticated methods against simpler alternatives.Aim The aim of this work is to replicate the previous studies and further investigate the validity of Fu and Menzies' claim by evaluating the DNN- and SVM-based …

Online Deep Learning: Learning Deep Neural Networks On The Fly, Doyen Sahoo, Hong Quang Pham, Jing Lu, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Deep Neural Networks (DNNs) are typically trained by backpropagation in a batch setting, requiring the entire training data to be made available prior to the learning task. This is not scalable for many real-world scenarios where new data arrives sequentially in a stream. We aim to address an open challenge of “Online Deep Learning” (ODL) for learning DNNs on the fly in an online setting. Unlike traditional online learning that often optimizes some convex objective function with respect to a shallow model (e.g., a linear/kernel-based hypothesis), ODL is more challenging as the optimization objective is non-convex, and regular DNN with …

Deep Specification Mining, Tien-Duy B. Le, David Lo

Research Collection School Of Computing and Information Systems

Formal specifications are essential but usually unavailable in software systems. Furthermore, writing these specifications is costly and requires skills from developers. Recently, many automated techniques have been proposed to mine specifications in various formats including finite-state automaton (FSA). However, more works in specification mining are needed to further improve the accuracy of the inferred specifications. In this work, we propose Deep Specification Miner (DSM), a new approach that performs deep learning for mining FSA-based specifications. Our proposed approach uses test case generation to generate a richer set of execution traces for training a Recurrent Neural Network Based Language Model (RNNLM). …

Worker Activity Recognition In Smart Manufacturing Using Imu And Semg Signals With Convolutional Neural Networks, Wenjin Tao, Ze-Hao Lai, Ming-Chuan Leu, Zhaozheng Yin

Mechanical and Aerospace Engineering Faculty Research & Creative Works

In a smart manufacturing system involving workers, recognition of the worker's activity can be used for quantification and evaluation of the worker's performance, as well as to provide onsite instructions with augmented reality. In this paper, we propose a method for activity recognition using Inertial Measurement Unit (IMU) and surface electromyography (sEMG) signals obtained from a Myo armband. The raw 10-channel IMU signals are stacked to form a signal image. This image is transformed into an activity image by applying Discrete Fourier Transformation (DFT) and then fed into a Convolutional Neural Network (CNN) for feature extraction, resulting in a high-level …

D-Pruner: Filter-Based Pruning Method For Deep Convolutional Neural Network, Nguyen Loc Huynh, Youngki Lee, Rajesh Krishna Balan

Research Collection School Of Computing and Information Systems

The emergence of augmented reality devices such as Google Glass and Microsoft Hololens has opened up a new class of vision sensing applications. Those applications often require the ability to continuously capture and analyze contextual information from video streams. They often adopt various deep learning algorithms such as convolutional neural networks (CNN) to achieve high recognition accuracy while facing severe challenges to run computationally intensive deep learning algorithms on resource-constrained mobile devices. In this paper, we propose and explore a new class of compression technique called D-Pruner to efficiently prune redundant parameters within a CNN model to run the model …

D-Pruner: Filter-Based Pruning Method For Deep Convolutional Neural Network, Nguyen Loc Huynh, Youngki Lee, Rajesh Krishna Balan

Research Collection School Of Computing and Information Systems

The emergence of augmented reality devices such as Google Glass and Microsoft Hololens has opened up a new class of vision sensing applications. Those applications often require the ability to continuously capture and analyze contextual information from video streams. They often adopt various deep learning algorithms such as convolutional neural networks (CNN) to achieve high recognition accuracy while facing severe challenges to run computationally intensive deep learning algorithms on resource-constrained mobile devices. In this paper, we propose and explore a new class of compression technique called D-Pruner to efficiently prune redundant parameters within a CNN model to run the model …

Opportunities And Obstacles For Deep Learning In Biology And Medicine, Travers Ching, S Joshua Swamidass, Et Al

2010-2019 OA Pubs

Deep learning describes a class of machine learning algorithms that are capable of combining raw inputs into layers of intermediate features. These algorithms have recently shown impressive results across a variety of domains. Biology and medicine are data-rich disciplines, but the data are complex and often ill-understood. Hence, deep learning techniques may be particularly well suited to solve problems of these fields. We examine applications of deep learning to a variety of biomedical problems-patient classification, fundamental biological processes and treatment of patients-and discuss whether deep learning will be able to transform these tasks or if the biomedical sphere poses unique …

Urlnet: Learning A Url Representation With Deep Learning For Malicious Url Detection, Hung Le, Hong Quang Pham, Doyen Sahoo, Steven C. H. Hoi

Research Collection School Of Computing and Information Systems

Malicious URLs host unsolicited content and are used to perpetrate cybercrimes. It is imperative to detect them in a timely manner. Traditionally, this is done through the usage of blacklists, which cannot be exhaustive, and cannot detect newly generated malicious URLs. To address this, recent years have witnessed several efforts to perform Malicious URL Detection using Machine Learning. The most popular and scalable approaches use lexical properties of the URL string by extracting Bag-of-words like features, followed by applying machine learning models such as SVMs. There are also other features designed by experts to improve the prediction performance of the …

Investigating The Application Of Deep Convolutional Neural Networks In Semi-Supervised Video Object Segmentation, Jayadeep Sasikumar

Dissertations

This thesis investigates the different approaches to video object segmentation and the current state-of-the-art in the discipline, focusing on the different deep learning techniques used to solve the problem. The primary contribution of the thesis is the investigation of usefulness of Exponential Linear Units as activation functions for deep convolutional neural architectures trained to perform object semi-supervised segmentation in videos. Mask R-CNN was chosen as the base convolutional neural architecture, with the view of extending the image segmentation algorithm to videos. Two models were created, one with Rectified Linear Units and the other with Exponential Linear Units as the respective …

Generating Diverse And Meaningful Captions: Unsupervised Specificity Optimization For Image Captioning, Annika Lindh, Robert J. Ross, Abhijit Mahalunkar, Giancarlo Salton, John D. Kelleher

Conference papers

Image Captioning is a task that requires models to acquire a multi-modal understanding of the world and to express this understanding in natural language text. While the state-of-the-art for this task has rapidly improved in terms of n-gram metrics, these models tend to output the same generic captions for similar images. In this work, we address this limitation and train a model that generates more diverse and specific captions through an unsupervised training approach that incorporates a learning signal from an Image Retrieval model. We summarize previous results and improve the state-of-the-art on caption diversity and novelty.

We make our …

A Frame-Based Nlp System For Cancer-Related Information Extraction., Yuqi Si, Kirk Roberts

Journal Articles

We propose a frame-based natural language processing (NLP) method that extracts cancer-related information from clinical narratives. We focus on three frames: cancer diagnosis, cancer therapeutic procedure, and tumor description. We utilize a deep learning-based approach, bidirectional Long Short-term Memory (LSTM) Conditional Random Field (CRF), which uses both character and word embeddings. The system consists of two constituent sequence classifiers: a frame identification (lexical unit) classifier and a frame element classifier. The classifier achieves an F