Digital Commons Network^™

Transfer Learning With Deep Recurrent Neural Networks For Remaining Useful Life Estimation, Ansi Zhang, Shaobo Li, Yuxin Cui, Zhonghao Liu, Guanci Yang, Jianjun Hu

Faculty Publications

Prognostics, such as remaining useful life (RUL) prediction, is a crucial task in condition-based maintenance. A major challenge in data-driven prognostics is the difficulty of obtaining a sufficient number of samples of failure progression. However, for traditional machine learning methods and deep neural networks, enough training data is a prerequisite to train good prediction models. In this work, we proposed a transfer learning algorithm based on Bi-directional Long Short-Term Memory (BLSTM) recurrent neural networks for RUL estimation, in which the models can be first trained on different but related datasets and then fine-tuned by the target dataset. Extensive experimental results …

Transfer Learning With Deep Recurrent Neural Networks For Remaining Useful Life Estimation, Ansi Zhang, Honglei Wang, Shaobo Li, Yuxin Cui, Guanci Yang, Jianjun Hu

Faculty Publications

Prognostics, such as remaining useful life (RUL) prediction, is a crucial task in condition-based maintenance. A major challenge in data-driven prognostics is the difficulty of obtaining a sufficient number of samples of failure progression. However, for traditional machine learning methods and deep neural networks, enough training data is a prerequisite to train good prediction models. In this work, we proposed a transfer learning algorithm based on Bi-directional Long Short-Term Memory (BLSTM) recurrent neural networks for RUL estimation, in which the models can be first trained on different but related datasets and then fine-tuned by the target dataset. Extensive experimental results …

Interfacing Iconicity - Addressing Software Divarication Through Diagrammatic Design Principles, George Akhvlediani

Theses and Dissertations

This research examines conflicts accompanying the proliferation of computer technology and, more specifically, constellations of dependency in the always expanding volume of software, platforms, and the firms/individuals using them. We identify a pervasive phenomenon of “divarication” in the growing variety of progressively specialized systems and system roles. As software systems enter new thresholds of sophistication, they effectively aggregate many distinct components and protocols. Consequently, we are confronted with a diverse ecology of stratified and thereby incompatible software systems. Software inherits the limitations and potential flaws of its constituent parts, but unlike physical machinery, it isn’t readily disassembled in instances of …

Algorithms For Robot Coverage Under Movement And Sensing Constraints, Jeremy S. Lewis

Theses and Dissertations

This thesis explores the problem of generating coverage paths—that is, paths that pass within some sensor footprint of every point in an environment—for mobile robots. It both considers models for which navigation is a solved problem but motions are constrained, as well for models in which navigation must be considered along with coverage planning because of the robot’s unreliable sensing and movements.

The motion constraint we adopt for the former is a common constraint, that of a Dubins vehicle. We extend previous work that solves this coverage problem as a traveling salesman problem (TSP) by introducing a practical heuristic algorithm …

Tourism Review Sentiment Classification Using A Bidirectional Recurrent Neural Network With An Attention Mechanism And Topic-Enriched Word Vectors, Qin Li, Shaobo Li, Jie Hu, Sen Zhang, Jianjun Hu

Faculty Publications

Sentiment analysis of online tourist reviews is playing an increasingly important role in tourism. Accurately capturing the attitudes of tourists regarding different aspects of the scenic sites or the overall polarity of their online reviews is key to tourism analysis and application. However, the performances of current document sentiment analysis methods are not satisfactory as they either neglect the topics of the document or do not consider that not all words contribute equally to the meaning of the text. In this work, we propose a bidirectional gated recurrent unit neural network model (BiGRULA) for sentiment analysis by combining a topic …

End-To-End Convolutional Neural Network Model For Gear Fault Diagnosis Based On Sound Signals, Yong Yao, Honglei Wang, Shaobo Li, Zhongnhao Liu, Gui Gui, Yabo Dan, Jianjun Hu

Faculty Publications

Currently gear fault diagnosis is mainly based on vibration signals with a few studies on acoustic signal analysis. However, vibration signal acquisition is limited by its contact measuring while traditional acoustic-based gear fault diagnosis relies heavily on prior knowledge of signal processing techniques and diagnostic expertise. In this paper, a novel deep learning-based gear fault diagnosis method is proposed based on sound signal analysis. By establishing an end-to-end convolutional neural network (CNN), the time and frequency domain signals can be fed into the model as raw signals without feature engineering. Moreover, multi-channel information from different microphones can also be fused …

Product Innovation Design Based On Deep Learning And Kansei Engineering, Huafeng Quan, Shaobo Li, Jianjun Hu

Faculty Publications

Creative product design is becoming critical to the success of many enterprises. However, the conventional product innovation process is hindered by two major challenges: the difficulty to capture users’ preferences and the lack of intuitive approaches to visually inspire the designer, which is especially true in fashion design and form design of many other types of products. In this paper, we propose to combine Kansei engineering and the deep learning for product innovation (KENPI) framework, which can transfer color, pattern, etc. of a style image in real time to a product’s shape automatically. To capture user preferences, we combine Kansei …

An Ensemble Stacked Convolutional Neural Network Model For Environmental Event Sound Recognition, Shaobo Li, Yong Yao, Jie Hu, Guokai Liu, Xuemei Yao, Jianjun Hu

Faculty Publications

Convolutional neural networks (CNNs) with log-mel audio representation and CNN-based end-to-end learning have both been used for environmental event sound recognition (ESC). However, log-mel features can be complemented by features learned from the raw audio waveform with an effective fusion method. In this paper, we first propose a novel stacked CNN model with multiple convolutional layers of decreasing filter sizes to improve the performance of CNN models with either log-mel feature input or raw waveform input. These two models are then combined using the Dempster–Shafer (DS) evidence theory to build the ensemble DS-CNN model for ESC. Our experiments over three …

A Bayesian Network Based Adaptability Design Of Product Structures For Function Evolution, Shaobo Li, Yongming Wu, Yan-Xia Xu, Jie Hu, Jianjun Hu

Faculty Publications

Structure adaptability design is critical for function evolution in product families, in which many structural and functional design factors are intertwined together with manufacturing cost, customer satisfaction, and final market sales. How to achieve a delicate balance among all of these factors to maximize the market performance of the product is too complicated to address based on traditional domain experts’ knowledge or some ad hoc heuristics. Here, we propose a quantitative product evolution design model that is based on Bayesian networks to model the dynamic relationship between customer needs and product structure design. In our model, all of the structural …

Aspie: A Framework For Active Sensing And Processing Of Complex Events In The Internet Of Manufacturing Things, Shaobo Li, Weixing Chen, Jie Hu, Jianjun Hu

Faculty Publications

Rapid perception and processing of critical monitoring events are essential to ensure healthy operation of Internet of Manufacturing Things (IoMT)-based manufacturing processes. In this paper, we proposed a framework (active sensing and processing architecture (ASPIE)) for active sensing and processing of critical events in IoMT-based manufacturing based on the characteristics of IoMT architecture as well as its perception model. A relation model of complex events in manufacturing processes, together with related operators and unified XML-based semantic definitions, are developed to effectively process the complex event big data. A template based processing method for complex events is further introduced to conduct …

A Novel Evolutionary Algorithm For Designing Robust Analog Filters, Shaobo Li, Wang Zou, Jianjun Hu

Faculty Publications

Designing robust circuits that withstand environmental perturbation and device degradation is critical for many applications. Traditional robust circuit design is mainly done by tuning parameters to improve system robustness. However, the topological structure of a system may set a limit on the robustness achievable through parameter tuning. This paper proposes a new evolutionary algorithm for robust design that exploits the open-ended topological search capability of genetic programming (GP) coupled with bond graph modeling. We applied our GP-based robust design (GPRD) algorithm to evolve robust lowpass and highpass analog filters. Compared with a traditional robust design approach based on a state-of-the-art …

Patent Keyword Extraction Algorithm Based On Distributed Representation For Patent Classification, Jie Hu, Shaobo Li, Yong Yao, Liya Yu, Guanci Yang, Jianjun Hu

Faculty Publications

Many text mining tasks such as text retrieval, text summarization, and text comparisons depend on the extraction of representative keywords from the main text. Most existing keyword extraction algorithms are based on discrete bag-of-words type of word representation of the text. In this paper, we propose a patent keyword extraction algorithm (PKEA) based on the distributed Skip-gram model for patent classification. We also develop a set of quantitative performance measures for keyword extraction evaluation based on information gain and cross-validation, based on Support Vector Machine (SVM) classification, which are valuable when human-annotated keywords are not available. We used a standard …

A Hierarchical Feature Extraction Model For Multi-Label Mechanical Patent Classification, Jie Hu, Shaobo Li, Jianjun Hu, Guanci Yang

Faculty Publications

Various studies have focused on feature extraction methods for automatic patent classification in recent years. However, most of these approaches are based on the knowledge from experts in related domains. Here we propose a hierarchical feature extraction model (HFEM) for multi-label mechanical patent classification, which is able to capture both local features of phrases as well as global and temporal semantics. First, a n-gram feature extractor based on convolutional neural networks (CNNs) is designed to extract salient local lexical-level features. Next, a long dependency feature extraction model based on the bidirectional long–short-term memory (BiLSTM) neural network model is proposed to …

Machine Learning Based Disease Gene Identification And Mhc Immune Protein-Peptide Binding Prediction, Zhonghao Liu

Theses and Dissertations

Machine learning and deep learning methods have been increasingly applied to solve challenging and important bioinformatics problems such as protein structure prediction, disease gene identification, and drug discovery. However, the performances of existing machine learning based predictive models are still not satisfactory. The question of how to exploit the specific properties of bioinformatics data and couple them with the unique capabilities of the learning algorithms remains elusive. In this dissertation, we propose advanced machine learning and deep learning algorithms to address two important problems: mislocation-related cancer gene identification and major histocompatibility complex-peptide binding affinity prediction. Our first contribution proposes a …

Phylogeny, Ancestral Genome, And Disease Diagnoses Models Constructions Using Biological Data, Bing Feng

Theses and Dissertations

Studies of bioinformatics develop methods and software tools to analyze the biological data and provide insight of the mechanisms of biological process. Machine learning techniques have been widely used by researchers for disease prediction, disease diagnosis, and bio-marker identification. Using machine-learning algorithms to diagnose diseases has a couple of advantages. Besides solely relying on the doctors’ experiences and stereotyped formulas, researchers could use learning algorithms to analyze sophisticated, high-dimensional and multimodal biomedical data, and construct prediction/classification models to make decisions even when some information was incomplete, unknown, or contradictory. In this study, first of all, we built an automated computational …

Authenticating Users With 3d Passwords Captured By Motion Sensors, Jing Tian

Theses and Dissertations

Authentication plays a key role in securing various resources including corporate facilities or electronic assets. As the most used authentication scheme, knowledgebased authentication is easy to use but its security is bounded by how much a user can remember. Biometrics-based authentication requires no memorization but ‘resetting’ a biometric password may not always be possible. Thus, we propose study several behavioral biometrics (i.e., mid-air gestures) for authentication which does not have the same privacy or availability concerns as of physiological biometrics.

In this dissertation, we first propose a user-friendly authentication system Kin- Write that allows users to choose arbitrary, short and …

Implementation Costs Of Spiking Versus Rate-Based Anns, Lacie Renee Stiffler

Theses and Dissertations

Artificial neural networks are an effective machine learning technique for a variety of data sets and domains, but exploiting the inherent parallelism in neural networks requires specialized hardware. Typically, computing the output of each neuron requires many multiplications, evaluation of a transcendental activation function, and transfer of its output to a large number of other neurons. These restrictions become more expensive when internal values are represented with increasingly higher data precision. A spiking neural network eliminates the limitations of typical rate-based neural networks by reducing neuron output and synapse weights to one-bit values, eliminating hardware multipliers, and simplifying the activation …

Uncertainty Estimation Of Deep Neural Networks, Chao Chen

Theses and Dissertations

Normal neural networks trained with gradient descent and back-propagation have received great success in various applications. On one hand, point estimation of the network weights is prone to over-fitting problems and lacks important uncertainty information associated with the estimation. On the other hand, exact Bayesian neural network methods are intractable and non-applicable for real-world applications. To date, approximate methods have been actively under development for Bayesian neural networks, including but not limited to: stochastic variational methods, Monte Carlo dropouts, and expectation propagation. Though these methods are applicable for current large networks, there are limits to these approaches with either underestimation …

Improving Speech-Related Facial Action Unit Recognition By Audiovisual Information Fusion, Zibo Meng

Theses and Dissertations

In spite of great progress achieved on posed facial display and controlled image acquisition, performance of facial action unit (AU) recognition degrades significantly for spontaneous facial displays. Furthermore, recognizing AUs accompanied with speech is even more challenging since they are generally activated at a low intensity with subtle facial appearance/geometrical changes during speech, and more importantly, often introduce ambiguity in detecting other co-occurring AUs, e.g., producing non-additive appearance changes. All the current AU recognition systems utilized information extracted only from visual channel. However, sound is highly correlated with visual channel in human communications. Thus, we propose to exploit both audio …