Digital Commons Network^™

Attention Mechanism In Deep Neural Networks For Computer Vision Tasks, Haohan Li

Doctoral Dissertations

“Attention mechanism, which is one of the most important algorithms in the deep Learning community, was initially designed in the natural language processing for enhancing the feature representation of key sentence fragments over the context. In recent years, the attention mechanism has been widely adopted in solving computer vision tasks by guiding deep neural networks (DNNs) to focus on specific image features for better understanding the semantic information of the image. However, the attention mechanism is not only capable of helping DNNs understand semantics, but also useful for the feature fusion, visual cue discovering, and temporal information selection, which are …

Benchmarking Machine Learning Methods For Molecular Property Prediction, Govinda Bahadur Kc

Open Access Theses & Dissertations

Machine learning (ML) techniques have been widely applied in a variety of areas ranging from pattern recognition, natural language processing, and computer games to self-driving cars, clinical diagnostics, and molecular structure prediction easing day to day life of human beings. Drug discovery is an expensive, complex, and time taking process. Currently, the pharma industry is hoping to leverage machine learning methods in expediting the drug discovery process. Molecular property prediction is one of the most important tasks in drug discovery. While developing a new drug relies on a proper understanding of molecular properties, there has been great interest in the …

Ordinal Hyperplane Loss, Bob Vanderheyden

Doctor of Data Science and Analytics Dissertations

This research presents the development of a new framework for analyzing ordered class data, commonly called “ordinal class” data. The focus of the work is the development of classifiers (predictive models) that predict classes from available data. Ratings scales, medical classification scales, socio-economic scales, meaningful groupings of continuous data, facial emotional intensity and facial age estimation are examples of ordinal data for which data scientists may be asked to develop predictive classifiers. It is possible to treat ordinal classification like any other classification problem that has more than two classes. Specifying a model with this strategy does not fully utilize …

Evaluating The Applicability Of Deep Learning Techniques In Agricultural Systems Modeling, Babak Saravi

Theses and Dissertations

A rapidly expanding world population and extreme climate change have made food production a crucial challenge in the twentyirst century. Therefore, improving crop production through agricultural management could be an effective solution for this challenge. However, due to the associated cost and time to perform field works, researchers widely rely on agricultural system modeling to examine the impacts of different crop management scenarios. However, due to the complexity of agricultural system modeling, their applications in producing practical knowledge for producers are limited. Concurrently, deep learning techniques have been recognized as a preferred method when dealing with large datasets. This study …

Using Machine Learning Classification Methods To Detect The Presence Of Heart Disease, Nestor Pereira

Dissertations

Cardiovascular disease (CVD) is the most common cause of death in Ireland, and probably, worldwide. According to the Health Service Executive (HSE) cardiovascular disease accounting for 36% of all deaths, and one important fact, 22% of premature deaths (under age 65) are from CVD.

Using data from the Heart Disease UCI Data Set (UCI Machine Learning), we use machine learning techniques to detect the presence or absence of heart disease in the patient according to 14 features provide for this dataset. The different results are compared based on accuracy performance, confusion matrix and area under the Receiver Operating Characteristics (ROC) …

Improved Generalisation Bounds For Deep Learning Through L∞ Covering Numbers, Antoine Ledent, Yunwen Lei, Marius Kloft

Research Collection School Of Computing and Information Systems

Using proof techniques involving L∞ covering numbers, we show generalisation error bounds for deep learning with two main improvements over the state of the art. First, our bounds have no explicit dependence on the number of classes except for logarithmic factors. This holds even when formulating the bounds in terms of the L 2 norm of the weight matrices, while previous bounds exhibit at least a square-root dependence on the number of classes in this case. Second, we adapt the Rademacher analysis of DNNs to incorporate weight sharing—a task of fundamental theoretical importance which was previously attempted only under very …

Dynamic Prediction Of Runway Configuration And Airport Acceptance Rate, Yuan Wang

USF Tampa Graduate Theses and Dissertations

Automated prediction of runway configuration and airport capacity is critical for the future generation of air traffic management. In the future aviation industry, multi-sources weather forecast information will be available for air traffic decision-making units; how to use these data efficiently is key for overall efficiency of air traffic management. Currently, air traffic management personnel lack tools to assist them to translate weather forecast data into real-time airport capacity. Runway configurations and AARs of airports in a multi-airport system are determined by different air traffic controller personnel. The lack of synchronization may lead to the loss of efficiency of the …

Essays Of The Visual Effects On Online Human Decision-Making And Data Science Applications, Jian-Ren Hou

Information Systems & Operations Management Dissertations - Archive

How human makes decisions online is an important research area in IS, many different techniques and online scenarios are used to probe into the related issues. In the first essay, this study investigates the crowdfunding platform. This study uses a deep neural network to extract the emotion metrics from the project images on the crowdfunding platform. The result shows that emotions in project images such as sadness and contentment can positively affect the performance of crowdfunding projects. This study also provides possible designs to change the emotions in the project images. In the second essay, this study investigates how to …

Precision Medicine In Pancreatic Disease-Knowledge Gaps And Research Opportunities: Summary Of A National Institute Of Diabetes And Digestive And Kidney Diseases Workshop., Mark E Lowe, Dana K Andersen, Richard M Caprioli, Jyoti Choudhary, Zobeida Cruz-Monserrate, Anil K Dasyam, Christopher E Forsmark, Fred S Gorelick, Joe W Gray, Mark Haupt, Kimberly A Kelly, Kenneth P Olive, Sylvia K Plevritis, Noa Rappaport, Holger R Roth, Hanno Steen, S Joshua Swamidass, Temel Tirkes, Aliye Uc, Kirill Veselkov, David C Whitcomb, Aida Habtezion

Articles, Abstracts, and Reports

A workshop on research gaps and opportunities for Precision Medicine in Pancreatic Disease was sponsored by the National Institute of Diabetes and Digestive Kidney Diseases on July 24, 2019, in Pittsburgh. The workshop included an overview lecture on precision medicine in cancer and 4 sessions: (1) general considerations for the application of bioinformatics and artificial intelligence; (2) omics, the combination of risk factors and biomarkers; (3) precision imaging; and (4) gaps, barriers, and needs to move from precision to personalized medicine for pancreatic disease. Current precision medicine approaches and tools were reviewed, and participants identified knowledge gaps and research needs …

A Deep Learning Approach For Tweet Classification And Rescue Scheduling For Effective Disaster Management, Md. Yasin Kabir, Sanjay Kumar Madria

Computer Science Faculty Research & Creative Works

Every activity in disaster management demands accurate and up-todate information to allow a quick, easy, and cost-efective response to reduce the possible loss of lives and properties. It is a challenging and complex task to acquire information from diferent regions of a disaster-afected area in a timely fashion. The extensive spread and reach of social media and networks such as Twitter allow people to share information in real-time. However, gathering of valuable information requires a series of operations such as (1) processing each tweet for the text classiication, (2) possible location determination of people needing help based on tweets, and …

Insider’S Misuse Detection: From Hidden Markov Model To Deep Learning, Ahmed Saaudi

Theses and Dissertations

Malicious insiders increasingly affect organizations by leaking classified data to unautho- rized entities. Detecting insiders’ misuses in computer systems is a challenging problem. In this dissertation, we propose two approaches to detect such threats: a probabilistic graph- ical model-based approach and a deep learning-based approach. We investigate the logs of computer-based activities to discover patterns of misuse. We model user’s behaviors as sequences of computer-based events.

For our probabilistic graphical model-based approach, we propose an unsupervised model for insider’s misuse detection. That is, we develop Stochastic Gradient Descent method to learn Hidden Markov Models (SGD-HMM) with the goal of analyzing …

Similarity-Based Chained Transfer Learning For Energy Forecasting With Big Data, Yifang Tian, Ljubisa Sehovac, Katarina Grolinger

Electrical and Computer Engineering Publications

Smart meter popularity has resulted in the ability to collect big energy data and has created opportunities for large-scale energy forecasting. Machine Learning (ML) techniques commonly used for forecasting, such as neural networks, involve computationally intensive training typically with data from a single building or a single aggregated load to predict future consumption for that same building or aggregated load. With hundreds of thousands of meters, it becomes impractical or even infeasible to individually train a model for each meter. Consequently, this paper proposes Similarity-Based Chained Transfer Learning (SBCTL), an approach for building neural network-based models for many meters by …

Pristine Sentence Translation: A New Approach To A Timeless Problem, Meenu Ahluwalia, Brian Coari, Ben Brock

SMU Data Science Review

Abstract.

Pristine Sentence Translation (PST) is a new approach to language translation based upon sentence-level granularity. Traditional translation approaches, including those utilizing advanced machine learning or neural network-based approaches, translate on a word-by-word or phrase-by-phrase basis; thereby, potentially missing the context or meaning of the complete sentence. Instead of these piecewise translations, PST utilizes deep learning and predictive modeling techniques to translate complete sentences from their source language into their target language. With these approaches we were able to translate sentences that closely conveyed the meaning of the original sentences. Our results demonstrated that PST’s method of translating an entire …

Texture-Based Deep Neural Network For Histopathology Cancer Whole Slide Image (Wsi) Classification, Nelson Zange Tsaku

Master of Science in Computer Science Theses

Automatic histopathological Whole Slide Image (WSI) analysis for cancer classification has been highlighted along with the advancements in microscopic imaging techniques. However, manual examination and diagnosis with WSIs is time-consuming and tiresome. Recently, deep convolutional neural networks have succeeded in histopathological image analysis. In this paper, we propose a novel cancer texture-based deep neural network (CAT-Net) that learns scalable texture features from histopathological WSIs. The innovation of CAT-Net is twofold: (1) capturing invariant spatial patterns by dilated convolutional layers and (2) Reducing model complexity while improving performance. Moreover, CAT-Net can provide discriminative texture patterns formed on cancerous regions of histopathological …

Acute Angle Repositioning In Mobile C-Arm Using Image Processing And Deep Learning, Armin Yazdanshenas

Mechanical Engineering Theses

During surgery, medical practitioners rely on the mobile C-Arm medical x-ray system (C-Arm) and its fluoroscopic functions to not only perform the surgery but also validate the outcome. Currently, technicians reposition the C-Arm arbitrarily through estimation and guesswork. In cases when the positioning and repositioning of the C-Arm are critical for surgical assessment, uncertainties in the angular position of the C-Arm components hinder surgical performance. This thesis proposes an integrated approach to automatically reposition C-Arms during critically acute movements in orthopedic surgery. Robot vision and control with deep learning are used to determine the necessary angles of rotation for desired …

Deep Anomaly Detection With Deviation Networks, Guansong Pang, Chunhua Shen, Anton Van Den Hengel

Research Collection School Of Computing and Information Systems

Although deep learning has been applied to successfully address many data mining problems, relatively limited work has been done on deep learning for anomaly detection. Existing deep anomaly detection methods, which focus on learning new feature representations to enable downstream anomaly detection methods, perform indirect optimization of anomaly scores, leading to data-inefficient learning and suboptimal anomaly scoring. Also, they are typically designed as unsupervised learning due to the lack of large-scale labeled anomaly data. As a result, they are difficult to leverage prior knowledge (e.g., a few labeled anomalies) when such information is available as in many real-world anomaly detection …

Towards Robust Resnet: A Small Step But A Giant Leap, Jingfeng Zhang, Bo Han, Laura Wynter, Bryan Kian Hsiang Low, Mohan Kankanhalli

Research Collection School Of Computing and Information Systems

This paper presents a simple yet principled approach to boosting the robustness of the residual network (ResNet) that is motivated by a dynamical systems perspective. Namely, a deep neural network can be interpreted using a partial differential equation, which naturally inspires us to characterize ResNet based on an explicit Euler method. This consequently allows us to exploit the step factor h in the Euler method to control the robustness of ResNet in both its training and generalization. In particular, we prove that a small step factor h can benefit its training and generalization robustness during backpropagation and forward propagation, respectively. …

Action Recognition In Manufacturing Assembly Using Multimodal Sensor Fusion, Md. Al-Amin, Wenjin Tao, David Doell, Ravon Lingard, Zhaozheng Yin, Ming-Chuan Leu, Ruwen Qin

Computer Science Faculty Research & Creative Works

Production innovations are occurring faster than ever. Manufacturing workers thus need to frequently learn new methods and skills. In fast changing, largely uncertain production systems, manufacturers with the ability to comprehend workers' behavior and assess their operation performance in near real-time will achieve better performance than peers. Action recognition can serve this purpose. Despite that human action recognition has been an active field of study in machine learning, limited work has been done for recognizing worker actions in performing manufacturing tasks that involve complex, intricate operations. Using data captured by one sensor or a single type of sensor to recognize …

Gold & Blue, Summer 2019, St. Mary's University- San Antonio, Texas

Gold & Blue

This is the Summer 2019 edition of Gold & Blue which is St. Mary's University's official school publication.

Sequential Survival Analysis With Deep Learning, Seth William Glazier

Theses and Dissertations

Survival Analysis is the collection of statistical techniques used to model the time of occurrence, i.e. survival time, of an event of interest such as death, marriage, the lifespan of a consumer product or the onset of a disease. Traditional survival analysis methods rely on assumptions that make it difficult, if not impossible to learn complex non-linear relationships between the covariates and survival time that is inherent in many real world applications. We first demonstrate that a recurrent neural network (RNN) is better suited to model problems with non-linear dependencies in synthetic time-dependent and non-time-dependent experiments.

Deepsz: A Novel Framework To Compress Deep Neural Networks By Using Error-Bounded Lossy Compression, Sian Jin, Sheng Di, Xin Liang, Jiannan Tian, Dingwen Tao, Franck Cappello

Computer Science Faculty Research & Creative Works

Today's deep neural networks (DNNs) are becoming deeper and wider because of increasing demand on the analysis quality and more and more complex applications to resolve. The wide and deep DNNs, however, require large amounts of resources (such as memory, storage, and I/O), significantly restricting their utilization on resource-constrained platforms. Although some DNN simplification methods (such as weight quantization) have been proposed to address this issue, they suffer from either low compression ratios or high compression errors, which may introduce an expensive fine-tuning overhead (i.e., a costly retraining process for the target inference accuracy). In this paper, we propose DeepSZ: …

A Study On Large-Scale Deep Learning In Bioinformatics And Biomedical Applications, Shayan Shams

LSU Doctoral Dissertations

Recent advances in Artificial Intelligence and deep learning have provided researchers in various fields insights into the analysis of multiple datasets. These applications include image analysis, text analysis, and many more. However, the effectiveness of deep learning in some areas, such as biomedical imaging and genomic research, has been overshadowed by the variance in the types and complexity of data. This is in addition to the expensive labeling process and the limited size of datasets in these fields. These challenges require advanced deep learning models capable of learning from a small dataset and also from a small number of labeled …

Multi-Resolution Models For Learning Multilevel Abstract Representation With Application To Information Retrieval, Tolgahan Cakaloglu

Theses and Dissertations

Deep language models learning a hierarchical representation proved to be a powerful tool for natural language processing, text mining, and information retrieval tasks. However, more specifically, representations that perform well for ad-hoc retrieval must capture semantic meaning at different levels of abstraction or context-scopes. The primary goal of ad-hoc retrieval is to find relevant documents satisfying the information need posted in a natural language query. It requires a good understanding of the query and all the documents in a corpus, which is difficult because the meaning of natural language texts depends on the context, syntax, and semantics. In this dissertation, …

Forecasting Building Energy Consumption With Deep Learning: A Sequence To Sequence Approach, Ljubisa Sehovac, Cornelius Nesen, Katarina Grolinger

Electrical and Computer Engineering Publications

Energy Consumption has been continuously increasing due to the rapid expansion of high-density cities, and growth in the industrial and commercial sectors. To reduce the negative impact on the environment and improve sustainability, it is crucial to efficiently manage energy consumption. Internet of Things (IoT) devices, including widely used smart meters, have created possibilities for energy monitoring as well as for sensor based energy forecasting. Machine learning algorithms commonly used for energy forecasting such as feedforward neural networks are not well-suited for interpreting the time dimensionality of a signal. Consequently, this paper uses Recurrent Neural Networks (RNN) to capture time …

Sliced Wasserstein Generative Models, Jiqing Wu, Zhiwu Huang, Dinesh Acharya, Wen Li, Janine Thoma, Danda Pani Paudel, Luc Van Gool

Research Collection School Of Computing and Information Systems

In generative modeling, the Wasserstein distance (WD) has emerged as a useful metric to measure the discrepancy between generated and real data distributions. Unfortunately, it is challenging to approximate the WD of high-dimensional distributions. In contrast, the sliced Wasserstein distance (SWD) factorizes high-dimensional distributions into their multiple one-dimensional marginal distributions and is thus easier to approximate. In this paper, we introduce novel approximations of the primal and dual SWD. Instead of using a large number of random projections, as it is done by conventional SWD approximation methods, we propose to approximate SWDs with a small number of parameterized orthogonal projections …

Deep Learning Based Real Time Devanagari Character Recognition, Aseem Chhabra

Master's Projects

The revolutionization of the technology behind optical character recognition (OCR) has helped it to become one of those technologies that have found plenty of uses in the entire industrial space. Today, the OCR is available for several languages and have the capability to recognize the characters in real time, but there are some languages for which this technology has not developed much. All these advancements have been possible because of the introduction of concepts like artificial intelligence and deep learning. Deep Neural Networks have proven to be the best choice when it comes to a task involving recognition. There are …

Learning To Play The Trading Game, Neeraj Kulkarni

Master's Projects

Can we train a stock trading bot that can take decisions in high-entropy envi- ronments like stock markets to generate profits based on some optimal policy? Can we further extend this learning for any general trading problem? Quantitative Al- gorithms are responsible for more than 75% of the stock trading around the world. Creating a stock market prediction model is comparatively easy. But creating a prof- itable prediction model is still considered as a challenging task in the field of machine learning and deep learning due to the unpredictability of the financial markets. Us- ing biologically inspired computing techniques of …

Detecting Cars In A Parking Lot Using Deep Learning, Samuel Ordonia

Master's Projects

Detection of cars in a parking lot with deep learning involves locating all objects of interest in a parking lot image and classifying the contents of all bounding boxes as cars. Because of the variety of shape, color, contrast, pose, and occlusion, a deep neural net was chosen to encompass all the significant features required by the detector to differentiate cars from not cars. In this project, car detection was accomplished with a convolutional neural net (CNN) based on the You Only Look Once (YOLO) model architectures. An application was built to train and validate a car detection CNN as …

Pose Estimation And Action Recognition In Sports And Fitness, Parth Vyas

Master's Projects

The emergence of large datasets and major improvements in Deep Learning has lead to many real-world applications. These applications have been focused on automotive markets, mobile markets, stock markets, and the healthcare market. Although Deep Learning has strong foundations across many areas, the few applications in Sports, Fitness, or even Injury Rehabilitation could benefit greatly from it. For example, if you are performing a workout and you need to evaluate your form, but do not have access or resources for an instructor to evaluate your form, it would be great to have an Artificial Intelligent agent provide real time feedback …

Self-Driving Cars: Evaluation Of Deep Learning Techniques For Object Detection In Different Driving Conditions, Ramesh Simhambhatla, Kevin Okiah, Shravan Kuchkula, Robert Slater

SMU Data Science Review

Deep Learning has revolutionized Computer Vision, and it is the core technology behind capabilities of a self-driving car. Convolutional Neural Networks (CNNs) are at the heart of this deep learning revolution for improving the task of object detection. A number of successful object detection systems have been proposed in recent years that are based on CNNs. In this paper, an empirical evaluation of three recent meta-architectures: SSD (Single Shot multi-box Detector), R-CNN (Region-based CNN) and R-FCN (Region-based Fully Convolutional Networks) was conducted to measure how fast and accurate they are in identifying objects on the road, such as vehicles, pedestrians, …