Data Science Commons^™

Lexical Complexity Prediction With Assembly Models, Aadil Islam

Dartmouth College Undergraduate Theses

Tuning the complexity of one's writing is essential to presenting ideas in a logical, intuitive manner to audiences. This paper describes a system submitted by team BigGreen to LCP 2021 for predicting the lexical complexity of English words in a given context. We assemble a feature engineering-based model and a deep neural network model with an underlying Transformer architecture based on BERT. While BERT itself performs competitively, our feature engineering-based model helps in extreme cases, eg. separating instances of easy and neutral difficulty. Our handcrafted features comprise a breadth of lexical, semantic, syntactic, and novel phonetic measures. Visualizations of BERT …

Fine-Grained Detection Of Hate Speech Using Bertoxic, Yakoob Khan

Dartmouth College Undergraduate Theses

This thesis describes our approach towards the fine-grained detection of hate speech using deep learning. We leverage the transformer encoder architecture to propose BERToxic, a system that fine-tunes a pre-trained BERT model to locate toxic text spans in a given text and utilizes additional post-processing steps to refine the prediction boundaries. The post-processing steps involve (1) labeling character offsets between consecutive toxic tokens as toxic and (2) assigning a toxic label to words that have at least one token labeled as toxic. Through experiments, we show that these two post-processing steps improve the performance of our model by 4.16% on …

Improving Existing Methods For Calculating Embodied Carbon Emissions In Trade Through Feature Discovery: An Information Theoretic Approach, Sam Morton

Dartmouth College Undergraduate Theses

The continued societal and ecological risks posed by climate change have spurred renewed interest in quantitative tools that can improve policy aimed at climate mitigation. In 2008, international trade accounted for up to 26\% of global anthropogenic emissions, and therefore trade has garnered increased attention from policymakers seeking carbon mitigation. The concept of embodied carbon emissions in trade (EET) quantifies overall carbon emitted in the production and transport of goods for the purposes of trade. EET in theory could prove an indispensable tool to climate-concerned policymakers, but current implementations and data availability limit EET calculation to annual snapshots that extend …

Exploring The Long Tail, Joseph H. Hajjar

Dartmouth College Undergraduate Theses

The migration of datasets online has created a near-infinite inventory for big name retailers such as Amazon and Netflix, giving rise to recommendation systems to assist users in navigating the massive catalog. This has also allowed for the possibility of retailers storing much less popular, uncommon items which would not appear in a more traditional brick-and-mortar setting due to the cost of storage. Nevertheless, previous work has highlighted the profit potential which lies in the so-called "long tail'' of niche, unpopular items. Unfortunately, due to the limited amount of data in this subset of the inventory, recommendation systems often struggle …

Advancing The Ability To Predict Cognitive Decline And Alzheimer’S Disease Based On Genetic Variants Beyond Amyloid-Beta And Tau, Naveen Rawat

Master's Projects

A growing amount of neurodegenerative R&D is focused on identifying genomic- based explanations of AD that are beyond Amyloid-b and Tau. The proposed effort involves identifying some of the genomic variations, such as single nucleotide polymorphisms (SNPs), allele , chromosome, epigenetic contributors to MCI and AD that are beyond Aβ and Tau.

The project involves building a prediction model based on a support vector machine (SVM) classifier that takes into account the genomic variations and epigenetic factors to predict the early stage of mild cognitive impairment (MCI) and Alzheimer disease (AD). To achieve this, picking up important feature sets which …

Do Animated Line Graphs Increase Risk Inferences?, Junghan Kim, Arun Lakshmanan

Research Collection Lee Kong Chian School Of Business

This article shows that animated display of time-varying data (e.g., stock or commodity prices) enhances risk judgments. We outline a process whereby animated display enhances the visual salience of transitions in a trajectory (i.e., successive changes in data values), which leads to transitions being utilized more to form cognitive inferences about risk. In turn, this leads to inflated risk judgments. The studies reported in this article provide converging evidence via eye tracking (Study 1), serial mediation analyses (Studies 2 and 3), and experimental manipulations of transition salience (graph type; Study 3) and utilization of transitions (global trend; Study 4 and …

Exploring The Use Of Social Media To Infer Relationships Between Demographics, Psychographics And Vaccine Hesitancy, Abhimanyu Kapur

Computer Science Senior Theses

The growing popularity of social media as a platform to obtain information and share one's opinions on various topics makes it a rich source of information for research. In this study, we aimed to develop a framework to infer relationships between demographic and psychographic characteristics of a user and their opinion on a specific narrative - in this case, their stance on taking the COVID-19 vaccine. Twitter was the chosen platform due to the large USA user base and easily available data. Demographic traits included Race, Age, Gender, and Human-vs-Organization Status. Psychographic traits included the Big Five personality traits (Conscientiousness, …

Investigating Daily Fantasy Baseball: An Approach To Automated Lineup Generation, Ryan Smith

Master's Theses

A recent trend among sports fans along both sides of the letterman jacket is that of Daily Fantasy Sports (DFS). The DFS industry has been under legal scrutiny recently, due to the view that daily sports data is too random to make its prediction skillful. Therefore, a common view is that it constitutes online gambling. This thesis proves that DFS, as it pertains to Baseball, is significantly more predictable than random chance, and thus does not constitute gambling.

We propose a system which generates daily lists of lineups for Fanduel Daily Fantasy Baseball contests. The system consists of two components: …

Soarnet, Deep Learning Thermal Detection For Free Flight, Jake T. Tallman

Master's Theses

Thermals are regions of rising hot air formed on the ground through the warming of the surface by the sun. Thermals are commonly used by birds and glider pilots to extend flight duration, increase cross-country distance, and conserve energy. This kind of powerless flight using natural sources of lift is called soaring. Once a thermal is encountered, the pilot flies in circles to keep within the thermal, so gaining altitude before flying off to the next thermal and towards the destination. A single thermal can net a pilot thousands of feet of elevation gain, however estimating thermal locations is not …

Modeling The Spread Of Covid-19 Over Varied Contact Networks, Ryan L. Solorzano

Master's Theses

When attempting to mitigate the spread of an epidemic without the use of a vaccine, many measures may be made to dampen the spread of the disease such as physically distancing and wearing masks. The implementation of an effective test and quarantine strategy on a population has the potential to make a large impact on the spread of the disease as well. Testing and quarantining strategies become difficult when a portion of the population are asymptomatic spreaders of the disease. Additionally, a study has shown that randomly testing a portion of a population for asymptomatic individuals makes a small impact …

A Performance Survey Of Text-Based Sentiment Analysis Methods For Automating Usability Evaluations, Kelsi Van Damme

Master's Theses

Usability testing, or user experience (UX) testing, is increasingly recognized as an important part of the user interface design process. However, evaluating usability tests can be expensive in terms of time and resources and can lack consistency between human evaluators. This makes automation an appealing expansion or alternative to conventional usability techniques.

Early usability automation focused on evaluating human behavior through quantitative metrics but the explosion of opinion mining and sentiment analysis applications in recent decades has led to exciting new possibilities for usability evaluation methods.

This paper presents a survey of modern, open-source sentiment analyzers’ usefulness in extracting and …

Per-Pixel Cloud Cover Classification Of Multispectral Landsat-8 Data, Salome E. Carrasco, Torrey J. Wagner, Brent T. Langhals

Faculty Publications

Random forest and neural network algorithms are applied to identify cloud cover using 10 of the wavelength bands available in Landsat 8 imagery. The methods classify each pixel into 4 different classes: clear, cloud shadow, light cloud, or cloud. The first method is based on a fully connected neural network with ten input neurons, two hidden layers of 8 and 10 neurons respectively, and a single-neuron output for each class. This type of model is considered with and without L2 regularization applied to the kernel weighting. The final model type is a random forest classifier created from an ensemble of …

Learn Biologically Meaningful Representation With Transfer Learning, Di He

Dissertations, Theses, and Capstone Projects

Machine learning has made significant contributions to bioinformatics and computational biol­ogy. In particular, supervised learning approaches have been widely used in solving problems such as bio­marker identification, drug response prediction, and so on. However, because of the limited availability of comprehensively labeled and clean data, constructing predictive models in super­ vised settings is not always desirable or possible, especially when using data­hunger, red­hot learning paradigms such as deep learning methods. Hence, there are urgent needs to develop new approaches that could leverage more readily available unlabeled data in driving successful machine learning ap­ plications in this area.

In my dissertation, …

Stationary Probability Distributions Of Stochastic Gradient Descent And The Success And Failure Of The Diffusion Approximation, William Joseph Mccann

Theses

In this thesis, Stochastic Gradient Descent (SGD), an optimization method originally popular due to its computational efficiency, is analyzed using Markov chain methods. We compute both numerically, and in some cases analytically, the stationary probability distributions (invariant measures) for the SGD Markov operator over all step sizes or learning rates. The stationary probability distributions provide insight into how the long-time behavior of SGD samples the objective function minimum.

A key focus of this thesis is to provide a systematic study in one dimension comparing the exact SGD stationary distributions to the Fokker-Planck diffusion approximation equations —which are commonly used in …

Time Series Forecasting With Applications To Finance, Viswapriya Misra

Theses

In finance, many phenomena are modeled as time series. This thesis investigates time series forecasting problems in finance, precisely the stock price prediction problem. We employ and compare traditional statistical algorithms like MA, ARIMA, and ARMA-GARCH with newly developed deep learning-based algorithms such RNNs, LSTMs, GRUs, TCNs, and bidirectional LSTMs and GRUs for predicting stock prices. We perform a comprehensive study and present all the experimental results on different datasets. We find that ARIMA and GRU perform better for single-step stock price prediction than other deep learning architectures. Adding market and economic indicators do not improve the performance of the …

Short Term Temperature Forecasting Using Lstms, And Cnn, Darshan Shah

Theses

Weather forecasting is a vital application in present times. We can use the predictions to minimize the weather related loss. Use of machine learning and deep learning algorithms for forecasting, can eliminate or reduce the necessity of big data and high computation dependent process of parameterization. Long Short-Term Memory (LSTM) is a widely used deep learning architecture for time series forecasting. In this paper, we aim to predict one day ahead average temperature using a 2-layer neural network consisting of one layer of LSTM and one layer of 1D convolution. The input is pre-processed using a smoothing technique and output …

Convolutional Neural Networks For Deflate Data Encoding Classification Of High Entropy File Fragments, Nehal Ameen

LSU New Orleans Theses and Dissertations

Data reconstruction is significantly improved in terms of speed and accuracy by reliable data encoding fragment classification. To date, work on this problem has been successful with file structures of low entropy that contain sparse data, such as large tables or logs. Classifying compressed, encrypted, and random data that exhibit high entropy is an inherently difficult problem that requires more advanced classification approaches. We explore the ability of convolutional neural networks and word embeddings to classify deflate data encoding of high entropy file fragments after establishing ground truth using controlled datasets. Our model is designed to either successfully classify file …

Machine Learning Based Restaurant Sales Forecasting, Austin B. Schmidt

LSU New Orleans Theses and Dissertations

To encourage proper employee scheduling for managing crew load, restaurants have a need for accurate sales forecasting. We predict partitions of sales days, so each day is broken up into three sales periods: 10:00 AM-1:59 PM, 2:00 PM-5:59 PM, and 6:00 PM-10:00 PM. This study focuses on the middle timeslot, where sales forecasts should extend for one week. We gather three years of sales between 2016-2019 from a local restaurant, to generate a new dataset for researching sales forecasting methods.

Outlined are methodologies used when going from raw data to a workable dataset. We test many machine learning models on …

Machine Learning For Terminal Procedure Chart Change Detection, Anthony M. Marchiafava

LSU New Orleans Theses and Dissertations

Terminal Procedure Charts are a constantly updated and necessary tool for aircraft personnel to approach and take off from airport runways safely. Detecting changes within these charts is a time-consuming and laborious process. Here machine learning techniques were used to predict regions of change in charts based on detecting the charts image regions and comparing features extracted from those regions. Outlined are methodologies to detect differences between two separate charts to produce images with changed regions clearly indicated. Both more conventional computer vision and machine learning techniques were applied. For images with minor shifts, the proposed model is able to …

Stock Markets Performance During A Pandemic: How Contagious Is Covid-19?, Yara Abushahba

Theses and Dissertations

Background and Motivation: The coronavirus (“COVID-19”) pandemic, the subsequent policies and lockdowns have unarguably led to an unprecedented fluid circumstance worldwide. The panic and fluctuations in the stock markets were unparalleled. It is inarguable that real-time availability of news and social media platforms like Twitter played a vital role in driving the investors’ sentiment during such global shock.

Purpose:The purpose of this thesis is to study how the investor sentiment in relation to COVID-19 pandemic influenced stock markets globally and how stock markets globally are integrated and contagious. We analyze COVID-19 sentiment through the Twitter posts and investigate its …