Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 4 of 4
Full-Text Articles in Physical Sciences and Mathematics
Exploring Tokenization Techniques To Optimize Patch-Based Time-Series Transformers, Gabriel L. Asher
Exploring Tokenization Techniques To Optimize Patch-Based Time-Series Transformers, Gabriel L. Asher
Computer Science Senior Theses
Transformer architectures have revolutionized deep learning, impacting natural language processing and computer vision. Recently, PatchTST has advanced long-term time-series forecasting by embedding patches of time-steps to use as tokens for transformers. This study examines and seeks to enhance PatchTST's embedding techniques. Using eight benchmark datasets, we explore explore novel token embedding techniques. To this end, we introduce several PatchTST variants, which alter the embedding methods of the original paper. These variants consist of the following architectural changes: using CNNs to embed inputs to tokens, embedding an aggregate measure like the mean, max, or sum of a patch, adding the exponential …
Mitigating Safety Issues In Pre-Trained Language Models: A Model-Centric Approach Leveraging Interpretation Methods, Weicheng Ma
Mitigating Safety Issues In Pre-Trained Language Models: A Model-Centric Approach Leveraging Interpretation Methods, Weicheng Ma
Dartmouth College Ph.D Dissertations
Pre-trained language models (PLMs), like GPT-4, which powers ChatGPT, face various safety issues, including biased responses and a lack of alignment with users' backgrounds and expectations. These problems threaten their sociability and public application. Present strategies for addressing these safety concerns primarily involve data-driven approaches, requiring extensive human effort in data annotation and substantial training resources. Research indicates that the nature of these safety issues evolves over time, necessitating continual updates to data and model re-training—an approach that is both resource-intensive and time-consuming. This thesis introduces a novel, model-centric strategy for understanding and mitigating the safety issues of PLMs by …
Towards A Computational Model Of Narrative On Social Media, Anne Bailey
Towards A Computational Model Of Narrative On Social Media, Anne Bailey
Dartmouth College Undergraduate Theses
This thesis describes a variety of approaches to developing a computational model of narrative on social media. Our goal is to use such a narrative model to identify efforts to manipulate public opinion on social media platforms like Twitter. We present a model in which narratives in a collection of tweets are represented as a graph. Elements from each tweet that are relevant to potential narratives are made into nodes in the graph; for this thesis, we populate graph nodes with tweets’ authors, hashtags, named entities (people, locations, organizations, etc.,), and moral foundations (central moral values framing the discussion). Two …
Entity Based Sentiment Analysis For Textual Health Advice, Dae Lim Chung
Entity Based Sentiment Analysis For Textual Health Advice, Dae Lim Chung
Computer Science Senior Theses
This work explores entity based sentiment analysis for textual health advice through deep learning. We fine tuned a pretrained BERT model to analyze sentiments across five different predetermined categories which consist of food, medicine, disease, exercise, and vitality for three different sentiments: positive, negative, and neutral. Original set of annotated medical dataset from Dartmouth College’s Persist Lab was used to conduct the experiments. For the aim of tailoring the data for the purpose of entity based sentiment analysis, we explored data transformation techniques to generate optimum training examples. During the experiments, we were able to discover that the wide variety …