Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 12 of 12
Full-Text Articles in Physical Sciences and Mathematics
Dispensing With Humans In Human-Computer Interaction Research, Courtni L. Byun
Dispensing With Humans In Human-Computer Interaction Research, Courtni L. Byun
Theses and Dissertations
Machine Learning models have become more advanced than could have been supposed even a few years ago, often surpassing human performance on many tasks. Large language models (LLM) can produce text indistinguishable from human-produced text. This begs the question, how necessary are humans - even for tasks where humans appear indispensable? Qualitative Analysis (QA) is integral to human-computer interaction research, requiring both human-produced data and human analysis of that data to illuminate human opinions about and experiences with technology. We use GPT-3 and ChatGPT to replace human analysis and then to dispense with human-produced text altogether. We find GPT-3 is …
Language Modeling Using Image Representations Of Natural Language, Seong Eun Cho
Language Modeling Using Image Representations Of Natural Language, Seong Eun Cho
Theses and Dissertations
This thesis presents training of an end-to-end autoencoder model using the transformer, with an encoder that can encode sentences into fixed-length latent vectors and a decoder that can reconstruct the sentences using image representations. Encoding and decoding sentences to and from these image representations are central to the model design. This method allows new sentences to be generated by traversing the Euclidean space, which makes vector arithmetic possible using sentences. Machines excel in dealing with concrete numbers and calculations, but do not possess an innate infrastructure designed to help them understand abstract concepts like natural language. In order for a …
Hierarchical Joint Entity Recognition And Relation Extraction Of Contextual Entities In Family History Records, Daniel Segrera
Hierarchical Joint Entity Recognition And Relation Extraction Of Contextual Entities In Family History Records, Daniel Segrera
Theses and Dissertations
Entity extraction is an important step in document understanding. Higher accuracy entity extraction on fine-grained entities can be achieved by combining the utility of Named Entity Recognition (NER) and Relation Extraction (RE) models. In this paper, a cascading model is proposed that implements NER and Relation extraction. This model utilizes relations between entities to infer context-dependent fine-grain named entities in text corpora. The RE module runs independent of the NER module, which reduces error accumulation from sequential steps. This process improves on the fine-grained NER F1-score of existing state-of-the-art from .4753 to .8563 on our data, albeit on a strictly …
Language Learning Using Models Of Intentionality In Repeated Games With Cheap Talk, Jonathan Berry Skaggs
Language Learning Using Models Of Intentionality In Repeated Games With Cheap Talk, Jonathan Berry Skaggs
Theses and Dissertations
Language is critical to establishing long-term cooperative relationships among intelligent agents (including people), particularly when the agents' preferences are in conflict. In such scenarios, an agent uses speech to coordinate and negotiate behavior with its partner(s). While recent work has shown that neural language modeling can produce effective speech agents, such algorithms typically only accept previous text as input. However, in relationships among intelligent agents, not all relevant context is expressed in conversation. Thus, in this paper, we propose and analyze an algorithm, called Llumi, that incorporates other forms of context to learn to speak in long-term relationships modeled as …
Symbolic Semantic Memory In Transformer Language Models, Robert Kenneth Morain
Symbolic Semantic Memory In Transformer Language Models, Robert Kenneth Morain
Theses and Dissertations
This paper demonstrates how transformer language models can be improved by giving them access to relevant structured data extracted from a knowledge base. The knowledge base preparation process and modifications to transformer models are explained. We evaluate these methods on language modeling and question answering tasks. These results show that even simple additional knowledge augmentation leads to a reduction in validation loss by 73%. These methods also significantly outperform common ways of improving language models such as increasing the model size or adding more data.
Turn Of Phrase: Contrastive Pre-Training For Discourse-Aware Conversation Models, Roland Laboulaye
Turn Of Phrase: Contrastive Pre-Training For Discourse-Aware Conversation Models, Roland Laboulaye
Theses and Dissertations
Understanding long conversations requires recognizing a discourse flow unique to conversation. Recent advances in unsupervised representation learning of text have been attained primarily through language modeling, which models discourse only implicitly and within a small window. These representations are in turn evaluated chiefly on sentence pair or paragraph-question pair benchmarks, which measure only local discourse coherence. In order to improve performance on discourse-reliant, long conversation tasks, we propose Turn-of-Phrase pre-training, an objective designed to encode long conversation discourse flow. We leverage tree-structured Reddit conversations in English to, relative to a chosen conversation path through the tree, select paths of varying …
Plot Extraction And The Visualization Of Narrative Flow, Michael A. Debuse
Plot Extraction And The Visualization Of Narrative Flow, Michael A. Debuse
Theses and Dissertations
In order to facilitate the automated extraction of complex features and structures within narrative, namely plot in this study, two proof-of-concept methods of narrative visualization are presented with the goal of representing the plot of the narrative. Plot is defined to give a basis for quality assessment and comparison. The first visualization presented is a scatter-plot of entities within the story, but due to failing to uphold the definition of plot, in-depth analysis is not performed. The second visualization presented is a graph structure that better represents a mapping of the plot of the story. Narrative structures commonly found within …
Leveraging The Inductive Bias Of Large Language Models For Abstract Textual Reasoning, Christopher Michael Rytting
Leveraging The Inductive Bias Of Large Language Models For Abstract Textual Reasoning, Christopher Michael Rytting
Theses and Dissertations
Large natural language models (such as GPT-2 or T5) demonstrate impressive abilities across a range of general NLP tasks. Here, we show that the knowledge embedded in such models provides a useful inductive bias, not just on traditional NLP tasks, but also in the nontraditional task of training a symbolic reasoning engine. We observe that these engines learn quickly and generalize in a natural way that reflects human intuition. For example, training such a system to model block-stacking might naturally generalize to stacking other types of objects because of structure in the real world that has been partially captured by …
Suggesting Missing Information In Text Documents, Grant Michael Hodgson
Suggesting Missing Information In Text Documents, Grant Michael Hodgson
Theses and Dissertations
A key part of contract drafting involves thinking of issues that have not been addressedand adding language that will address the missing issues. To assist attorneys with this task, we present a pipeline approach for identifying missing information within a contract section. The pipeline takes a contract section as input and includes 1) identifying sections that are similar to the input section from a corpus of contract sections; and 2) identifying and suggesting information from the similar sections that are missing from the input section. By taking advantage of sentence embedding and principal component analysis, this approach suggests sentences that …
Practical Cost-Conscious Active Learning For Data Annotation In Annotator-Initiated Environments, Robbie A. Haertel
Practical Cost-Conscious Active Learning For Data Annotation In Annotator-Initiated Environments, Robbie A. Haertel
Theses and Dissertations
Many projects exist whose purpose is to augment raw data with annotations that increase the usefulness of the data. The number of these projects is rapidly growing and in the age of “big data” the amount of data to be annotated is likewise growing within each project. One common use of such data is in supervised machine learning, which requires labeled data to train a predictive model. Annotation is often a very expensive proposition, particularly for structured data. The purpose of this dissertation is to explore methods of reducing the cost of creating such data sets, including annotated text corpora.We …
Generating Paraphrases With Greater Variation Using Syntactic Phrases, Rebecca Diane Madsen
Generating Paraphrases With Greater Variation Using Syntactic Phrases, Rebecca Diane Madsen
Theses and Dissertations
Given a sentence, a paraphrase generation system produces a sentence that says the same thing but usually in a different way. The paraphrase generation problem can be formulated in the machine translation paradigm; instead of translation of English to a foreign language, the system translates an English sentence (for example) to another English sentence. Quirk et al. (2004) demonstrated this approach to generate almost 90% acceptable paraphrases. However, most of the sentences had little variation from the original input sentence. Leveraging syntactic information, this thesis project presents an approach that successfully generated more varied paraphrase sentences than the approach of …
Surface Realization Using A Featurized Syntactic Statistical Language Model, Thomas L. Packer
Surface Realization Using A Featurized Syntactic Statistical Language Model, Thomas L. Packer
Theses and Dissertations
An important challenge in natural language surface realization is the generation of grammatical sentences from incomplete sentence plans. Realization can be broken into a two-stage process consisting of an over-generating rule-based module followed by a ranker that outputs the most probable candidate sentence based on a statistical language model. Thus far, an n-gram language model has been evaluated in this context. More sophisticated syntactic knowledge is expected to improve such a ranker. In this thesis, a new language model based on featurized functional dependency syntax was developed and evaluated. Generation accuracies and cross-entropy for the new language model did not …