Data Science Commons^™

Identifying Textual Predictors Of Early Termination In Clinical Trials In Medicine: An Explainable Machine-Learning Study, Rohan Ramnarain

Dissertations, Theses, and Capstone Projects

About one in five clinical trials in medicine ends early, wasting valuable resources and reducing the evidence available for developing life-saving medical treatments. This project uses a method called Trial2Vec, which is a self-supervised machine-learning method that converts clinical trial documents into dense numerical representations that capture their key design and clinical characteristics, to turn each proposed clinical trial’s written protocol into a compact numerical profile (a process referred to as embedding). These profiles are then paired with a predictive machine learning models to identify the words and phrases in the trial documents that can signal a higher risk of …

A Modular Framework For Cost-Efficient Aspect-Based Sentiment Analysis Using Small Language Models, Senthil Kumar, Nibhrat Lohia

SMU Data Science Review

Aspect-based sentiment analysis (ABSA) links opinions in text to specific product attributes (for example, battery life, screen quality, or delivery speed) rather than only assigning an overall star rating. This level of detail is important in domains such as e-commerce, where teams need to know which features customers praised and which they criticized. Traditional ABSA pipelines have relied on large language models (LLMs), which achieved high quality but were expensive to run and difficult to scale. This study evaluated whether small language models (SLMs) in the 1–3 billion parameter range could serve as a lower-cost alternative. We implemented a modular …

Topic Modeling The Cuny Graduate Center's Dissertations And Theses, Michael Mandiberg

Open Educational Resources

This 4 week module is designed for Data Analysis, Data Visualization, and Digital Humanities courses at the MA/MS or advanced 400-level undergraduate level. It introduces students to textual analysis with topic modeling and requires a solid foundation in Python. The module uses Gensim and a Colab notebook to introduce a standard text analysis workflow used in Digital Humanities, archival research, and exploratory data analysis.

Students build a topic model describing 19,000 CUNY Graduate Center dissertations and theses. They work with an unexplored dataset to load and explore the data, prepare the corpus, train and evaluate a topic model, and interpret, …

Resale Revolution: Trend Implications From Media Presence Transcended To Luxury Retail Markets, Penelope Prochnow

Capstone Projects

This study aims to deepen understanding of fashion trend decline from peak popularity to obsolescence, with implications for sustainability and producer profit margins. It investigates how the attributes and media presence of fashion items influence their journey from high-end editorial coverage to resale platforms. Using survival analysis to model trend lifetimes and cosine similarity metrics to compare resale and magazine keyword frequencies, alongside machine learning for price prediction, the study uncovers critical temporal patterns. Results show that resale trends reflect magazine content with a lag of approximately 18 to 30 months and draw from long-wave revivals spanning 6 to 14 …

Toward The Application Of Natural Language Processing In Electronic Health Record Analysis For Taxonomy Development, Latoya Mcdonald

All Dissertations

Electronic health records (EHRs) are pivotal resources for nurse practice because they increase the timeliness and reliability of patient information at the point of care and support access by multiple healthcare providers and the individual patients themselves. However, it is widely recognized that data extraction from EHRs is challenging due to the variability in the language used in clinical care notes and the lack of standardized terminology across healthcare systems. The broad objective of this dissertation is to develop taxonomy-based classification models for nursing care by applying feature engineering approaches to EHRs that include nursing care of ostomy patients following …

From Adversarial Attacks To Robust Classifiers - A Study In Social Media Spam Detection - Black Box & White Box, Jonathan Jose Penaloza Rumie

Undergraduate Theses

Adversarial attacks pose a significant threat to the reliability of machine learning-based spam detection systems in social media. This undergraduate thesis, "From Adversarial Attacks to Robust Classifiers: A Study in Social Media Spam Detection – Black Box & White Box," systematically examines the impact of both black-box and white-box adversarial attacks on a range of spam classifiers, including Logistic Regression, Decision Trees, Random Forests, K-Nearest Neighbors, Bagging, Gradient Boosting, and Support Vector Machines. Leveraging a novel dataset derived from Twitter spam messages and enhanced with adversarial perturbations such as synonym replacement and character-level modifications, this study evaluates classifier performance under …

Leveraging Large Language Models For Knowledge-Free Weak Supervision In Clinical Natural Language Processing, Enshuo Hsu, Kirk Roberts

Faculty, Staff and Student Publications

The performance of deep learning-based natural language processing systems is based on large amounts of labeled training data which, in the clinical domain, are not easily available or affordable. Weak supervision and in-context learning offer partial solutions to this issue, particularly using large language models (LLMs), but their performance still trails traditional supervised methods with moderate amounts of gold-standard data. In particular, inferencing with LLMs is computationally heavy. We propose an approach leveraging fine-tuning LLMs and weak supervision with virtually no domain knowledge that still achieves consistently dominant performance. Using a prompt-based approach, the LLM is used to generate weakly-labeled …

A Comparative Study Of Machine Learning Models For Javanese Wuku Classification: Exploring Svm, Naïve Bayes, And Cnn For Cultural Texts, Danang Arbian Sulistyo, Aji Prasetya Wibawa, Didik Dwi Prasetya, Fadhli Almu'iini Ahda, Agung Bella Putra Utama

Knowledge Engineering and Data Science

This study rigorously evaluates machine learning models for classifying culturally significant Javanese Wuku texts from the “Keagamaan atau Spiritual” category, a domain challenged by unique linguistic nuances and limited digitized resources. We compared Support Vector Machine (SVM), Naïve Bayes, and Convolutional Neural Network (CNN) on texts from five pivotal Wuku types (Sinta, Galungan, Kuningan, Sungsang, Warigalit) sourced from sastra.org, aiming to identify the most effective computational approach. The dataset comprises N = 1419 documents (T = 751.290 tokens), with per-class document counts reported for all five Wuku types. Our evaluation uses accuracy, precision, recall, F1-score, and …

Multitask Learning For Named Entity Recognition And Relationship Extraction, Adrienne D. Hembrick

Theses and Dissertations

Information Extraction (IE) is a fundamental task in Natural Language Processing (NLP), involving the identification of structured information from unstructured text. Two core components of IE—Named Entity Recognition (NER) and Relation Extraction (RE)—are widely used to extract key concepts and the relationships between them across various domains. However, the sequential dependency of RE on the output of NER makes it vulnerable to error propagation: inaccuracies in entity recognition can negatively affect downstream relation extraction.

To mitigate this issue, Multitask Learning (MTL) has been proposed as an approach that jointly models NER and RE, aiming to improve overall performance and reduce …

Enhancing Password Security And Memorability Using Machine Learning And Linguistic Patterns, Jared Wise

LSU New Orleans Theses and Dissertations

In the digital age, text-based passwords remain a primary method for securing online accounts. Yet, users frequently face a dilemma between creating passwords that are easy to remember and sufficiently secure against cyberattacks. This research introduces an approach to password generation that bridges this gap by utilizing linguistic patterns, particularly song lyrics, to develop highly secure and naturally memorable passwords. Using large lyric datasets gained from web scrapes from popular song lyric websites (AZ Lyrics, Genius), features are extracted from a corpus of over 5 million lyrics using sentence structure and natural language processing in a novel way. In using …

Improving Large Language Models For Clinical Named Entity Recognition Via Prompt Engineering, Yan Hu, Qingyu Chen, Jingcheng Du, Xueqing Peng, Vipina Kuttichi Keloth, Xu Zuo, Yujia Zhou, Zehan Li, Xiaoqian Jiang, Zhiyong Lu, Kirk Roberts, Hua Xu

Faculty, Staff and Student Publications

IMPORTANCE: The study highlights the potential of large language models, specifically GPT-3.5 and GPT-4, in processing complex clinical data and extracting meaningful information with minimal training data. By developing and refining prompt-based strategies, we can significantly enhance the models' performance, making them viable tools for clinical NER tasks and possibly reducing the reliance on extensive annotated datasets.

OBJECTIVES: This study quantifies the capabilities of GPT-3.5 and GPT-4 for clinical named entity recognition (NER) tasks and proposes task-specific prompts to improve their performance.

MATERIALS AND METHODS: We evaluated these models on 2 clinical NER tasks: (1) to extract medical problems, treatments, …

Ensemble Pretrained Language Models To Extract Biomedical Knowledge From Literature, Zhao Li, Qiang Wei, Liang-Chin Huang, Jianfu Li, Yan Hu, Yao-Shun Chuang, Jianping He, Avisha Das, Vipina Kuttichi Keloth, Yuntao Yang, Chiamaka S Diala, Kirk E Roberts, Cui Tao, Xiaoqian Jiang, W Jim Zheng, Hua Xu

Faculty, Staff and Student Publications

OBJECTIVES: The rapid expansion of biomedical literature necessitates automated techniques to discern relationships between biomedical concepts from extensive free text. Such techniques facilitate the development of detailed knowledge bases and highlight research deficiencies. The LitCoin Natural Language Processing (NLP) challenge, organized by the National Center for Advancing Translational Science, aims to evaluate such potential and provides a manually annotated corpus for methodology development and benchmarking.

MATERIALS AND METHODS: For the named entity recognition (NER) task, we utilized ensemble learning to merge predictions from three domain-specific models, namely BioBERT, PubMedBERT, and BioM-ELECTRA, devised a rule-driven detection method for cell line and …

Automatic Uncovering Of Patient Primary Concerns In Portal Messages Using A Fusion Framework Of Pretrained Language Modelsautomatic Uncovering Of Patient Primary Concerns In Portal Messages Using A Fusion Framework Of Pretrained Language Models, Yang Ren, Yuqi Wu, Jungwei W Fan, Aditya Khurana, Sunyang Fu, Dezhi Wu, Hongfang Liu, Ming Huang

Faculty, Staff and Student Publications

OBJECTIVES: The surge in patient portal messages (PPMs) with increasing needs and workloads for efficient PPM triage in healthcare settings has spurred the exploration of AI-driven solutions to streamline the healthcare workflow processes, ensuring timely responses to patients to satisfy their healthcare needs. However, there has been less focus on isolating and understanding patient primary concerns in PPMs-a practice which holds the potential to yield more nuanced insights and enhances the quality of healthcare delivery and patient-centered care.

MATERIALS AND METHODS: We propose a fusion framework to leverage pretrained language models (LMs) with different language advantages via a Convolution Neural …

Employing Natural Language Processing To Link Customer Survey Feedback With Net Promoter Scores, Gerardo Moreno

Data Science Undergraduate Honors Theses

This project leverages Natural Language Processing (NLP) to analyze customer feedback from Sam’s Club, aiming to pinpoint key factors influencing Net Promoter Score (NPS). Using sentiment analysis, bigram, and trigram techniques, the project analyses textual data to identify underlying themes and patterns that affect customer satisfaction. These analyses reveal actionable insights into customer preferences and pain points, facilitating a deeper understanding of what drives customer satisfaction in retail environments. By correlating these findings with NPS, this paper details strategies to enhance customer experiences at Sam’s Club, ultimately aiming to improve both satisfaction levels and NPS.

The Importance Of Text Representation For Neural Networks Through Natural Language Processing Techniques, William Parsley

Data Science Undergraduate Honors Theses

Text representation is a fundamental aspect of natural language processing (NLP) when it comes to the performance of neural networks. Free-form text fields are being utilized in more and more industries. Anything from a description of an item on a web store to tracking service events to military-grade aircraft is being collected in free-form text. The goal of the thesis is to highlight best practices and discuss trends in data to prepare text for a neural network. It will demonstrate various techniques for representing free-form text in the context of neural networks, focusing on data preparation decisions, embedding techniques, and …

Predicting 30-Day Unplanned Icu Readmissions Using Deep Learning And Natural Language Processing Techniques: A Mimic Iv Data Analysis, David Licerio

Computational and Data Sciences (MS) Theses

We design and implement a multi-stage modeling approach focused on predicting unplanned 30-day all- cause intensive care unit (ICU) hospital readmissions using the Medical Information Mart for Intensive Care (MIMIC IV) dataset. Structured data consisting of demographic information, comorbidities, lab results, and vital signs are combined with features extracted from medical text data consisting of patients’ diagnoses, procedures, and discharge notes and further engineered using several methods, including Latent Dirichlet Allocation (LDA), Latent Semantic Analysis (LSA), and word embeddings.

We sequentially implement three distinct Dense Neural Networks (DNNs) combined with the LightGBM gradient-boosting framework. Our model attained a 5-fold cross-validated …

Representation Learning For Generative Models With Applications To Healthcare, Astronautics, And Aviation, Van Minh Nguyen

Theses and Dissertations

This dissertation explores applications of representation learning and generative models to challenges in healthcare, astronautics, and aviation.

The first part investigates the use of Generative Adversarial Networks (GANs) to synthesize realistic electronic health record (EHR) data. An initial attempt at training a GAN on the MIMIC-IV dataset encountered stability and convergence issues, motivating a deeper study of 1-Lipschitz regularization techniques for Auxiliary Classifier GANs (AC-GANs). An extensive ablation study on the CIFAR-10 dataset found that Spectral Normalization is key for AC-GAN stability and performance, while Weight Clipping fails to converge without Spectral Normalization. Analysis of the training dynamics provided further …

A Holistic And Collaborative Behavioral Health Detection Framework Using Sensitive Police Narratives, Martin Keagan Wynne Brown

Dissertations

Identifying behavioral health is paramount for law enforcement officers to provide appropriate follow-up community care. In the current practice, law enforcement offices manually identify these behavioral health cases to allow the designation of the relevant follow-up resources. Police reports generated by officers' response to 911 calls remain an untapped resource for identifying such incidents. Therefore, we advocate for the incorporation of manual annotations from experts, natural language processing (NLP), active learning, advanced machine learning, and ensemble techniques to detect behavioral health cases within police reports. In this dissertation, we develop tools and frameworks to automatically detect behavioral health cases from …

Spec: A Soft Prompt-Based Calibration On Performance Variability Of Large Language Model In Clinical Notes Summarization, Yu-Neng Chuang, Ruixiang Tang, Xiaoqian Jiang, Xia Hu

Faculty, Staff and Student Publications

Electronic health records (EHRs) store an extensive array of patient information, encompassing medical histories, diagnoses, treatments, and test outcomes. These records are crucial for enabling healthcare providers to make well-informed decisions regarding patient care. Summarizing clinical notes further assists healthcare professionals in pinpointing potential health risks and making better-informed decisions. This process contributes to reducing errors and enhancing patient outcomes by ensuring providers have access to the most pertinent and current patient data. Recent research has shown that incorporating instruction prompts with large language models (LLMs) substantially boosts the efficacy of summarization tasks. However, we show that this approach also …

Language Models For Rare Disease Information Extraction: Empirical Insights And Model Comparisons, Shashank Gupta

Theses and Dissertations--Computer Science

End-to-end relation extraction (E2ERE) is a crucial task in natural language processing (NLP) that involves identifying and classifying semantic relationships between entities in text. This thesis compares three paradigms for end-to-end relation extraction (E2ERE) in biomedicine, focusing on rare diseases with discontinuous and nested entities. We evaluate Named Entity Recognition (NER) to Relation Extraction (RE) pipelines, sequence-to-sequence models, and generative pre-trained transformer (GPT) models using the RareDis information extraction dataset. Our findings indicate that pipeline models are the most effective, followed closely by sequence-to-sequence models. GPT models, despite having eight times as many parameters, perform worse than sequence-to-sequence models and …

Sticky Charters? The Surprisingly Tepid Embrace Of Officer-Protecting Waivers In Delaware, Jens Frankenreiter, Eric L. Talley

Scholarship@WashULaw

This article investigates the reaction to a much-heralded 2022 legal reform in Delaware that permitted a corporation’s charter to exculpate its officers from monetary exposure for breaching their fiduciary duty of care. To isolate reactions to this statutory reform, we make extensive use of generative AI tools to identify and interpret charter amendments that introduce officer-facing waivers. We find a surprisingly tepid rate of uptake among Delaware corporations through the end of the first post-reform year, notwithstanding widespread predictions that corporate entities would quickly storm the exculpation exits once permitted to do so.

Our study makes two contributions to the …

An Open Natural Language Processing (Nlp) Framework For Ehr-Based Clinical Research: A Case Demonstration Using The National Covid Cohort Collaborative (N3c), Sijia Liu, Andrew Wen, Liwei Wang, Huan He, Sunyang Fu, Robert Miller, Andrew Williams, Daniel Harris, Ramakanth Kavuluru, Mei Liu, Noor Abu-El-Rub, Dalton Schutte, Rui Zhang, Masoud Rouhizadeh, John D Osborne, Yongqun He, Umit Topaloglu, Stephanie S Hong, Joel H Saltz, Thomas Schaffter, Emily Pfaff, Christopher G Chute, Tim Duong, Melissa A Haendel, Rafael Fuentes, Peter Szolovits, Hua Xu, Hongfang Liu

Faculty, Staff and Student Publications

Despite recent methodology advancements in clinical natural language processing (NLP), the adoption of clinical NLP models within the translational research community remains hindered by process heterogeneity and human factor variations. Concurrently, these factors also dramatically increase the difficulty in developing NLP models in multi-site settings, which is necessary for algorithm robustness and generalizability. Here, we reported on our experience developing an NLP solution for Coronavirus Disease 2019 (COVID-19) signs and symptom extraction in an open NLP framework from a subset of sites participating in the National COVID Cohort (N3C). We then empirically highlight the benefits of multi-site data for both …

Improving Semantic Document Classification Accuracy By Integrating Human-Crafted Knowledge, Zachary Weinfeld, Lubomir Stanchev

College of Engineering Summer Undergraduate Research Program

Document classification is a pivotal task in various domains, warranting the development of robust algorithms. Among these, the Bidirectional Encoder Representations from Transformers (BERT) algorithm, introduced by Google, has proven to perform well when fine-tuned for the task at hand. Leveraging transformer architecture, BERT demonstrates stellar language understanding capabilities. However, the integration of BERT with a range of techniques has shown potential for further enhancing classification accuracy. This work investigates several techniques that leverage semantic understanding to improve the performance of document classification models trained with BERT. Specifically, we explore three methods. First, we will balance corpuses afflicted by imbalanced …

A Method For Generating A Non-Manual Feature Model For Sign Language Processing, Robert G. Smith Dr, Markus Hofmann Dr

Articles

While recent approaches to sign language processing have shifted to the domain of Machine Learning (ML), the treatment of Non-Manual Features (NMFs) remains an open question. The principal challenge facing this method is the comparatively small sign language corpora available for training machine learning models. This study produces a statistical model which may be used in future ML, rules-based, and hybrid-learning approaches for sign language processing tasks. In doing so, this research explores the emerging patterns of non-manual articulation concerning grammatical classes in Irish Sign Language (ISL). The experimental method applied here is a novel implementation of an association rules …

Exploiting Association Rules Mining To Inform The Use Of Non-Manual Features In Sign Language Processing, Robert G. Smith

Other Resources

In recent years, the use of virtual assistants and voice user interfaces has become a latent part of modern living. Unseen to the user are the various artificial intelligence and natural language processing technologies, the vast datasets, and the linguistic insights that underpin such tools. The technologies supporting them have chiefly targeted widely used spoken languages, leaving sign language users at a disadvantage. One important reason why sign languages are unsupported by such tools is a requirement of the underpinning technologies for a comprehensive description of the language. Sign language processing technologies endeavour to bridge this technology inequality.

Recent approaches …

Ai Approaches To Understand Human Deceptions, Perceptions, And Perspectives In Social Media, Chih-Yuan Li

Dissertations

Social media platforms have created virtual space for sharing user generated information, connecting, and interacting among users. However, there are research and societal challenges: 1) The users are generating and sharing the disinformation 2) It is difficult to understand citizens' perceptions or opinions expressed on wide variety of topics; and 3) There are overloaded information and echo chamber problems without overall understanding of the different perspectives taken by different people or groups.

This dissertation addresses these three research challenges with advanced AI and Machine Learning approaches. To address the fake news, as deceptions on the facts, this dissertation presents Machine …

Quehry: A Question Answering System To Query Electronic Health Records, Sarvesh Soni, Surabhi Datta, Kirk Roberts

Faculty, Staff and Student Publications

OBJECTIVE: We propose a system, quEHRy, to retrieve precise, interpretable answers to natural language questions from structured data in electronic health records (EHRs).

MATERIALS AND METHODS: We develop/synthesize the main components of quEHRy: concept normalization (MetaMap), time frame classification (new), semantic parsing (existing), visualization with question understanding (new), and query module for FHIR mapping/processing (new). We evaluate quEHRy on 2 clinical question answering (QA) datasets. We evaluate each component separately as well as holistically to gain deeper insights. We also conduct a thorough error analysis for a crucial subcomponent, medical concept normalization.

RESULTS: Using gold concepts, the precision of quEHRy …

Toward A Neural Semantic Parsing System For Ehr Question Answering, Sarvesh Soni, Kirk Roberts

Faculty, Staff and Student Publications

Clinical semantic parsing (SP) is an important step toward identifying the exact information need (as a machine-understandable logical form) from a natural language query aimed at retrieving information from electronic health records (EHRs). Current approaches to clinical SP are largely based on traditional machine learning and require hand-building a lexicon. The recent advancements in neural SP show a promise for building a robust and flexible semantic parser without much human effort. Thus, in this paper, we aim to systematically assess the performance of two such neural SP models for EHR question answering (QA). We found that the performance of these …

Quantification Of Various Types Of Biases In Large Language Models, Sudhashree Sayenju

Doctor of Data Science and Analytics Dissertations

Natural Language Processing (NLP) systems are included everywhere on the internet from search engines, language translations to more advanced systems like voice assistant and customer service. Since humans are always on the receiving end of NLP technologies, it is very important to analyze whether or not the Large Language Models (LLMs) in use have bias and are therefore unfair. The majority of the research in NLP bias has focused on societal stereotype biases embedded in LLMs. However, our research focuses on all types of biases, namely model class level bias, stereotype bias and domain bias present in LLMs. Model class …

Using Machine Learning To Measure Political Polarization On Social Media, Veronica Cagle

Departmental Honors & Graduate Capstone Projects

Polarization in the political sphere, seen through combative communication and stalemate, may impose negative social impacts on the population. Attempting to measure political polarization in the masses through self-reported surveys and interviews can present response biases of social desirability. The classification of thought freely written online allows political polarization to be measured in an impartial manner. Reddit is one application that enables users to share opinions and create discussions anonymously; this text can be used to measure the political climate at any given time. Disagreement has grown over the perceived level of polarization in our society. The purpose of my …