Computer Sciences Commons^™

Chinese Idiom Understanding With Transformer-Based Pretrained Language Models, Minghuan Tan

Dissertations and Theses Collection (Open Access)

In this dissertation, I study the understanding of Chinese idioms using transformer-based pretrained language models. By ``understanding", I confine the topics to word embeddings learning, contextualized word representations learning, multiple-choice cloze-test reading comprehension and conditional text generation. Chinese idioms are fixed phrases that have special meanings usually derived from an ancient story. The meanings of these idioms are oftentimes not directly related to their component characters, which makes it hard to model them compared with standard phrases whose meanings are compositional. We initiate the work with studying idiom representations derived from pretrained language models, in particular, BERT. We adopt probing-based …

Toward Suicidal Ideation Detection With Lexical Network Features And Machine Learning, Ulya Bayram, William Lee, Daniel Santel, Ali Minai, Peggy Clark, Tracy Glauser, John Pestian

Northeast Journal of Complex Systems (NEJCS)

In this study, we introduce a new network feature for detecting suicidal ideation from clinical texts and conduct various additional experiments to enrich the state of knowledge. We evaluate statistical features with and without stopwords, use lexical networks for feature extraction and classification, and compare the results with standard machine learning methods using a logistic classifier, a neural network, and a deep learning method. We utilize three text collections. The first two contain transcriptions of interviews conducted by experts with suicidal (n=161 patients that experienced severe ideation) and control subjects (n=153). The third collection consists of interviews conducted by experts …

Symbolic Semantic Memory In Transformer Language Models, Robert Kenneth Morain

Theses and Dissertations

This paper demonstrates how transformer language models can be improved by giving them access to relevant structured data extracted from a knowledge base. The knowledge base preparation process and modifications to transformer models are explained. We evaluate these methods on language modeling and question answering tasks. These results show that even simple additional knowledge augmentation leads to a reduction in validation loss by 73%. These methods also significantly outperform common ways of improving language models such as increasing the model size or adding more data.

Improving Anonymized Search Relevance With Natural Language Processing And Machine Learning, Niko A. Petrocelli

Theses and Dissertations

Users often sacrifice personal data for more relevant search results, presenting a problem to communities that desire both search anonymity and relevant results. To balance these priorities, this research examines the impact of using Siamese networks to extend word embeddings into document embeddings and detect similarities between documents. The predicted similarity can locally re-rank search results provided from various sources. This technique is leveraged to limit the amount of information collected from a user by a search engine. A prototype is produced by applying the methodology in a real-world search environment. The prototype yielded an additional function of finding new …

Aspect-Based Api Review Classification: How Far Can Pre-Trained Transformer Model Go?, Chengran Yang, Bowen Xu, Junaed Younus Khan, Gias Uddin, Donggyun Han, Zhou Yang, David Lo

Research Collection School Of Computing and Information Systems

APIs (Application Programming Interfaces) are reusable software libraries and are building blocks for modern rapid software development. Previous research shows that programmers frequently share and search for reviews of APIs on the mainstream software question and answer (Q&A) platforms like Stack Overflow, which motivates researchers to design tasks and approaches related to process API reviews automatically. Among these tasks, classifying API reviews into different aspects (e.g., performance or security), which is called the aspect-based API review classification, is of great importance. The current state-of-the-art (SOTA) solution to this task is based on the traditional machine learning algorithm. Inspired by the …

Event-Related Microblog Retrieval In Turkish, Çağri Toraman

Turkish Journal of Electrical Engineering and Computer Sciences

Microblogs, such as tweets, are short messages in which users are able to share any opinion and information. Microblogs are mostly related to real-life events reported in news articles. Finding event-related microblogs is important to analyze online social networks and understand public opinion on events. However, finding such microblogs is a challenging task due to the dynamic nature of microblogs and their limited length. In this study, assuming that news articles are given as queries and microblogs as documents, we find event-related microblogs in Turkish. In order to represent news articles and microblogs, we examine encoding methods, namely traditional bag-of-words …

Nlp@Vcu: Crop Characteristic Extraction Framework, Cora Lewis, Bridget Mcinnes, Getiria Onsongo

Summer REU Program

We developed a crop characteristic extraction framework. Starting from a custom SpaCy named entity recognition model, we added pre-trained word embeddings and a part-of-speech based entity expansion post-processing step. Then, we implemented an evaluation framework that functioned as a 5-fold cross validation wrapper for SpaCy custom training. Preliminary results showed improvement in the extraction framework after these additions.

Humanizing Computational Literature Analysis Through Art-Based Visualizations, Alexandria Leto

Electronic Theses and Dissertations

Inequalities in gender representation and characterization in fictional works are issues that have long been discussed by social scientists. This work addresses these inequalities with two interrelated components. First, it contributes a sentiment and word frequency analysis task focused on gender-specific nouns and pronouns in 15,000 fictional works taken from the online library, Project Gutenberg. This analysis allows for both quantifying and offering further insight on the nature of this disparity in gender representation. Then, the outcomes of the analysis are harnessed to explore novel data visualization formats using computational and studio art techniques. Our results call attention to the …

Computer Enabled Interventions To Communication And Behavioral Problems In Collaborative Work Environments, Ashutosh Shivakumar

Browse all Theses and Dissertations

Task success in co-located and distributed collaborative work settings is characterized by clear and efficient communication between participating members. Communication issues like 1) Unwanted interruptions and 2) Delayed feedback in collaborative work based distributed scenarios have the potential to impede task coordination and significantly decrease the probability of accomplishing task objective. Research shows that 1) Interrupting tasks at random moments can cause users to take up to 30% longer to resume tasks, commit up to twice the errors, and experience up to twice the negative effect than when interrupted at boundaries 2) Skill retention in collaborative learning tasks improves with …

Machine Learning In Requirements Elicitation: A Literature Review, Cheligeer Cheligeer, Jingwei Huang, Guosong Wu, Nadia Bhuiyan, Yuan Xu, Yong Zeng

Engineering Management & Systems Engineering Faculty Publications

A growing trend in requirements elicitation is the use of machine learning (ML) techniques to automate the cumbersome requirement handling process. This literature review summarizes and analyzes studies that incorporate ML and natural language processing (NLP) into demand elicitation. We answer the following research questions: (1) What requirement elicitation activities are supported by ML? (2) What data sources are used to build ML-based requirement solutions? (3) What technologies, algorithms, and tools are used to build ML-based requirement elicitation? (4) How to construct an ML-based requirements elicitation method? (5) What are the available tools to support ML-based requirements elicitation methodology? Keywords …

Evaluating Similarity Of Cross-Architecture Basic Blocks, Elijah L. Meyer

Browse all Theses and Dissertations

Vulnerabilities in source code can be compiled for multiple processor architectures and make their way into several different devices. Security researchers frequently have no way to obtain this source code to analyze for vulnerabilities. Therefore, the ability to effectively analyze binary code is essential. Similarity detection is one facet of binary code analysis. Because source code can be compiled for different architectures, the need can arise for detecting code similarity across architectures. This need is especially apparent when analyzing firmware from embedded computing environments such as Internet of Things devices, where the processor architecture is dependent on the product and …

Exploring Cyberterrorism, Topic Models And Social Networks Of Jihadists Dark Web Forums: A Computational Social Science Approach, Vivian Fiona Guetler

Graduate Theses, Dissertations, and Problem Reports (ETD)

This three-article dissertation focuses on cyber-related topics on terrorist groups, specifically Jihadists’ use of technology, the application of natural language processing, and social networks in analyzing text data derived from terrorists' Dark Web forums. The first article explores cybercrime and cyberterrorism. As technology progresses, it facilitates new forms of behavior, including tech-related crimes known as cybercrime and cyberterrorism. In this article, I provide an analysis of the problems of cybercrime and cyberterrorism within the field of criminology by reviewing existing literature focusing on (a) the issues in defining terrorism, cybercrime, and cyberterrorism, (b) ways that cybercriminals commit a crime in …

Using Data Science Tools For Investigating Chat Logs From The Conti Ransomware Group, Boyan Kostadinov, Joseph Liu, Julio Rayme

Publications and Research

The main goal of this paper is to showcase some results from a comprehensive data analysis that we did on the cache of chat logs from the notorious ransomware group Conti. The chat logs were made publicly available on February 27, 2022. They were translated from Russian into English, and contain 393 json files with chat logs from the instant messaging service Jabber. We employ a variety of modern data science tools for text mining, natural language processing, network analysis and geospatial analysis to investigate the Conti chat logs so that we can understand the command and control structure of …

Representation Learning On Heterogeneous Spatiotemporal Networks, Dakshak Keerthi Chandra

Doctoral Dissertations

“The problem of learning latent representations of heterogeneous networks with spatial and temporal attributes has been gaining traction in recent years, given its myriad of real-world applications. Most systems with applications in the field of transportation, urban economics, medical information, online e-commerce, etc., handle big data that can be structured into Spatiotemporal Heterogeneous Networks (SHNs), thereby making efficient analysis of these networks extremely vital. In recent years, representation learning models have proven to be quite efficient in capturing effective lower-dimensional representations of data. But, capturing efficient representations of SHNs continues to pose a challenge for the following reasons: (i) Spatiotemporal …

Leveraging Natural Language Processing To Analyse The Temporal Behavior Of Extremists On Social Media, May El Barachi, Sujith Samuel Mathew, Farhad Oroumchian, Imene Ajala, Saad Lutfi, Rand Yasin

All Works

Aiming at achieving sustainability and quality of life for citizens, future smart cities adopt a data-centric approach to decision making in which assets, people, and events are constantly monitored to inform decisions. Public opinion monitoring is of particular importance to governments and intelligence agencies, who seek to monitor extreme views and attempts of radicalizing individuals in society. While social media platforms provide increased visibility and a platform to express public views freely, such platforms can also be used to manipulate public opinion, spread hate speech, and radicalize others. Natural language processing and data mining techniques have gained popularity for the …

Temporal Disambiguation Of Relative Temporal Expressions In Clinical Texts Using Temporally Fine-Tuned Contextual Word Embeddings., Amy L. Olex

Theses and Dissertations

Temporal reasoning is the ability to extract and assimilate temporal information to reconstruct a series of events such that they can be reasoned over to answer questions involving time. Temporal reasoning in the clinical domain is challenging due to specialized medical terms and nomenclature, shorthand notation, fragmented text, a variety of writing styles used by different medical units, redundancy of information that has to be reconciled, and an increased number of temporal references as compared to general domain texts. Work in the area of clinical temporal reasoning has progressed, but the current state-of-the-art still has a ways to go before …

An Empirical Study Of Developers' Discussions About Security Challenges Of Different Programming Languages, Roland Croft, Yongzheng Xie, Mansooreh Zahedi, Muhammad Ali Babar, Christoph Treude

Research Collection School Of Computing and Information Systems

In collaborative software development projects, work items are used as a mechanism to coordinate tasks and track shared development work. In this paper, we explore how “tagging,” a lightweight social computing mechanism, is used to communicate matters of concern in the management of development tasks. We present the results from two empirical studies over 36 and 12 months, respectively, on how tagging has been adopted and what role it plays in the development processes of several professional development projects with more than 1,000 developers in total. Our research shows that the tagging mechanism was eagerly adopted by the teams, and …

Private And Federated Deep Learning: System, Theory, And Applications For Social Good, Han Hu

Dissertations

During the past decade, drug abuse continues to accelerate towards becoming the most severe public health problem in the United States. The ability to detect drug­abuse risk behavior at a population scale, such as among the population of Twitter users, can help to monitor the trend of drug­abuse incidents. However, traditional methods do not effectively detect drug­abuse risk behavior in tweets, mainly due to the sparsity of such tweets and the noisy nature of tweets. In the first part of this dissertation work, the task of classifying tweets as containing drug­abuse risk behavior or not, is studied. Millions of public …

Semantically Meaningful Sentence Embeddings, Rojina Deuja

School of Computing: Dissertations, Theses, and Student Research

Text embedding is an approach used in Natural Language Processing (NLP) to represent words, phrases, sentences, and documents. It is the process of obtaining numeric representations of text to feed into machine learning models as vectors (arrays of numbers). One of the biggest challenges in text embedding is representing longer text segments like sentences. These representations should capture the meaning of the segment and the semantic relationship between its constituents. Such representations are known as semantically meaningful embeddings. In this thesis, we seek to improve upon the quality of sentence embeddings that capture semantic information.

The current state-of-the-art models are …

Language Pre-Training And Auxiliary Tasks For Vision And Language Navigation, Saumya Bhatt

Computer Science and Engineering Theses - Archive

The Vision and Language Navigation task came to life from the idea that we can build a robot or an autonomous system that can be instructed in human language and that will navigate using the instructions given. For example, we tell the agent to “Go down past some room dividers toward a glass top desk and turn into the dining area. Wait next to the large glass dining table” and not only does it reach the goal state but it follows the instructions while navigating. With the current developments, this may not seem like a distant problem anymore and in …

From Language Comprehension Towards General Ai, Binay Dahal

UNLV Theses, Dissertations, Professional Papers, and Capstones

Language comprehension or more formally, natural language understanding is one of the major undertakings in Artificial Intelligence. In this work, we explore a few of the problems in language understanding using fixed deep learning models. Specifically, first, we look into question generation. Asking questions relates to the cognitive ability of language comprehension and context understanding. For that reason, making progress in question generation is significant. We introduce a novel task called “question generation with masked target answer” and propose various models and present the baseline result for the task. Next, we extend on the question generation task and develop a …

A Bert-Based Two-Stage Model For Chinese Chengyu Recommendation, Minghuan Tan, Jing Jiang, Bingtian Dai

Research Collection School Of Computing and Information Systems

In Chinese, Chengyu are fixed phrases consisting of four characters. As a type of idioms, their meanings usually cannot be derived from their component characters. In this paper, we study the task of recommending a Chengyu given a textual context. Observing some of the limitations with existing work, we propose a two-stage model, where during the first stage we re-train a Chinese BERT model by masking out Chengyu from a large Chinese corpus with a wide coverage of Chengyu. During the second stage, we fine-tune the retrained, Chengyu-oriented BERT on a specific Chengyu recommendation dataset. We evaluate this method on …

Modeling And Analyzing Users' Privacy Disclosure Behavior To Generate Personalized Privacy Policies, A.K.M. Nuhil Mehdy

Boise State University Theses and Dissertations

Privacy and its importance to society have been studied for centuries. While our understanding and continued theory building to hypothesize how users make privacy disclosure decisions has increased over time, the struggle to find a one-size solution that satisfies the requirements of each individual remains unsolved. Depending on culture, gender, age, and other situational factors, the concept of privacy and users' expectations of how their privacy should be protected varies from person to person. The goal of this dissertation is to design and develop tools and algorithms to support personal privacy management for end-users. The foundation of this research is …

Identifying Optimal Course Structures Using Topic Models, Tehut Tesfaye Biru

Dartmouth College Undergraduate Theses

This research project investigates whether there exists an optimal way to structure topics in educational course content that results in higher levels of engagement among students. It is implemented by fitting topic models to transcripts of educational videos contained in the Khan Academy platform. The fitted models were used to extract topic trajectories across time for each video and subsequently clustered based on whether they have similar “shapes”. The differences in mean engagement metrics per cluster suggest that some course shapes are more palatable to students regardless of subject matter. Additionally, the topic trajectories suggest a constant progression of topics …

Nomophobia Before And After The Covid-19 Pandemic-Can Social Media Be Used To Understand Mobile Phone Dependency, Vaishnavi Visweswaraiah, Tanvi Banerjee, William Romine, Sarah Fryman

Computer Science and Engineering Faculty Publications

No abstract provided.

Learning Intermediate Representations For Question Answering Systems, Zakery T. Clarke

Computer Science ETDs

Question answering systems are models that can perform natural language processing (NLP) on a question, retrieve an answer from a datasource, and communicate it to a user. In question answering systems, it is important for the system to learn an underlying representation for a piece of text. There are many systems that have achieved incredible accuracy on question answering datasets such as the Stanford Question and Answer Dataset (SQuAD), but these systems often encode their knowledge in a manner that is impossible to verify. Many current models would benefit more from verifiability, than marginal accuracy improvements.

We propose a method …

Fine-Grained Sentiment Analysis For Customer Review, Bing Han, Meng Han, Jing (Selena) He

Master of Science in Computer Science Theses

Natural Language Processing (NLP) is one of the most attractive technologies in many applications in real-life. Sentiment analysis, which has devoted to know others' think or feel about an experience or an item and hence take an action, is one of the most developed area in both academia and industry. Among sentiment analysis, fine-grained aspect sentiment analysis attempts to analyze emotional attitude categorized into different aspects or features of an(a) experience/service/product. Although aspect level sentiment analysis could provide more useful information, the proposed models' performance were relative poor compared with document-level or sentence-level sentiment analysis due to the lack of …

A Comparison Of Word Embedding Techniques For Similarity Analysis, Tyler Gerth

Computer Science and Computer Engineering Undergraduate Honors Theses

There have been a multitude of word embedding techniques developed that allow a computer to process natural language and compare the relationships between different words programmatically. In this paper, similarity analysis, or the testing of words for synonymic relations, is used to compare several of these techniques to see which performs the best. The techniques being compared all utilize the method of creating word vectors, reducing words down into a single vector of numerical values that denote how the word relates to other words that appear around it. In order to get a holistic comparison, multiple analyses were made, with …

Bibliometric Analysis Of Named Entity Recognition For Chemoinformatics And Biomedical Information Extraction Of Ovarian Cancer, Vijayshri Khedkar, Charlotte Fernandes, Devshi Desai, Mansi R, Gurunath Chavan Dr, Sonali Tidke Dr., M. Karthikeyan Dr.

Library Philosophy and Practice (e-journal)

With the massive amount of data that has been generated in the form of unstructured text documents, Biomedical Named Entity Recognition (BioNER) is becoming increasingly important in the field of biomedical research. Since currently there does not exist any automatic archiving of the obtained results, a lot of this information remains hidden in the textual details and is not easily accessible for further analysis. Hence, text mining methods and natural language processing techniques are used for the extraction of information from such publications.Named entity recognition, is a subtask that comes under information extraction that focuses on finding and categorizing specific …

Deep Learning Based Models For Classification From Natural Language Processing To Computer Vision, Xianshan Qu

Theses and Dissertations

With the availability of large scale data sets, researchers in many different areas such as natural language processing, computer vision, recommender systems have started making use of deep learning models and have achieved great progress in recent years. In this dissertation, we study three important classification problems based on deep learning models.

First, with the fast growth of e-commerce, more people choose to purchase products online and browse reviews before making decisions. It is essential to build a model to identify helpful reviews automatically. Our work is inspired by the observation that a customer's expectation of a review can be …