Digital Commons Network^™

Stock Forecasts With Lstm And Web Sentiment, Michael Burgess, Faizan Javed, Nnenna Okpara, Chance Robinson

SMU Data Science Review

Traditional time-series techniques, such as auto-regressive and moving average models, can have difficulties when applied to stock data due to the randomness inherent to the markets. In this study, Long Short-Term Memory Recurrent Neural Networks, or LSTMs, have been applied to pricing data along with sentiment scores derived from web sources such as Twitter and other financial media outlets. The project team utilized this approach to complement the technical indicators observed at the end of each trading day for three stocks from the NASDAQ stock exchange over a 12-year span. A common benchmark to assess model performance on time series …

The Evolution Of Quail Research: Trends In Themes, Cognitive Extent, And Lexical Diversity, Fidel Hernández

National Quail Symposium Proceedings

Language is an interesting characteristic that is unique to humans. Language represents a method of human communication and is believed to reflect a person’s view of reality (Kramsch 2004). The words used by a person or a community provide insight into the ideas, concepts, and worldview held by people (Pennebaker et al. 2003). In science, publications represent a primary form of communication of ideas among scientists. Publications provide a historical record of a discipline and reflect the relative interest of a scientific community in particular concepts during a given era (Kim et al. 2018). Consequently, how word use in publications …

Led Down The Rabbit Hole: Exploring The Potential Of Global Attention For Biomedical Multi-Document Summarisation, Yulia Otmakhova, Hung Thinh Truong, Timothy Baldwin, Trevor Cohn, Karin Verspoor, Jey Han Lau

Natural Language Processing Faculty Publications

In this paper we report on our submission to the Multidocument Summarisation for Literature Review (MSLR) shared task. Specifically, we adapt PRIMERA (Xiao et al., 2022) to the biomedical domain by placing global attention on important biomedical entities in several ways. We analyse the outputs of the 23 resulting models, and report patterns in the results related to the presence of additional global attention, number of training steps, and the input configuration. © 2022, CC BY-SA.

Unsupervised Lexical Substitution With Decontextualised Embeddings, Takashi Wada, Timothy Baldwin, Yuji Matsumoto, Jey Han Lau

Natural Language Processing Faculty Publications

We propose a new unsupervised method for lexical substitution using pre-trained language models. Compared to previous approaches that use the generative capability of language models to predict substitutes, our method retrieves substitutes based on the similarity of contextualised and decontextualised word embeddings, i.e. the average contextual representation of a word in multiple contexts. We conduct experiments in English and Italian, and show that our method substantially outperforms strong baselines and establishes a new state-of-the-art without any explicit supervision or fine-tuning. We further show that our method performs particularly well at predicting low-frequency substitutes, and also generates a diverse list of …

Beat Transformer: Demixed Beat And Downbeat Tracking With Dilated Self-Attention, Jingwei Zhao, Gus Xia, Ye Wang

Machine Learning Faculty Publications

We propose Beat Transformer, a novel Transformer encoder architecture for joint beat and downbeat tracking. Different from previous models that track beats solely based on the spectrogram of an audio mixture, our model deals with demixed spectrograms with multiple instrument channels. This is inspired by the fact that humans perceive metrical structures from richer musical contexts, such as chord progression and instrumentation. To this end, we develop a Transformer model with both time-wise attention and instrument-wise attention to capture deep-buried metrical cues. Moreover, our model adopts a novel dilated self-attention mechanism, which achieves powerful hierarchical modelling with only linear complexity. …

Domain Adversarial Training On Conditional Variational Auto-Encoder For Controllable Music Generation, Jingwei Zhao, Gus Xia, Ye Wang

Machine Learning Faculty Publications

The variational auto-encoder has become a leading framework for symbolic music generation, and a popular research direction is to study how to effectively control the generation process. A straightforward way is to control a model using different conditions during inference. However, in music practice, conditions are usually sequential (rather than simple categorical labels), involving rich information that overlaps with the learned representation. Consequently, the decoder gets confused about whether to “listen to” the latent representation or the condition, and sometimes just ignores the condition. To solve this problem, we leverage domain adversarial training to disentangle the representation from condition cues …

Learning Experience With Learnwithemma, Clara Susaie, Choo-Kim Tan, Pey-Yun Goh

Journal of Informatics and Web Engineering

The presence of Covid-19 was a game-changer in all the sectors that traditional learning, working, selling even living methods have changed from basic methods to something else to curb Covid-19. The impact was huge on most sectors due to the lack of experience in overcoming pandemics. One of the sectors that face the most struggle was educational institutions. For many years, face-to-face learning and teaching method have been in practice. While during Covid-19, everyone was forced to attend online classes as a precautious measure. Online learning is completely based on digital study without the physical presence of students or lecturers. …

Leveraging Natural Learning Processing To Uncover Themes In Clinical Notes Of Patients Admitted For Heart Failure, Ankita Agarwal, Krishnaprasad Thirunarayan, William Romine, Amanuel Alambo, Mia Cajita, Tanvi Banerjee

Computer Science and Engineering Faculty Publications

Heart failure occurs when the heart is not able to pump blood and oxygen to support other organs in the body as it should. Treatments include medications and sometimes hospitalization. Patients with heart failure can have both cardiovascular as well as non-cardiovascular comorbidities. Clinical notes of patients with heart failure can be analyzed to gain insight into the topics discussed in these notes and the major comorbidities in these patients. In this regard, we apply machine learning techniques, such as topic modeling, to identify the major themes found in the clinical notes specific to the procedures performed on 1,200 patients …

Decipherment Challenges Due To Tamga And Letter Mix-Ups In An Old Hungarian Runic Inscription From The Altai Mountains, Peter Revesz

School of Computing: Faculty Publications

An Old Hungarian Runic inscription from the Altai Mountains with 40 signs has posed some special challenges for decipherment due to several letter mix-ups and the use of a tamga sign, which is the first reported use of a tamga within this type of script. This paper gives a complete and correct translation and draws some lessons that can be learned about decipherment. It introduces sign similarity matrices as a method of detecting accidental misspellings and shows that sign similarity matrices can be efficiently computed. It also explains the importance of simultaneously achieving the three criteria for a valid decipherment: …

Overview Of The Clef-2022 Checkthat! Lab Task 2 On Detecting Previously Fact-Checked Claims, Preslav Nakov, Giovanni Da San Martino, Firoj Alam, Shaden Shaar, Hamdy Mubarak, Nikolay Babulkov

Natural Language Processing Faculty Publications

We describe the fourth edition of the CheckThat! Lab, part of the 2022 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting three tasks related to factuality, and it covers seven languages such as Arabic, Bulgarian, Dutch, English, German, Spanish, and Turkish. Here, we present the task 2, which asks to detect previously fact-checked claims (in two languages). A total of six teams participated in this task, submitted a total of 37 runs, and most submissions managed to achieve sizable improvements over the baselines using transformer based models such as BERT, RoBERTa. In this paper, we …

Overview Of The Clef-2022 Checkthat! Lab Task 1 On Identifying Relevant Claims In Tweets, Preslav Nakov, Alberto Barrón-Cedeño, Giovanni Da San Martino, Firoj Alam, Mucahid Kutlu, Wajdi Zaghouani, Mucahid Kutlu, Wajdi Zaghouani, Chengkai Li, Shaden Shaar, Hamdy Mubarak, Alex Nikolov

Natural Language Processing Faculty Publications

We present an overview of CheckThat! lab 2022 Task 1, part of the 2022 Conference and Labs of the Evaluation Forum (CLEF). Task 1 asked to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics in six languages: Arabic, Bulgarian, Dutch, English, Spanish, and Turkish. A total of 19 teams participated and most submissions managed to achieve sizable improvements over the baselines using Transformer-based models such as BERT and GPT-3. Across the four subtasks, approaches that targetted multiple languages (be it individually or in conjunction, in general obtained the best performance. We describe the …

Environmental Justice Mapping In The U.S. Pacific Island Territory Of Guam, Ben Rocha

Master of Science in Environmental Sciences and Management Projects

Academics, regulators, and the public currently use geospatial analysis tools to identify locations that may be candidates for further environmental justice review in the continental United States (U.S.). However, current environmental justice geospatial analysis tools overlook a small but significant portion of the U.S. - the U.S. Pacific Islands. This study analyzes environmental justice within the U.S. Territory of Guam using existing geospatial analysis methods and publicly available environmental, climate, and socio-economic data to: (1) Spatially map relevant demographic and environmental data and (2) determine the correlation, if any, between the exposure to environmental hazards and the socio-economic status of …

Environmental Justice Mapping In The U.S. Pacific Island Territory Of Guam, Sina Schwenk-Mueller

Master of Science in Environmental Sciences and Management Projects

Academics, regulators, and the public currently use geospatial analysis tools to identify locations that may be candidates for further environmental justice review in the continental United States (U.S.). However, current environmental justice geospatial analysis tools overlook a small but significant portion of the U.S. - the U.S. Pacific Islands. This study analyzes environmental justice within the U.S. Territory of Guam using existing geospatial analysis methods and publicly available environmental, climate, and socio-economic data. This research aims to: (1) spatially map relevant demographic, environmental, and climate data and (2) determine the correlation, if any, between the exposure to environmental hazards, climate …

Invariance Explains Empirical Success Of Many Intelligent Techniques, Olga Kosheleva, Vladik Kreinovich

Departmental Technical Reports (CS)

In many applications of intelligent computing, we need to choose an appropriate function -- e.g., an appropriate re-scaling function, or an appropriate aggregation function. In applications of intelligent techniques, the problem of selecting an optimal function is usually too complex or too imprecise to be solved analytically, so the best functions are found empirically, by trying a large number of alternatives. In this paper, we show that in many such cases, the resulting empirical choice can be explained by natural invariance ideas. Example range from applications to building blocks of intelligent techniques -- such as aggregation (including hierarchical aggregation) and …

A Quantitative Evaluation Of Explainable Ai Methods Using The Depth Of Decision Tree, Nizar Abdulaziz Mahyoub Ahmed, Adi̇l Alpkoçak

Turkish Journal of Electrical Engineering and Computer Sciences

It is necessary to develop an explainable model to clarify how and why a medical model makes a particular decision. Local posthoc explainable AI (XAI) techniques, such as SHAP and LIME, interpret classification system predictions by displaying the most important features and rules underlying any prediction locally. Therefore, in order to compare two or more XAI methods, they must first be evaluated qualitatively or quantitatively. This paper proposes quantitative XAI evaluation metrics that are not based on biased and subjective human judgment. On the other hand, it is dependent on the depth of the decision tree (DT) to automatically and …

Chemical Disease Relation Extraction Through The Combination Of Multiple Mention Levels: Relscan+, Stanley Chika Onye, Nazi̇fe Di̇mi̇li̇ler, Ari̇f Akkeleş

Turkish Journal of Electrical Engineering and Computer Sciences

Chemical-induced disease (CID) relation extraction has been pivotal in the understanding of biological processes. A CID relation between a chemical and disease entity may be extracted either from a single sentence or from two or more adjacent sentences. We use `intrasentence level' to refer to the mention of the desired entities in the same sentence and `intersentence level? to refer to the mention of these entities in two or more adjacent sentences. This study proposes a three-phase architecture for extracting CID relations from biomedical literature by considering both sentence levels and additionally the combination of these two sentence levels which …

Hierarchical Semantic-Aware Neural Code Representation, Yuan Jiang, Xiaohong Su, Christoph Treude, Tiantian Wang

Research Collection School Of Computing and Information Systems

Code representation is a fundamental problem in many software engineering tasks. Despite the effort made by many researchers, it is still hard for existing methods to fully extract syntactic, structural and sequential features of source code, which form the hierarchical semantics of the program and are necessary to achieve a deeper code understanding. To alleviate this difficulty, we propose a new supervised approach based on the novel use of Tree-LSTM to incorporate the sequential and the global semantic features of programs explicitly into the representation model. Unlike previous techniques, our proposed model can not only learn low-level syntactic information within …

Diacritics Correction In Turkish With Context-Aware Sequence To Sequence Modeling, Asi̇ye Tuba Özge, Özge Bozal, Umut Özge

Turkish Journal of Electrical Engineering and Computer Sciences

Digital texts in many languages have examples of missing or misused diacritics which makes it hard for natural language processing applications to disambiguate the meaning of words. Therefore, diacritics restoration is a crucial step in natural language processing applications for many languages. In this study we approach this problem as bidirectional transformation of diacritical letters and their ASCII counterparts, rather than unidirectional diacritic restoration. We propose a context-aware character-level sequence to sequence model for this transformation. The model is language independent in the sense that no language-specific feature extraction is necessary other than the utilization of word embeddings and is …

Robustness And Cross-Lingual Transfer: An Exploration Of Out-Of-Distribution Scenario In Natural Language Processing, Yu, Sicheng

Dissertations and Theses Collection (Open Access)

Most traditional machine learning or deep learning methods are based on the premise that training data and test data are independent and identical distributed, i.e., IID. However, it is just an ideal situation. In real-world applications, test set and training data often follow different distributions, which we refer to as the out of distribution, i.e., OOD, setting. As a result, models trained with traditional methods always suffer from an undesirable performance drop on the OOD test set. It's necessary to develop techniques to solve this problem for real applications. In this dissertation, we present four pieces of work in the …

Towards Explaining Variation In Entrainment, Andreas Weise

Dissertations, Theses, and Capstone Projects

Entrainment refers to the tendency of human speakers to adapt to their interlocutors to become more similar to them. This affects various dimensions and occurs in many contexts, allowing for rich applications in human-computer interaction. However, it is not exhibited by every speaker in every conversation but varies widely across features, speakers, and contexts, hindering broad application. This variation, whose guiding principles are poorly understood even after decades of entrainment research, is the subject of this thesis. We begin with a comprehensive literature review that serves as the foundation of our own work and provides a reference to guide future …

Grammar Competition Explored In Two Case Studies: The Null Subject Stage In English-Speaking Children And The Variation Observed In Old English, Soumik Dey

Dissertations, Theses, and Capstone Projects

Grammar competition theory postulates that variation in a speaker is the result of different grammars competing against each other. This study performs an analysis of two case studies of empirical observations attributed to possible grammar competition — subject drop in English-speaking children and variation observed in Old English.

Children in an English-speaking environment drop subjects early on during acquisition. Orfitelli and Hyams (2012) find that young English-speaking children mistakenly interpret imperative null subject utterances as declaratives. They suggest that this misinterpretation can be attributed to performance factors, which leads to grammar competition and subsequently subject drop in English children. We …

Influence Level Prediction On Social Media Through Multi-Task And Sociolinguistic User Characteristics Modeling, Denys Katerenchuk

Dissertations, Theses, and Capstone Projects

Prediction of a user’s influence level on social networks has attracted a lot of attention as human interactions move online. Influential users have the ability to influence others’ behavior to achieve their own agenda. As a result, predicting users’ level of influence online can help to understand social networks, forecast trends, prevent misinformation, etc. The research on user influence in social networks has attracted much attention across multiple disciplines, from social sciences to mathematics, yet it is still not well understood. One of the difficulties is that the definition of influence is specific to a particular problem or a domain, …

Data-Centric Machine Learning For Speech And Audio, Ali Raza Syed

Dissertations, Theses, and Capstone Projects

There is growing recognition of the importance of data-centric methods for building machine learning systems. Data-centric methods assume a fixed model and iterate over the data to improve system performance. This is in contrast to traditional model-centric approaches, which assume a fixed dataset and iterate over models for the same ends. Data-centric machine learning is driven by the observation that, beyond the size of the training data, model performance depends on factors such as the quality of the annotations, and whether the data are representative of conditions in which models will be deployed. This is particularly of interest in the …

Analyzing Fluctuation Of Topics And Public Sentiment Through Social Media Data, Haoyue Liu

Dissertations

Over the past decade years, Internet users were expending rapidly in the world. They form various online social networks through such Internet platforms as Twitter, Facebook and Instagram. These platforms provide a fast way that helps their users receive and disseminate information and express personal opinions in virtual space. When dealing with massive and chaotic social media data, how to accurately determine what events or concepts users are discussing is an interesting and important problem.

This dissertation work mainly consists of two parts. First, this research pays attention to mining the hidden topics and user interest trend by analyzing real-world …

Perspectives On Reproductive Health Education Among Javanese Parents, Bagoes Widjanarko, Ratih Indraswari, Aditya Kusumawati, Novia Handayani

Kesmas

This study aimed to investigate the perspectives of reproductive health education among Javanese parents of children aged 9-11 years. This cross-sectional study was conducted with 12,306 parents in Semarang City, Central Java Province, Indonesia, using a purposive sampling technique. Some parents agreed that reproductive health education at home was unnecessary (29.5%), taboo (45%), difficult (73.1%), and awkward (41.5%). Most parents (72.7%) were not transparent in providing reproductive health information to their children by using other terms to name the genitals, considering the politeness aspect. Good practices of providing reproductive health information are slightly more common in mothers (54.2%), unemployed parents …

Understanding Consumers' Use Experience On Electrically Heated Jacket: A Study On Online Review Using Topic Modeling, Md Nakib-Ul Hasan

LSU Doctoral Dissertations

The demand for heated jackets is anticipated to be fuelled by frequent temperature drops, severe winter weather, and increasing outdoor activities. Electrically heated jackets (EHJ) are primarily marketed through online distribution channels and expansion of online sales channels is expected to boost the global market. Consumers are increasingly relying on online reviews from other consumers to help them decide what to buy. Businesses also actively monitor and manage their online reviews to build trust in their brand and make it more likely that customers will buy. Traditional approaches for assessing customer behavior, such as market research surveys and focus groups, …

Interpreting Song Lyrics With An Audio-Informed Pre-Trained Language Model, Yixiao Zhang, Junyan Jiang, Gus Xia, Simon Dixon

Machine Learning Faculty Publications

Lyric interpretations can help people understand songs and their lyrics quickly, and can also make it easier to manage, retrieve and discover songs efficiently from the growing mass of music archives. In this paper we propose BART-fusion, a novel model for generating lyric interpretations from lyrics and music audio that combines a large-scale pre-trained language model with an audio encoder. We employ a cross-modal attention module to incorporate the audio representation into the lyrics representation to help the pre-trained language model understand the song from an audio perspective, while preserving the language model’s original generative performance. We also release the …

Data Engineering Techniques And Designs With Music Generating Neural Networks, Noah Solomon

Honors Program Theses and Projects

The generation of music artificially is an interesting concept to many and has received a lot of attention in recent years. The advancement of neural networks has allowed for the creation of models that can seemingly generate music creatively to mimic a specific genre or composer. This project delved deep into the many ways to construct music generating neural networks and compared different model architectures and data engineering techniques. Three main types of models were implemented and the resulting generated music was evaluated with respect to the melody, note agreeableness, and rhythm. These models used the Bach Chorales corpus as …

Spectral Analysis Of Multiscale Cultural Traits On Twitter, Chandler Squires, Nikhil Kunapuli, Yaneer Bar-Yam, Alfredo Morales

Northeast Journal of Complex Systems (NEJCS)

Understanding and mapping the emergence and boundaries of cultural areas is a challenge for social sciences. In this paper, we present a method for analyzing the cultural composition of regions via Twitter hashtags. Cultures can be described as distinct combination of traits which we capture via principal component analysis (PCA). We investigate the top 8 PCA components of an area including France, Spain, and Portugal, in terms of the geographic distribution of their hashtag composition. We also discuss relationships between components and the insights those relationships can provide into the structure of a cultural space. Finally, we compare the spatial …

Augmented Creativity: Leveraging Natural Language Processing For Creative Writing, Daniel Plate, James Hutson

Faculty Scholarship

Recent advances have moved natural language processing (NLP) capabilities with artificial intelligence beyond mere grammar and spell-checking functionality. One such new use that has arisen is the ability to suggest new content to writers to inspire new ideas by using “machine-in-the-loop” strategies in creative writing. In order to explore the possibilities of such a strategy, this study provides a model to be adopted in creative writing courses in higher education. An NLP application was created using Python and spaCy and deployed via Streamlit. The AI allowed students to see if their grammar aligned with those principles and techniques taught in …