Digital Commons Network^™

Rethinking Retrieval Augmented Fine-Tuning In An Evolving Llm Landscape, Nicholas Sager, Timothy Cabaza, Matthew Cusack, Ryan Bass, Joaquin Dominguez

SMU Data Science Review

This study explores the utilization of Retrieval Augmented Fine-Tuning (RAFT) to enhance the performance of Large Language Models (LLMs) in domain-specific Retrieval Augmented Generation (RAG) tasks. By integrating domain-specific information during the retrieval process, RAG aims to reduce hallucination and improve the accuracy of LLM outputs. We investigate the use of RAFT, an approach that enhances LLMs by incorporating domain-specific knowledge and effectively handling distractor documents. This paper validates previous work, which found that RAFT can considerably improve the performance of Llama2-7B in specific domains. We also expand upon previous work into new state-of-the-art open-source models and other datasets with …

Technoculture And Language Models In Archaeology: Reconstructing And Preserving Cultural Narratives Through Digital Humanities, James Hutson

Faculty Scholarship

Technoculture, which examines the intersection of culture and technology, has increasingly permeated archaeological practice, transforming both scholarly research and public engagement [1-3]. The introduction of digital tools such as virtual reality (VR), geographic information systems (GIS), and large language models (LLMs) has democratized access to archaeological knowledge, enabling communities to engage more actively with their cultural heritage [4-6]. This short article explores the mutual influence of technocultural studies and AI technologies on archaeology, with a focus on the preservation and reconstruction of cultural narratives through digital means.

The first aspect of this intersection lies in how technocultural tools are creating …

Ensemble Pretrained Language Models To Extract Biomedical Knowledge From Literature, Zhao Li, Qiang Wei, Liang-Chin Huang, Jianfu Li, Yan Hu, Yao-Shun Chuang, Jianping He, Avisha Das, Vipina Kuttichi Keloth, Yuntao Yang, Chiamaka S Diala, Kirk E Roberts, Cui Tao, Xiaoqian Jiang, W Jim Zheng, Hua Xu

Faculty, Staff and Student Publications

OBJECTIVES: The rapid expansion of biomedical literature necessitates automated techniques to discern relationships between biomedical concepts from extensive free text. Such techniques facilitate the development of detailed knowledge bases and highlight research deficiencies. The LitCoin Natural Language Processing (NLP) challenge, organized by the National Center for Advancing Translational Science, aims to evaluate such potential and provides a manually annotated corpus for methodology development and benchmarking.

MATERIALS AND METHODS: For the named entity recognition (NER) task, we utilized ensemble learning to merge predictions from three domain-specific models, namely BioBERT, PubMedBERT, and BioM-ELECTRA, devised a rule-driven detection method for cell line and …

Improving Large Language Models For Clinical Named Entity Recognition Via Prompt Engineering, Yan Hu, Qingyu Chen, Jingcheng Du, Xueqing Peng, Vipina Kuttichi Keloth, Xu Zuo, Yujia Zhou, Zehan Li, Xiaoqian Jiang, Zhiyong Lu, Kirk Roberts, Hua Xu

Faculty, Staff and Student Publications

IMPORTANCE: The study highlights the potential of large language models, specifically GPT-3.5 and GPT-4, in processing complex clinical data and extracting meaningful information with minimal training data. By developing and refining prompt-based strategies, we can significantly enhance the models' performance, making them viable tools for clinical NER tasks and possibly reducing the reliance on extensive annotated datasets.

OBJECTIVES: This study quantifies the capabilities of GPT-3.5 and GPT-4 for clinical named entity recognition (NER) tasks and proposes task-specific prompts to improve their performance.

MATERIALS AND METHODS: We evaluated these models on 2 clinical NER tasks: (1) to extract medical problems, treatments, …

Putting Gpt-4o To The Sword: A Comprehensive Evaluation Of Language, Vision, Speech, And Multimodal Proficiency, Sakib Shahriar, Brady D. Lund, Nishith Reddy Mannuru, Muhammad Arbab Arshad, Kadhim Hayawi, Ravi Varma Kumar Bevara, Aashrith Mannuru, Laiba Batool

All Works

As large language models (LLMs) continue to advance, evaluating their comprehensive capabilities becomes significant for their application in various fields. This research study comprehensively evaluates the language, vision, speech, and multimodal capabilities of GPT-4o. The study employs standardized exam questions, reasoning tasks, and translation assessments to assess the model’s language capability. Additionally, GPT-4o’s vision and speech capabilities are tested through image classification and object-recognition tasks, as well as accent classification. The multimodal evaluation assesses the model’s performance in integrating visual and linguistic data. Our findings reveal that GPT-4o demonstrates high accuracy and efficiency across multiple domains in language and reasoning …

Artificial Intelligence In Orthopaedic Education: A Comparative Analysis Of Chatgpt And Bing Ai’S Orthopaedic In-Training Examination Performance, Clark Chen, Vivek Biololikar, Duncan Vannest, James Raphael, Gene Shaffer

Einstein Health Papers

Background: This study evaluated the performance of generative artificial intelligence (AI) models on the Orthopaedic In-Training Examination (OITE), an annual exam administered to U.S. orthopaedic residency programs. Methods: ChatGPT 3.5 and Bing AI GPT 4.0 were evaluated on standardised sets of multiple-choice questions drawn from the American Academy of Orthopaedic Surgeons OITE online question bank spanning 5 years (2018–2022). A total of 1165 questions were posed to each AI system. The performance of both systems was standardised using the latest versions of ChatGPT 3.5 and Bing AI GPT 4.0. Historical data of resident scores taken from the annual OITE technical …

Multi-Modal Alignment Via Hyperbolic Geometry, Suyu Liu

Dissertations and Theses Collection (Open Access)

Strong capabilities of generalization to unseen domains are vital for deep neural networks. While existing methods have shown promising results without source domain access, they mostly rely on models that are extensively pre-trained on source domains or overlook the intricate hierarchical structures inherent in visual and textual features. These limitations may have bad impacts on performances, especially on datasets with many classes. To overcome this, in this paper we propose a novel approach that projects the model onto hyperbolic geometry and employs geometric optimal transport to align cross-modal features in an unsupervised manner. Unlike Euclidean geometry, hyperbolic geometry is characterized …

Text-Driven Video Prediction, Xue Song, Jingjing Chen, Bin Zhu, Yu-Gang Jiang

Research Collection School Of Computing and Information Systems

Current video generation models usually convert signals indicating appearance and motion received from inputs (e.g., image and text) or latent spaces (e.g., noise vectors) into consecutive frames, fulfilling a stochastic generation process for the uncertainty introduced by latent code sampling. However, this generation pattern lacks deterministic constraints for both appearance and motion, leading to uncontrollable and undesirable outcomes. To this end, we propose a new task called Text-driven Video Prediction (TVP). Taking the first frame and text caption as inputs, this task aims to synthesize the following frames. Specifically, appearance and motion components are provided by the image and caption …

Aligning Human And Computational Coherence Evaluations, Jia Peng Lim, Hady Wirawan Lauw

Research Collection School Of Computing and Information Systems

Automated coherence metrics constitute an efficient and popular way to evaluate topic models. Previous work presents a mixed picture of their presumed correlation with human judgment. This work proposes a novel sampling approach to mining topic representations at a large scale while seeking to mitigate bias from sampling, enabling the investigation of widely used automated coherence metrics via large corpora. Additionally, this article proposes a novel user study design, an amalgamation of different proxy tasks, to derive a finer insight into the human decision-making processes. This design subsumes the purpose of simple rating and outlier-detection user studies. Similar to the …

Probing Effects Of Contextual Bias On Number Magnitude Estimation, Xuehao Du, Ping Ji, Wei Qin, Lei Wang, Yunshi Lan

Research Collection School Of Computing and Information Systems

The semantic understanding of numbers requires association with context. However, powerful neural networks overfit spurious correlations between context and numbers in training corpus can lead to the occurrence of contextual bias, which may affect the network's accurate estimation of number magnitude when making inferences in real-world data. To investigate the resilience of current methodologies against contextual bias, we introduce a novel out-of- distribution (OOD) numerical question-answering (QA) dataset that features specific correlations between context and numbers in the training data, which are not present in the OOD test data. We evaluate the robustness of different numerical encoding and decoding methods …

Getting To The Point: Contrasting Directness And Warmth In Motivational Embodied Conversational Agents, Michael O'Mahony, Cathy Ennis, Robert Ross

Conference papers

Enhancing long-term engagement with conversational agents remains a significant challenge. Controlling the perceived warmth or directness of an agent’s personality through the style of its generated text could be used to increase user likeability. This paper reports an investigation of a Wizard-of-Oz (WoZ) mediated study of two variants of a motivational embodied conversational agent to measure user perception of and attitudes towards warmth in interaction style. Results show a significant effect of users preferring an agent with a "more direct" personality for this scenario, though this effect is in many ways nuanced.

Climate Change Prediction Model Using Mcdm Technique Based On Neutrosophic Soft Functions With Aggregate Operators, Kainat Muniba, Muhammad Naveed Jafar, Asma Riffat, Jawaria Mukhtar, Adeel Saleem

Neutrosophic Systems with Applications

The increasing impact of climate change necessitates innovative approaches in modeling and prediction to mitigate its adverse effects. This paper introduces a novel methodology integrating Neutrosophic Soft Functions (NSFs) into climate change prediction frameworks. NSFs, a hybrid of Neutrosophic Set Theory and Soft Set Theory, provide a flexible framework for handling uncertain and imprecise information inherent in climate data. This study explores the application of NSFs in capturing the complex interplay of various climatic variables, including temperature, precipitation, humidity, and atmospheric pressure, thereby enhancing the accuracy and reliability of climate change predictions. By incorporating NSFs into existing predictive models, such …

Genixer : Empowering Multimodal Large Language Models As A Powerful Data Generator, Henry Hengyuan Zhao, Pan Zhou, Mike Zheng Shou

Research Collection School Of Computing and Information Systems

Multimodal Large Language Models (MLLMs) demonstrate exceptional problem-solving capabilities, but few research studies aim to gauge the ability to generate visual instruction tuning data. This paper proposes to explore the potential of empowering MLLMs to generate data independently without relying on GPT-4. We introduce Genixer, a comprehensive data generation pipeline consisting of four key steps: (i) instruction data collection, (ii) instruction template design, (iii) empowering MLLMs, and (iv) data generation and filtering. Additionally, we outline two modes of data generation: task-agnostic and task-specific, enabling controllable output. We demonstrate that a synthetic VQA-like dataset trained with LLaVA1.5 enhances performance on 10 …

Enhancing Stance Classification On Social Media Using Quantified Moral Foundations, Hong Zhang, Quoc-Nam Nguyen, Prasanta Bhattacharya, Wei Gao, Liang Ze Wong, Brandon Siyuan Loh, Joseph J. P. Simons, Jisun An

Research Collection School Of Computing and Information Systems

This study enhances stance detection on social media by incorporating deeper psychological attributes, specifically individuals’ moral foundations. These theoretically-derived dimensions aim to provide an interpretable profile of an individual’s moral concerns which, in recent work, has been linked to behaviour in a range of domains including society, politics, health, and the environment. In this paper, we investigate how moral foundation dimensions can contribute to detecting an individual’s stance on a given target. Specifically, we incorporate moral foundation features extracted from text, along with semantic features, to classify stances at both message-and user-levels using traditional machine learning and Large Language Models …

Granular3d: Delving Into Multi-Granularity 3d Scene Graph Prediction, Kaixiang Huang, Jingru Yang, Jin Wang, Shengfeng He, Zhan Wang, Haiyan He, Qifeng Zhang, Guodong Lu

Research Collection School Of Computing and Information Systems

This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intricate 3D scenes, largely due to a focus on global scene processing and single-scale feature extraction. To overcome these limitations, we introduce Granular3D, a novel approach that shifts the focus towards multi-granularity analysis by predicting relation triplets from specific sub-scenes. One key is the Adaptive Instance Enveloping Method (AIEM), which establishes an approximate envelope structure around irregular instances, providing shape-adaptive local point cloud sampling, thereby …

Learners’ Mathematics Identity And Achievement: Where Does The Teacher Come In?, Luis M. Fernandez, Ursula Nguyen, Rebecca Callahan

School of Mathematical & Statistical Sciences Faculty Publications

In response to recent interest in K-12th students’ mathematics identity formation and its implications for achievement, the present study examines the relationship between learners’ mathematics identity and student achievement while simultaneously accounting for teacher-enacted instructional practices in mathematics. Drawing from HSLS:2009, a nationally representative dataset of 9th–12th students within the United States, we use multiple linear regression analyses to examine how teachers’ mathematical pedagogies and 9th grade students’ perceptions of teacher equity are associated, first, with students’ mathematics identity, and, subsequently, math achievement after accounting for students’ mathematics identity. Ultimately, results from our models reveal a statistically significant relationship between …

Reclaiming Healing Spaces: A Phenomenological Study On The Transformative Power Of Outdoor Therapy From The Lived Experiences Of Black Clinicians Working With Black Clients, Lynn Murphy

Dissertations

This phenomenological study involved assessing the experiences of Black therapists who engaged Black clients in outdoor therapeutic contexts. The study was founded on the existing literature that shows the quality of the therapeutic relationship is pivotal for client retention and the Western standards that have historically favored treatment within indoor environments. To contextualize this research, a comprehensive literature review was commenced, covering topics such as the decolonization of therapy, the historical and present-day relationship between Blacks and the outdoors in the United States, sedentary lifestyles, the psychological benefits of time spent in nature, various types of outdoor therapy, and the …

Health Benefits And Adverse Effects Of Kratom: A Social Media Text-Mining Approach, Abdullah Wahbeh, Mohammad A. Al-Ramahi, Omar El-Gayar, Tareq Nasralah, Ahmed El Noshokaty

All Faculty Scholarship (Archived)

Background: Kratom is a substance that alters one’s mental state and is used for pain relief, mood enhancement, and opioid withdrawal, despite potential health risks. In this study, we aim to analyze the social media discourse about kratom to provide more insights about kratom’s benefits and adverse effects. Also, we aim to demonstrate how algorithmic machine learning approaches, qualitative methods, and data visualization techniques can complement each other to discern diverse reactions to kratom’s effects, thereby complementing traditional quantitative and qualitative methods. Methods: Social media data were analyzed using the latent Dirichlet allocation (LDA) algorithm, PyLDAVis, and t-distributed stochastic neighbor …

The Naked Truth Of My Voice: Surveying The Soul Of An Aspiring Educator From Dusk To Dawn: An Arts-Based Auto-Criticism Inquiry, Siddharth Maan

Electronic Theses and Dissertations

This dissertation is based on auto-criticism, a new qualitative inquiry that combines educational criticism and arts-based research. It focuses on my own experiences as an aspiring educator who struggles with stuttering and public speaking anxiety. The study aims to describe, interpret, evaluate, and thematize my experiences of dealing with public speaking anxiety and stuttering before, during, and after classroom teaching. In addition, this dissertation highlights the potential contributions of auto-criticism as a methodology to qualitative research and higher education. I utilized journaling to document data. Photos and music were also used to enhance the understanding of my lived experiences and …

Diagrams In Involutive Residuated Lattices, Isis A. Gallardo

Electronic Theses and Dissertations

First, we show that every distributive lattice-ordered pregroup can be embedded into a functional algebra over an integral chain, thereby improving the existing Cayley/Holland style embedding theorem. Using this result, we demonstrate that the variety of all dis tributive lattice-ordered pregroups is generated by the functional algebra on the integers. Additionally, we prove that the equational theory of this variety is decidable.

Next, we establish that DLP is equal to the join of its subvarieties LP_n, where 𝑛 ∈ ℤ⁺, consisting of 𝑛-periodic ℓ-pregroups. We also prove that every algebra in LP_n can be embedded …

Developing A Comprehensive Cognitive Model Of Math Achievement, Nina Anderson

Electronic Theses and Dissertations

Recently, downward trends have been reported in U.S. children’s math performance following school disruptions during COVID-19 amidst longstanding concerns for instruction and curricula within the subject. In support of efforts to remedy these declines, the current work presents two studies dedicated to identifying cognitive factors that are most strongly related to math performance, and which therefore offer promising potential targets for assessment and intervention. Both studies use data from the Colorado Learning Disabilities Research Center, which includes participants ages 8 - 16 and multiple well-validated measures of all constructs of interest. Study 1 tests three alternative latent cognitive models of …

Soft Sets Extensions: Innovating Healthcare Claims Analysis, Daniela Gifu, Florentin Smarandache

Branch Mathematics and Statistics Faculty and Staff Publications

In the dynamic arena of healthcare research, where the complexities of data often rival the intricacies of biological systems, the ability to model and analyze such multifaceted datasets is crucial. This comprehensive review delves into the evolution and application of Soft Sets and their extensions, including HyperSoft Sets, SuperHyperSoft Sets, IndetermSoft Sets, IndetermHyperSoft Sets, and TreeSoft Sets, in healthcare claims data analysis.

Scene Text Detection And Recognition Via Discriminative Representation, Liang Zhao

Theses and Dissertations

Scene texts refer to arbitrary text presented in an image captured by a camera in the real world. The tasks of scene text detection and recognition from complex images play a crucial role in computer vision, with potential applications in scene understanding, information retrieval, robotics, autonomous driving, etc. Despite the notable progress made by existing deep-learning methods, achieving accurate text detection and recognition remains challenging for robust real-world applications. The challenges in scene text detection and recognition stem from: 1) diverse text shapes, fonts, colors, styles, layouts, etc.; 2) countless combinations of characters with unfixed attributes for complete detection, coupled …

On Parallelization Of Graph Algorithms, Performance Modelling And Autonomous 3d Printable Object Synthesis, Shams-Ul-Haq Syed

Theses and Dissertations

The degree of hardware level parallelism offered by today’s GPU architecture makes it ideal for problem domains with massive inherent parallelism potential, fields such as computer vision, image processing, graph theory and graph computations. We have identified three problem areas for purpose of this research dissertation, under the umbrella of performance improvement by harnessing the power of GPUs for novel applications. The first area is concerned with k-vertex connectivity in graph theory, the second area deals performance evaluation using extended roofline models for GPU parallel applications and finally the third problem area is related to synthesis 3D printable objects from …

Coffee In Brasil: Sustainability And The Role Of Intermediaries, Livia Petzinger

Capstone Collection

ABSTRACT

Coffee is traded in a highly complex global value chain where countless sustainability initiatives have emerged to address social, economic, and environmental issues, challenges which persist due to dominance of the market structure by largescale traders and roasters. Specialty coffee and direct trade have emerged as alternative trade arrangements that promote high-quality coffee through the use of sustainable practices, transparency, and a shortened supply chain. This research analyzes the role that intermediaries in the Brazilian specialty coffee supply chain play in the implementation of sustainability initiatives asking what kinds of sustainability practices they implement and why, and how this …

Enhancing Clinical Relevance Of Pretrained Language Models Through Integration Of External Knowledge: Case Study On Cardiovascular Diagnosis From Electronic Health Records, Qiuhao Lu, Andrew Wen, Thien Nguyen, Hongfang Liu

Faculty, Staff and Student Publications

Background: Despite their growing use in health care, pretrained language models (PLMs) often lack clinical relevance due to insufficient domain expertise and poor interpretability. A key strategy to overcome these challenges is integrating external knowledge into PLMs, enhancing their adaptability and clinical usefulness. Current biomedical knowledge graphs like UMLS (Unified Medical Language System), SNOMED CT (Systematized Medical Nomenclature for Medicine-Clinical Terminology), and HPO (Human Phenotype Ontology), while comprehensive, fail to effectively connect general biomedical knowledge with physician insights. There is an equally important need for a model that integrates diverse knowledge in a way that is both unified and compartmentalized. …

Ee-Lce: An Event Extraction Framework Based On Llm-Generated Cot Explanation, Yanhua Yu, Yuanlong Wang, Yunshan Ma, Jie Li, Kangkang Lu, Zhiyong Huang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Generative models have been widely used in event extraction. However, the interpretability of event extraction has not been fully investigated. In this paper, we propose an Event Extraction framework based on LLM-generated CoT Explanation EE-LCE, which can generate chain-of-thought-style (CoT-style) explanations for events. To this end, we provide each sample of event datasets with an explanation of the reasoning process using a large language model (LLM) GPT-3.5, and fine-tune the Flan-T5 lightweight language model (LM) supervised by the augmented dataset, enhancing both interpretability and performance of the event extraction. Moreover, we use a prefix tree (trie) to normalize the decoding …

Cybersecurity In Education, Rahima Shelim

Theses, Dissertations and Culminating Projects

Cybersecurity is becoming increasingly important as we rely more on digital devices and programs to conduct our daily lives, including the transfer and storage of personal information. According to research, one of the most critical stages in improving cybersecurity is to implement an effective security awareness program. In this work, we seek to understand the existing level of security knowledge among college students, industry professionals and create a module to help raise the awareness. Our module's primary elements are interaction and the display of alarming effects of reckless cyber behaviors among common Internet/technology users. This report presents a simple systematic …

Chain-Of-Exemplar: Enhancing Distractor Generation For Multimodal Educational Question Generation, Haohao Luo, Yang Deng, Ying Shen, See-Kiong Ng, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Multiple-choice questions (MCQs) are important in enhancing concept learning and student engagement for educational purposes. Despite the multimodal nature of educational content, current methods focus mainly on text-based inputs and often neglect the integration of visual information. In this work, we study the problem of multimodal educational question generation, which aims at generating subject-specific educational questions with plausible yet incorrect distractors based on multimodal educational content. To tackle this problem, we introduce a novel framework, named Chain-of-Exemplar (CoE), which utilizes multimodal large language models (MLLMs) with Chain-of-Thought reasoning to improve the generation of challenging distractors. Furthermore, CoE leverages three-stage contextualized …

Interpretable Tensor Fusion, Saurabh Varshneya, Antoine Ledent, Philipp Liznerski, Andriy Balinskyy, Purvanshi Mehta, Waleed Mustafa, Marius Kloft

Research Collection School Of Computing and Information Systems

Conventional machine learning methods are predominantly designed to predict outcomes based on a single data type. However, practical applications may encompass data of diverse types, such as text, images, and audio. We introduce interpretable tensor fusion (InTense), a multimodal learning method training a neural network to simultaneously learn multiple data representations and their interpretable fusion. InTense can separately capture both linear combinations and multiplicative interactions of the data types, thereby disentangling higher-order interactions from the individual effects of each modality. InTense provides interpretability out of the box by assigning relevance scores to modalities and their associations, respectively. The approach is …