Digital Commons Network^™

Frame-Voyager: Learning To Query Frames For Video Large Language Models, Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang, Jiawei Wu, Hao Zhang, Qianru Sun

Research Collection School Of Computing and Information Systems

Video Large Language Models (Video-LLMs) have made remarkable progress in video understanding tasks. However, they are constrained by the maximum length of input tokens, making it impractical to input entire videos. Existing frame selection approaches, such as uniform frame sampling and text-frame retrieval, fail to account for the information density variations in the videos or the complex instructions in the tasks, leading to sub-optimal performance. In this paper, we propose Frame-Voyager that learns to query informative frame combinations, based on the given textual queries in the task. To train Frame-Voyager, we introduce a new data collection and labeling pipeline, by …

Pearl: Towards Permutation-Resilient Llms, Liang Chen, Li Shen, Yang Deng, Xiaoyan Zhao, Bin Liang, Kam-Fai Wong

Research Collection School Of Computing and Information Systems

The in-context learning (ICL) capability of large language models (LLMs) enables them to perform challenging tasks using provided demonstrations. However, ICL is highly sensitive to the ordering of demonstrations, leading to instability in predictions. This paper shows that this vulnerability can be exploited to design a natural attack - difficult for model providers to detect - that achieves nearly 80% success rate on LLaMA-3 by simply permuting the demonstrations. Existing mitigation methods primarily rely on post-processing and fail to enhance the model's inherent robustness to input permutations, raising concerns about safety and reliability of LLMs. To address this issue, we …

Alphaedit: Null-Space Constrained Knowledge Editing For Language Models, Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, Tat‑Seng Chua

Research Collection School Of Computing and Information Systems

Large language models (LLMs) often exhibit hallucinations, producing incorrector outdated knowledge. Hence, model editing methods have emerged to enabletargeted knowledge updates. To achieve this, a prevailing paradigm is the locatingthen-editing approach, which first locates influential parameters and then edits themby introducing a perturbation. While effective, current studies have demonstrated that this perturbation inevitably disrupt the originally preserved knowledge within LLMs, especially in sequential editing scenarios. To address this, we introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters. We theoretically prove that this projection ensures the output …

Code Of Faith: Programming Spirituality In The Digital Age, Caleb Martin

Senior Honors Theses

With the increasing pervasiveness of technology in our daily lives, it is critical to consider the potential effects on an individual's religious practices and beliefs from software applications developed within a religious framework. This thesis delves into the relationship between software development and religious experiences, examining the manner in which the creation, operation, and application of religious technology can mold and impact an individual's spiritual development. This thesis aims to inform the implementation of technologies with a nonsecular application, ensuring that they are developed with a deep respect for the nuances of religious experience. The findings of this study will …

Can Llms Replace Manual Annotation Of Software Engineering Artifacts?, Toufique Ahmed, Premkumar Devanbu, Christoph Treude, Michael Pradel

Research Collection School Of Computing and Information Systems

Experimental evaluations of software engineering innovations, e.g., tools and processes, often include human-subject studies as a component of a multi-pronged strategy to obtain greater generalizability of the findings. However, human-subject studies in our field are challenging, due to the cost and difficulty of finding and employing suitable subjects, ideally, professional programmers with varying degrees of experience. Meanwhile, large language models (LLMs) have recently started to demonstrate human-level performance in several areas. This paper explores the possibility of substituting costly human subjects with much cheaper LLM queries in evaluations of code and code-related artifacts. We study this idea by applying six …

Chatcrs: Incorporating External Knowledge And Goal Guidance For Llm-Based Conversational Recommender Systems, Chuang Li, Yang Deng, Hengchang Hu, Min-Yen Kan, Haizhou Li

Research Collection School Of Computing and Information Systems

This paper aims to efficiently enable large language models (LLMs) to use external knowledge and goal guidance in conversational recommender system (CRS) tasks. Advanced LLMs (e.g., ChatGPT) are limited in domain-specific CRS tasks for 1) generating grounded responses with recommendation-oriented knowledge, or 2) proactively leading the conversations through different dialogue goals. In this work, we first analyze those limitations through a comprehensive evaluation, showing the necessity of external knowledge and goal guidance which contribute significantly to the recommendation accuracy and language quality. In light of this finding, we propose a novel ChatCRS framework to decompose the complex CRS task into …

Agentstudio: A Toolkit For Building General Virtual Agents, Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, Shuicheng Yan

Research Collection School Of Computing and Information Systems

General virtual agents need to handle multimodal observations, master complex action spaces, and self-improve in dynamic, open-domain environments. However, existing environments are often domain-specific and require complex setups, which limits agent development and evaluation in real-world settings. As a result, current evaluations lack in-depth analyses that decompose fundamental agent capabilities. We introduce AgentStudio, a trinity of environments, tools, and benchmarks to address these issues. AgentStudio provides a lightweight, interactive environment with highly generic observation and action spaces, e.g., video observations and GUI/API actions. It integrates tools for creating online benchmark tasks, annotating GUI elements, and labeling actions in videos. Based …

The Impact Of Drought And Socio-Economic Factors On Attitudes Toward Immigrants In South Africa, Pamela Antwi-Nimarko

Graduate Program in International Studies Theses & Dissertations

Estimates of the global migrant population have steadily increased, with current data from the International Organization for Migration (IOM) reporting approximately 281 million migrants worldwide. Alongside this growth, anti-immigrant sentiments have risen in many host countries. This study seeks to contribute to the research on attitudes toward immigrants by exploring climate-related factors, particularly drought, as a key influence.

Leveraging a unique dataset combining the World Values Survey (WVS) and Afrobarometer, the study examines how drought, along with individual and contextual factors, affects attitudes toward immigrants. This dataset allows for an in-depth analysis of the direct impact of drought on public …

Bigcodebench: Benchmarking Code Generation With Diverse Function Calls And Complex Instructions, T.Y. Zhuo, M.C. Vu, J. Chim, ..., David Lo

Research Collection School Of Computing and Information Systems

Task automation has been greatly empowered by the recent advances in Large Language Models (LLMs) via Python code, where the tasks ranging from software engineering development to general-purpose reasoning. While current benchmarks have shown that LLMs can solve tasks using programs like human developers, the majority of their evaluations are limited to short and self-contained algorithmic tasks or standalone function calls. Solving challenging and practical tasks requires the capability of utilizing diverse function calls as tools to efficiently implement functionalities like data analysis and web development. In addition, using multiple tools to solve a task needs compositional reasoning by accurately …

Automated Data Analysis For Concussion Patient Records: A Flutter-Based Desktop Application, Fhaheem Tadamarry

USF Tampa Graduate Theses and Dissertations

Concussions are a prevalent and complex medical condition requiring careful clinical assessment and data-driven insights for effective management. This thesis presents the development of an automated data analysis system for concussion patient records, integrating Flutter-based desktop application development with SQL-driven data processing. The system provides a streamlined, interactive interface for clincians and researchers to upload, visualize, and analyze patient data efficiently.

The proposed solution automates data cleaning, preprocessing, and statistical analysis, ensuring robust and reliable insights into demographic, clinical, and recovery-related factors. Key analyses include sex-based differences injury mechanisms, prior head injury impact, mood disorder correlations, and time-to-treatment variations. The …

Addressing Equity Issues In Elementary Computer Science Education: Knowns, Unknowns, And Implications For Future Work, Mike Karlin, Yin-Chan Janet Liao, Swati Mehta, Afreen Iqbal, Mahya Minaiy, Minhye Son, Jessica Pandya

Journal of Computer Science Integration

In 2016, a national coalition of stakeholders released the K-12 computer science (CS) framework. In the time since, there has been an increased push at the local, state, and national level to integrate CS knowledge and skills into K-12 education. Despite this push, significant equity issues exist within the field. While growing research has been done on CS equity issues at the high school level, we know these equity gaps often begin to emerge in elementary school where less is known. Therefore, we conducted a systematic literature review to better understand and explore the elementary CS equity research landscape from …

New View Of Some Propositions On Hesitant Fuzzy Set And Its Application In Selecting The Best Person In Any Job, Manar Mohamed Omran Dr., Arafa A. Nasef A.Dr, Reham Abd El-Aziz Abo Khadra Dr., Mahmoud Arafa Nasef Dr.

Journal of Engineering Research

An essential part of uncertainty is played by the hesitant fuzzy set (HFS). So, it can be utilized when making decisions. The suggested use of HFS to choose the best candidate for any post is thoroughly discussed and introduces new HFS concepts

On Large Language Models In National Security Applications, William N. Caballero, Phillip R. Jenkins

Faculty Publications

The overwhelming success of GPT-4 in early 2023 highlighted the transformative potential of large language models (LLMs) across various sectors, including national security. This article explores the implications of LLM integration within national security contexts, analyzing their potential to revolutionize information processing, decision-making, and operational efficiency. Whereas LLMs offer substantial benefits, such as automating tasks and enhancing data analysis, they also pose significant risks, including hallucinations, data privacy concerns, and vulnerability to adversarial attacks. Through their coupling with decision-theoretic principles and Bayesian reasoning, LLMs can significantly improve decision-making processes within national security organizations. Namely, LLMs can facilitate the transition from …

Fuzzy-Virtual Inertia Control To Improve The Frequency Response Of Multi-Area Power Systems, Nourelhouda Djaraf, Yacine Daili, Abderrahim Zemmit, Abdelghani Harrag

Turkish Journal of Electrical Engineering and Computer Sciences

Virtual inertia control (VIC) is essential for power systems dominated by electronic devices to compensate for the lack of inertia and ensure frequency regulation. However, most existing VICs often focus solely on optimizing the virtual inertia parameter to adapt to the high penetration of renewable energy sources (RESs) without considering the damping factor. This oversight can lead to significant fluctuations and power mismatches, especially in interconnected systems where the coordination between MGs is sensitive and essential, and there is a risk of propagation of deviations between MGs, which makes the control more complex. To address these issues, this paper presents …

Conversational Open-Domain Question Answering For Resource-Constrained Languages, Emrah Budur, Tunga Güngör

Turkish Journal of Electrical Engineering and Computer Sciences

The growing interest in Conversational AI has led to the development of Conversational OpenQA systems as a crucial step for meeting users' information needs in real world scenarios. Conversational OpenQA systems enhance standard OpenQA performance by leveraging conversation history of the users. However, building effective Conversational OpenQA systems requires large-scale Conversational OpenQA datasets, often limited to the English language, hindering progress in low-resource languages. We present a robust Conversational OpenQA system enhanced by conversational context, designed for languages with limited resources and exemplified in our case study for Turkish. To address data limitations in a cost-effective way, we repurpose existing …

Three-Way Decision Clustering Algorithm Fusion Of Mutant Fireflies Algorithm, Zhaobin Li, Jun Ye, Haoyan Zhou, Yixin Wang, Yuzhen Han

Journal of System Simulation

Abstract: To address problems such as the premature phenomenon in the three-way clustering algorithm caused by the random selection of initial cluster centers and the need for repeated experiments to determine the value of q in the q-nearest neighbor concept, a three-way clustering algorithm optimized by a variant of the firefly algorithm is proposed. The firefly algorithm is employed to solve the problem of sensitivity to initial cluster centers. The target function value is taken as the brightness intensity of firefly to search the clustering center point, and the optimal solution is taken as the clustering center of the algorithm …

The Sustainability Stones: Culturally Embedded Conservation Strategies And Their Vulnerability In Maupiti, French Polynesia, Russell Fielding, Fiona Gimenez

Honors College Faculty Publications

This paper explores culturally embedded conservation strategies through the lens of the traditional agroforestry and fisheries cycles in Maupiti, French Polynesia. By pairing certain breadfruit cultivars with specific fish species, the island’s community created a culinary system that regulates the seasonal consumption of marine resources, making the sustainable use of those resources more likely. While modern pressures such as reduced breadfruit diversity, competition with imported foods, and climate change have weakened these traditional practices, they remain an example of local ecological knowledge guiding conservation. The study highlights the threat of losing sustainable resource-use practices and biodiversity as both biodiversity and …

Examining Intersectional Queer Biases In Large Language Models: A Combined Statistical And Visual-Qualitative Approach For Quantification And Explanation, Huu Duong (Chip) Nguyen

Computer Science Senior Theses

Despite significant advancements in research on (intersectional) social biases in Large Language Models (LLMs), intersectional biases affecting subgroups within the LGBTQ+ community remain critically understudied. Existing bias detection methodologies often prioritize quantification but lack depth in explaining the specific stereotypes/biases that shape evaluation metrics. To address these gaps, this study proposes a combined statistical and visual-qualitative approach to quantify and identify persistent intersectional queer biases in five recent, state-of-the-art LLMs through a downstream story generation task. Findings from analysis uncover substantial evidence of stereotypes that perpetuate harmful, reductive narratives against intersectionally marginalized groups within the LGBTQ+ community. To promote public …

Feature Manifold Transformer For Detection Of Differential Item Functioning: Visual Detection Of Categorical Feature Nonconformity Through Attention-Based Analysis, Derrick A. Cox, Tanvi Banerjee, William L. Romine

Computer Science and Engineering Faculty Publications

Methods for interpreting complex feature interactions in educational assessment data remain a critical challenge, with traditional statistical approaches often creating barriers to accessibility and interpretability. We introduce the Feature Manifold Transformer (FMT), a novel machine learning approach that leverages dimensionality reduction, representation learning, and transformer architectures to visualize and interpret feature relationships in categorical data. Using the Concept Inventory of Natural Selection (CINS) and Concept Assessment of Natural Selection (CANS) datasets as testbeds, we demonstrate the FMT’s ability to capture subtle relationships between student demographics and response patterns. Our methodology enables both global and local pattern analysis, providing interpretable visualizations …

Next Arrival And Destination Prediction Via Spatiotemporal Embedding With Urban Geography And Human Mobility Data, Pengjiang Li, Zaitian Wang, Xinhao Zhang, Pengfei Wang, Kunpeng Liu

Computer Science Faculty Publications and Presentations

With the development of transportation networks, countless trajectory data are accumulated, and understanding human mobility from traffic data could be helpful for smart cities, urban computing, and urban planning. Extracting valuable insights from traffic data, such as taxi trajectories, can significantly improve residents’ daily lives. There are many studies on spatiotemporal data mining. As we know, arrival prediction or regional function detection encompasses important tasks for traffic management and urban planning. However, trajectory data are often mutilated because of personal privacy and hardware limitations, i.e., we usually can only obtain partial trajectory information. In this paper, we develop an embedding …

Energy-Aware Clustering Using Intelligent Scheme For Heterogeneous Wireless Sensor Networks, Enaam A. Al-Hussain, Ghaida A. Al-Suhail

Karbala International Journal of Modern Science

Heterogeneous Wireless Sensor Networks (WSNs) involve nodes with varying capabilities, such as different energy levels, sensing ranges, and computational abilities, which enable them to execute different tasks professionally. Clustering techniques play a crucial role in improving energy efficiency and reliability in WSNs. The evolution of cluster based WSNs from homogeneous into heterogeneous techniques allowed the deployment of smart devices capable of performing complex operations in in diverse environments. However, the heterogeneity of nodes necessitates more sophisticated and adaptive algorithms to fully exploit these capabilities. This paper proposes a new protocol, referred to as IT2F-HLEACH, which integrates Interval Type-2 Fuzzy Logic …

Logiclm: Robust Application Of Large Language Models With Logic Programming For Data Analytics, Evgeny Skvortsov, Shayan Mirjafari, Ojaswa Garg, Yilin Xia, Shaun Bowers, Bertram Ludäscher

Computer Science Faculty Scholarship

We present LogicLM, an OLAP-style interactive data analysis system that leverages large language models (LLMs) and is configured using Logica, an enhanced logic programming language with aggregation support that compiles to SQL. LogicLM uses an LLM to translate natural language queries by end users into executable code for automatically generating data visualizations. For each natural-language query, LogicLM provides a verifiable OLAP-based configuration that users can view and modify to help ensure results are reliable and accurate. This configuration, with measures, dimensions, and filters defined as logical predicates, offers a unified and user-friendly approach to naturallanguage data exploration, while keeping end …

Leveraging Large Language Models For Knowledge-Free Weak Supervision In Clinical Natural Language Processing, Enshuo Hsu, Kirk Roberts

Faculty, Staff and Student Publications

The performance of deep learning-based natural language processing systems is based on large amounts of labeled training data which, in the clinical domain, are not easily available or affordable. Weak supervision and in-context learning offer partial solutions to this issue, particularly using large language models (LLMs), but their performance still trails traditional supervised methods with moderate amounts of gold-standard data. In particular, inferencing with LLMs is computationally heavy. We propose an approach leveraging fine-tuning LLMs and weak supervision with virtually no domain knowledge that still achieves consistently dominant performance. Using a prompt-based approach, the LLM is used to generate weakly-labeled …

Llms In Network Intrusion Detection – A Comprehensive Analysis, Sudharshan Balaji

USF Tampa Graduate Theses and Dissertations

Network Intrusion Detection Systems (NIDS) play a critical role in identifying and mitigating malicious activities within computer networks. With the rapid evolution of natural language processing (NLP), Large Language Models (LLMs) have emerged as transformative tools across various domains. LLMs, such as OpenAI’s GPT series and Meta’s LLaMA models,have demonstrated remarkable performance in tasks like language generation, reasoning, and classification. Their ability to understand and process vast amounts of data has enabled groundbreaking advancements in areas like healthcare, finance, and cybersecurity. Recent trends highlight their potential to handle unstructured data, perform complex reasoning, and adapt to a wide range of …

Model Explanations For Gender And Ethnicity Bias Mitigation In Ai-Generated Narratives, Martha Otisi Dimgba

Dissertations and Theses

Large Language Models (LLMs) are increasingly utilized in diverse applications, ranging from professional content creation to decision-making systems. However, their outputs often amplify the biases present in their training data, perpetuating stereotypes and reinforcing societal inequities, particularly regarding gender and ethnicity. Such biases can cause tangible harm, especially for underrepresented groups, and require awareness and effective mitigation strategies.

This work explores gender and ethnicity representation in narratives created by generative AI describing 25 occupational fields defined by the U.S. Bureau of Labor Statistics. We examine three large language models (LLMs)--Llama 3.1 70B Instruct, Claude 3.5 Sonnet, and GPT 4.0 Turbo. …

The Future Of Ai: Join The Conversation, Jennifer Wojton, Cassandra Branham, Vijay Tummala, Laxima Niure Kandel, Kayla D. Taylor

Publications

Join the Conversation! The Future is AI? There is so much conflicting information about what AI is capable of, how it could/should be used, by whom and for what purpose. In this panel discussion, we hope to provide a baseline of information that will help all participants think critically and articulate thoughtful questions about the mechanics of AI, ethical use or non-use of AI in particular contexts (school, industry, business, art, etc.), and the impacts we are currently experiencing or are likely to experience. Hear from ERAU faculty of different disciplines to discuss what the current state of AI technology …

Language Processing: The Precedence Of Neural Networks On The Account Of Hidden Markov Models, Dia Eddin Abuzeina

An-Najah University Journal for Research - B (Humanities)

Background: since its discovery at the beginning of the last century, Markov models gain a great popularity, and have been widely used in different domains. However, the most prominent use was in computational linguistics, or what is known as natural language processing (NLP). Abstractly, Markov models are nothing but a statistical representation of a particular system. The mathematical statistical representation of a given system is the heart of Markov theory. Markov models characterized by solid mathematical representation, which significantly promotes using it. No doubt, Markov models are mainly used in prediction and classification, to serve computational linguistics as well as …

If You Were A Sesame Street Character, Which One Would You Be? Natural Language Processing And Personality With Big Bird And Friends, Joseph Uran Meyer

Doctoral Dissertations

This paper examined and compared several natural language processing and machine learning techniques in predicting self-reported Big Five personality traits from text responses. The models were validated on the open-source 2019 SIOP Machine Learning Competition dataset (N = 1,689). The techniques evaluated included bag-of-words, Empath dictionary, LSTM networks, fine-tuning Transformer models, and stacked generalization. Results indicated that the present study’s models had lower error in four of the five constructs analyzed. Limitations of the study include use of an MTurk sample and small sample size. Future research should explore similar techniques on larger applicant samples. Practical implications and contributions to …

Adapting Knowledge Prompt Tuning For Enhanced Automated Program Repair, Xuemeng Cai, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Automated Program Repair (APR) aims to enhance software reliability by automatically generating bug-fixing patches. Recent work has improved the state-of-the-art of APR by fine-tuning pre-trained large language models (LLMs), such as CodeT5, for APR. However, the effectiveness of fine-tuning be-comes weakened in data scarcity scenarios, and data scarcity can be a common issue in practice, limiting fine-tuning performance. To alleviate this limitation, this paper adapts prompt tuning for enhanced APR and conducts a comprehensive study to evaluate its effectiveness in data scarcity scenarios, using three LLMs of different sizes and six diverse datasets across four programming languages. Prompt tuning rewrites …

Mimic: Ai And Ar-Enhanced Multi-Modal, Immersive, Relative Instruction Comprehension, Dhanuja Wanniarachchi, Archan Misra

Research Collection School Of Computing and Information Systems

We present a multimodal instruction comprehension framework, called MImIC, that utilizes visual sensing (including LIDAR and 2D RGB sensing) & AI spatial reasoning capabilities to support more seamless and immersive interaction between humans and AI-driven situated assistive agents. MImIC's key new capability is to support disambiguation of a wider set of relative spatial references that users naturally employ while issuing spatially-situated instructions. To support enhanced visual grounding via a combination of both fully-qualified and relative attribute references, MImIC uses (a) a fine-tuned transformer-based language translation DNN to accurately convert natural verbal commands into a structured set of machine understandable constraints …