Digital Commons Network^™

Digitizing The Cultural Capital: Harnessing Digital Humanities For Heritage Preservation In Bujumbura, Burundi, James Hutson, Pace Ellsworth, Matt Ellsworth, Jean Bosco Ntungirimana

Faculty Scholarship

In an era where the erosion of cultural heritage is increasingly prevalent, there exists a critical imperative to explore and implement innovative methods for the preservation and revitalization of cultural identities, as exemplified by the urgent situation in Bujumbura, Burundi. Central to this study is the exploration of innovative digital methodologies for archiving a wide spectrum of cultural artifacts, including both notable and everyday heritage elements, in Bujumbura. Traditional approaches to biographical and historical profiling have predominantly focused on official records and significant events, often neglecting the richness of personal experiences and everyday interactions that substantially shape cultural identities. To …

Towards Llm-Based Fact Verification On News Claims With A Hierarchical Step-By-Step Prompting Method, Xuan Zhang, Wei Gao

Research Collection School Of Computing and Information Systems

While large pre-trained language models (LLMs) have shown their impressive capabilities in various NLP tasks, they are still underexplored in the misinformation domain. In this paper, we examine LLMs with in-context learning (ICL) for news claim verification, and find that only with 4-shot demonstration examples, the performance of several prompting methods can be comparable with previous supervised models. To further boost performance, we introduce a Hierarchical Step-by-Step (HiSS) prompting method which directs LLMs to separate a claim into several subclaims and then verify each of them via multiple questionsanswering steps progressively. Experiment results on two public misinformation datasets show that …

Understanding The Contribution Of Recommendation Algorithms On Misinformation Recommendation And Misinformation Dissemination On Social Networks, Royal Pathak, Francesca Spezzano, Maria Soledad Pera

Computer Science Faculty Publications and Presentations

Social networks are a platform for individuals and organizations to connect with each other and inform, advertise, spread ideas, and ultimately influence opinions. These platforms have been known to propel misinformation. We argue that this could be compounded by the recommender algorithms that these platforms use to suggest items potentially of interest to their users, given the known biases and filter bubbles issues affecting recommender systems. While much has been studied about misinformation on social networks, the potential exacerbation that could result from recommender algorithms in this environment is in its infancy. In this manuscript, we present the result of …

Cgt-Gan: Clip-Guided Text Gan For Image Captioning, Jiarui Yu, Haoran Li, Yanbin Hao, Bin Zhu, Tong Xu, Xiangnan He

Research Collection School Of Computing and Information Systems

The large-scale visual-language pre-trained model, Contrastive Language-Image Pre-training (CLIP), has significantly improved image captioning for scenarios without human-annotated image-caption pairs. Recent advanced CLIP-based image captioning without human annotations follows a text-only training paradigm, i.e., reconstructing text from shared embedding space. Nevertheless, these approaches are limited by the training/inference gap or huge storage requirements for text embeddings. Given that it is trivial to obtain images in the real world, we propose CLIP-guided text GAN (CgT-GAN), which incorporates images into the training process to enable the model to "see" real visual modality. Particularly, we use adversarial training to teach CgT-GAN to mimic …

A Survey On Aspect-Based Sentiment Analysis: Tasks, Methods, And Challenges, Wenxuan Zhang, Xin Li, Yang Deng, Lidong Bing, Wai Lam

Research Collection School Of Computing and Information Systems

As an important fine-grained sentiment analysis problem, aspect-based sentiment analysis (ABSA), aiming to analyze and understand people's opinions at the aspect level, has been attracting considerable interest in the last decade. To handle ABSA in different scenarios, various tasks are introduced for analyzing different sentiment elements and their relations, including the aspect term, aspect category, opinion term, and sentiment polarity. Unlike early ABSA works focusing on a single sentiment element, many compound ABSA tasks involving multiple elements have been studied in recent years for capturing more complete aspect-level sentiment information. However, a systematic review of various ABSA tasks and their …

Multimodal Fashion Knowledge Extraction As Captioning, Yifei Yuan, Wenxuan Zhang, Yang Deng, Wai Lam

Research Collection School Of Computing and Information Systems

Social media plays a significant role in boosting the fashion industry, where a massive amount of fashion-related posts are generated every day. In order to obtain the rich fashion information from the posts, we study the task of social media fashion knowledge extraction. Fashion knowledge, which typically consists of the occasion, person attributes, and fashion item information, can be effectively represented as a set of tuples. Most previous studies on fashion knowledge extraction are based on the fashion product images without considering the rich text information in social media posts. Existing work on fashion knowledge extraction in social media is …

Constructing Holistic Spatio-Temporal Scene Graph For Video Semantic Role Labeling, Yu Zhao, Hao Fei, Yixin Cao, Bobo Li, Meishan Zhang, Jianguo Wei, Min Zhang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

As one of the core video semantic understanding tasks, Video Semantic Role Labeling (VidSRL) aims to detect the salient events from given videos, by recognizing the predict-argument event structures and the interrelationships between events. While recent endeavors have put forth methods for VidSRL, they can be mostly subject to two key drawbacks, including the lack of fine-grained spatial scene perception and the insufficiently modeling of video temporality. Towards this end, this work explores a novel holistic spatio-temporal scene graph (namely HostSG) representation based on the existing dynamic scene graph structures, which well model both the fine-grained spatial semantics and temporal …

Understanding The Impact Of Trade Policy Effect Uncertainty On Firm-Level Innovation Investment: A Deep Learning Approach, Daniel. Chen, Nan Hu, Peng. Liang, Morgan. Swink

Research Collection School Of Computing and Information Systems

Integrating the real options perspective and resource dependence theory, this study examines how firms adjust their innovation investments to trade policy effect uncertainty (TPEU), a less studied type of firm specific, perceived environmental uncertainty in which managers have difficulty predicting how potential policy changes will affect business operations. To develop a text-based, context-dependent, time-varying measure of firm-level perceived TPEU, we apply Bidirectional Encoder Representations from Transformers (BERT), a state-of-the-art deep learning approach. We apply BERT to analyze the texts of mandatory Management Discussion and Analysis (MD&A) sections of annual reports for a sample of 22,669 firm-year observations from 3,181 unique …

Voxelhap: A Toolkit For Constructing Proxies Providing Tactile And Kinesthetic Haptic Feedback In Virtual Reality, M. Feick, C. Biyikli, K. Gani, A. Wittig, Anthony Tang, A. Krüger

Research Collection School Of Computing and Information Systems

Experiencing virtual environments is often limited to abstract interactions with objects. Physical proxies allow users to feel virtual objects, but are often inaccessible. We present the VoxelHap toolkit which enables users to construct highly functional proxy objects using Voxels and Plates. Voxels are blocks with special functionalities that form the core of each physical proxy. Plates increase a proxy’s haptic resolution, such as its shape, texture or weight. Beyond providing physical capabilities to realize haptic sensations, VoxelHap utilizes VR illusion techniques to expand its haptic resolution. We evaluated the capabilities of the VoxelHap toolkit through the construction of a range …

Matk: The Meme Analytical Tool Kit, Ming Shan Hee, Aditi Kumaresan, Nguyen Khoi Hoang, Nirmalendu Prakash, Rui Cao, Roy Ka-Wei Lee

Research Collection School Of Computing and Information Systems

The rise of social media platforms has brought about a new digital culture called memes. Memes, which combine visuals and text, can strongly influence public opinions on social and cultural issues. As a result, people have become interested in categorizing memes, leading to the development of various datasets and multimodal models that show promising results in this field. However, there is currently a lack of a single library that allows for the reproduction, evaluation, and comparison of these models using fair benchmarks and settings. To fill this gap, we introduce the Meme Analytical Tool Kit (MATK), an open-source toolkit specifically …

Revisiting Disentanglement And Fusion On Modality And Context In Conversational Multimodal Emotion Recognition, Bobo Li, Hao Fei, Lizi Liao, Yu Zhao, Chong Teng, Tat-Seng Chua, Donghong Ji, Fei Li

Research Collection School Of Computing and Information Systems

It has been a hot research topic to enable machines to understand human emotions in multimodal contexts under dialogue scenarios, which is tasked with multimodal emotion analysis in conversation (MM-ERC). MM-ERC has received consistent attention in recent years, where a diverse range of methods has been proposed for securing better task performance. Most existing works treat MM-ERC as a standard multimodal classification problem and perform multimodal feature disentanglement and fusion for maximizing feature utility. Yet after revisiting the characteristic of MM-ERC, we argue that both the feature multimodality and conversational contextualization should be properly modeled simultaneously during the feature disentanglement …

Complex Knowledge Base Question Answering: A Survey, Yunshi Lan, Gaole He, Jinhao Jiang, Jing Jiang, Zhao Wayne Xin, Ji Rong Wen

Research Collection School Of Computing and Information Systems

Knowledge base question answering (KBQA) aims to answer a question over a knowledge base (KB). Early studies mainly focused on answering simple questions over KBs and achieved great success. However, their performances on complex questions are still far from satisfactory. Therefore, in recent years, researchers propose a large number of novel methods, which looked into the challenges of answering complex questions. In this survey, we review recent advances in KBQA with the focus on solving complex questions, which usually contain multiple subjects, express compound relations, or involve numerical operations. In detail, we begin with introducing the complex KBQA task and …

Pro-Cap: Leveraging A Frozen Vision-Language Model For Hateful Meme Detection, Rui Cao, Ming Shan Hee, Adriel Kuek, Wen Haw Chong, Roy Ka-Wei Lee, Jing Jiang

Research Collection School Of Computing and Information Systems

Hateful meme detection is a challenging multimodal task that requires comprehension of both vision and language, as well as cross-modal interactions. Recent studies have tried to fine-tune pre-trained vision-language models (PVLMs) for this task. However, with increasing model sizes, it becomes important to leverage powerful PVLMs more efficiently, rather than simply fine-tuning them. Recently, researchers have attempted to convert meme images into textual captions and prompt language models for predictions. This approach has shown good performance but suffers from non-informative image captions. Considering the two factors mentioned above, we propose a probing-based captioning approach to leverage PVLMs in a zero-shot …

Effective And Efficient Semantic Representations And Their Applications, Chong Cher Chia

Dissertations and Theses Collection (Open Access)

The proliferation of affordable and compact digital storage has also led to the creation of enormous databases of information, and much attention has been focused on the problem of processing unorganized and unstructured information into some form from which additional value can be extracted. Contemporary approaches to this problem virtually necessitate the use of complex models running on computational systems due to the sheer volume of information to be processed. While it is possible for the model to be fed the actual data as input, typically a representation of the data is used instead. These representations are therefore of interest, …

A Novel Multidimensional Reference Model For Heterogeneous Textual Datasets Using Context, Semantic And Syntactic Clues, Ganesh Kumar, Shuib Basri, Abdullahi Abubakar Imam, Abdullateef Abdullateef Oluwagbemiga Balogun, Hussaini Mamman, Luiz Fernando Capretz

Electrical and Computer Engineering Publications

With the advent of technology and use of latest devices, they produces voluminous data. Out of it, 80% of the data are unstructured and remaining 20% are structured and semi-structured. The produced data are in heterogeneous format and without following any standards. Among heterogeneous (structured, semi-structured and unstructured) data, textual data are nowadays used by industries for prediction and visualization of future challenges. Extracting useful information from it is really challenging for stakeholders due to lexical and semantic matching. Few studies have been solving this issue by using ontologies and semantic tools, but the main limitations of proposed work were …

A Fuzzy Group Decision-Making-Based Method For Green Supplier Selection And Order Allocation, Lu Liu, Wenxin Li, Xiao Song, Bingli Sun, Guanghong Gong

Journal of System Simulation

Abstract: With the intensity of market competitiveness, the worsening of the global environment, and the improvement of public concern about environmental protection, the issue of green purchasing has received considerable attention. The vast majority of existing studies on green purchasing have concentrated on supplier selection with green criteria, so as to realize sustainable operations, whereas it is more feasible and economical for businesses to obtain the proper products from adaptable and suitable suppliers at the right times, rates, and volumes, which is referred to as supplier selection and order allocation. To resolve the aforementioned two crucial challenges, we propose a …

Executive Order On The Safe, Secure, And Trustworthy Development And Use Of Artificial Intelligence, Joseph R. Biden

Copyright, Fair Use, Scholarly Communication, etc.

Section 1. Purpose. Artificial intelligence (AI) holds extraordinary potential for both promise and peril. Responsible AI use has the potential to help solve urgent challenges while making our world more prosperous, productive, innovative, and secure. At the same time, irresponsible use could exacerbate societal harms such as fraud, discrimination, bias, and disinformation; displace and disempower workers; stifle competition; and pose risks to national security. Harnessing AI for good and realizing its myriad benefits requires mitigating its substantial risks. This endeavor demands a society-wide effort that includes government, the private sector, academia, and civil society.

My Administration places the highest urgency …

12th International Conference On Business, Technology And Innovation 2023, University For Business And Technology - Ubt

UBT International Conference

Welcome to IC – UBT 2023

UBT Annual International Conference is the 12th international interdisciplinary peer reviewed conference which publishes works of the scientists as well as practitioners in the area where UBT is active in Education, Research and Development. The UBT aims to implement an integrated strategy to establish itself as an internationally competitive, research-intensive university, committed to the transfer of knowledge and the provision of a world-class education to the most talented students from all background. The main perspective of the conference is to connect the scientists and practitioners from different disciplines in the same place and make …

Artst: Arabic Text And Speech Transformer, Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, Hanan Al Darmaki

Natural Language Processing Faculty Publications

We present ArTST, a pre-trained Arabic text and speech transformer for supporting open-source speech technologies for the Arabic language. The model architecture follows the unified-modal framework, SpeechT5, that was recently released for English, and is focused on Modern Standard Arabic (MSA), with plans to extend the model for dialectal and code-switched Arabic in future editions. We pre-trained the model from scratch on MSA speech and text data, and fine-tuned it for the following tasks: Automatic Speech Recognition (ASR), Text-To-Speech synthesis (TTS), and spoken dialect identification. In our experiments comparing ArTST with SpeechT5, as well as with previously reported results in …

A Psychometric Analysis Of Natural Language Inference Using Transformer Language Models, Antonio Laverghetta Jr.

USF Tampa Graduate Theses and Dissertations

Large language models (LLMs) are poised to transform both academia and industry. But the excitement around these generative AIs has also been met with concern for the true extent of their capabilities. This dissertation helps to address these questions by examining the capabilities of LLMs using the tools of psychometrics. We focus on analyzing the capabilities of LLMs on the task of natural language inference (NLI), a foundational benchmark often used to evaluate new models. We demonstrate that LLMs can reliably predict the psychometric properties of NLI items were those items administered to humans. Through a series of experiments, we …

Similarity Measure Of Plithogenic Cubic Vague Sets: Examples And Possibilities, S. Anitha, A. Francina Shalini

Neutrosophic Systems with Applications

The crisp, fuzzy, intuitionistic fuzzy, and neutrosophic sets are the extension of the plithogenic set, in which elements are characterized by the number of attributes and each attribute can assume many values. To achieve more accuracy and precise exclusion, a contradiction or dissimilarity degree is specified between each attribute and the values of the dominating attribute. A plithogenic cubic vague set is a combination of a plithogenic cubic set and a vague set. The key tool for resolving problems with pattern recognition and clustering analysis is the similarity measure. In this research, we characterize and investigate the similarities between two …

Yet Another Model For Arabic Dialect Identification, Ajinkya Kulkarni, Hanan Al Darmaki

Natural Language Processing Faculty Publications

In this paper, we describe a spoken Arabic dialect identification (ADI) model for Arabic that consistently outperforms previously published results on two benchmark datasets: ADI-5 and ADI-17. We explore two architectural variations: ResNet and ECAPA-TDNN, coupled with two types of acoustic features: MFCCs and features exratected from the pre-trained self-supervised model UniSpeech-SAT Large, as well as a fusion of all four variants. We find that individually, ECAPA-TDNN network outperforms ResNet, and models with UniSpeech-SAT features outperform models with MFCCs by a large margin. Furthermore, a fusion of all four variants consistently outperforms individual models. Our best models outperform previously reported …

Adapting The Adapters For Code-Switching In Multilingual Asr, Atharva Kulkarni, Ajinkya Kulkarni, Miguel Couceiro, Hanan Al Darmaki

Natural Language Processing Faculty Publications

Recently, large pre-trained multilingual speech models have shown potential in scaling Automatic Speech Recognition (ASR) to many low-resource languages. Some of these models employ language adapters in their formulation, which helps to improve monolingual performance and avoids some of the drawbacks of multi-lingual modeling on resource-rich languages. However, this formulation restricts the usability of these models on code-switched speech, where two languages are mixed together in the same utterance. In this work, we propose ways to effectively fine-tune such models on code-switched speech, by assimilating information from both language adapters at each language adaptation point in the network. We also …

Cybervandalism Detection In Wikipedia Using Light Architecture Of 1d-Cnn, Azha Talal Mohammed Ali, Huda Hallawi, Noor D. Al-Shakarchy

Karbala International Journal of Modern Science

The rapid expansion of human-software-agent interaction has come with new issues. Accordingly, different engage-ments are necessary to adapt to changing human needs in dynamic socio-technical systems. Generally, cybervandalism is the act of leaving any negative impact on any piece of writing in an attempt to modify it. In Wikipedia, vandalism is any attempt to modify an article in a way that negatively affects the article's quality. Recently, several automatic detec-tion techniques and related features have been developed to address this issue. This work introduces a deep learning model with a new and light architecture to detect vandalism in Wikipedia articles. …

Trcaptionnet: A Novel And Accurate Deep Turkish Image Captioning Model With Vision Transformer Based Image Encoders And Deep Linguistic Text Decoders, Serdar Yildiz, Abbas Memi̇ş, Songül Varli

Turkish Journal of Electrical Engineering and Computer Sciences

Image captioning is known as a fundamental computer vision task aiming to figure out and describe what is happening in an image or image region. Through an image captioning process, it is ensured to describe and define the actions and the relations of the objects within the images. In this manner, the contents of the images can be understood and interpreted automatically by visual computing systems. In this paper, we proposed the TRCaptionNet a novel deep learning-based Turkish image captioning (TIC) model for the automatic generation of Turkish captions. The model we propose essentially consists of a basic image encoder, …

Infrared Imaging Segmentation Employing An Explainable Deep Neural Network, Xinfei Liao, Dan Wang, Zairan Li, Nilanjan Dey, Rs Simon, Fuqian Shi

Turkish Journal of Electrical Engineering and Computer Sciences

Explainable AI (XAI) improved by a deep neural network (DNN) of a residual neural network (ResNet) and long short-term memory networks (LSTMs), termed XAIRL, is proposed for segmenting foot infrared imaging datasets. First, an infrared sensor imaging dataset is acquired by a foot infrared sensor imaging device and preprocessed. The infrared sensor image features are then defined and extracted with XAIRL being applied to segment the dataset. This paper compares and discusses our results with XAIRL. Evaluation indices are applied to perform various measurements for foot infrared image segmentation including accuracy, precision, recall, F1 score, intersection over union (IoU), Dice …

Synthesizing Sentience: Integrating Large Language Models And Autonomous Agents For Emulating Human Cognitive Complexity, Jay Ratican, James Hutson, Daniel Plate

Faculty Scholarship

The paper aims to present a novel methodology for emulating the intricacies of human cognitive complexity by ingeniously integrating large language models with autonomous agents. Grounded in the theoretical framework of the modular mind theory-originally espoused by Fodor and later refined by scholars such as Joanna Bryson—the study seeks to venture into the untapped potential of large language models and autonomous agents in mirroring human cognition. Recent advancements in artificial intelligence, exemplified by the inception of autonomous agents like Age in GPT, auto GPT, and baby AGI, underscore the transformative capacities of these technologies in diverse applications. Moreover, empirical studies …

Csci 632/Csci 492: Machine Learning, David Harrison

GMAS Course Syllabi

No abstract provided.

Discoverpath: A Knowledge Refinement And Retrieval System For Interdisciplinarity On Biomedical Research, Yu-Neng Chuang, Guanchu Wang, Chia-Yuan Chang, Kwei-Herng Lai, Daochen Zha, Ruixiang Tang, Fan Yang, Alfredo Costilla Reyes, Kaixiong Zhou, Xiaoqian Jiang, Xia Hu

Faculty, Staff and Student Publications

The exponential growth in scholarly publications necessitates advanced tools for efficient article retrieval, especially in interdisciplinary fields where diverse terminologies are used to describe similar research. Traditional keyword-based search engines often fall short in assisting users who may not be familiar with specific terminologies. To address this, we present a knowledge graph based paper search engine for biomedical research to enhance the user experience in discovering relevant queries and articles. The system, dubbed DiscoverPath, employs Named Entity Recognition (NER) and part-of-speech (POS) tagging to extract terminologies and relationships from article abstracts to create a KG. To reduce information overload, DiscoverPath …

Hallucination Detection: Robustly Discerning Reliable Answers In Large Language Models, Yuyuan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Dayiheng Liu, Dongmei Zhang, Zhixu Li, Yanghua Xiao

Research Collection School Of Computing and Information Systems

Large language models (LLMs) have gained widespread adoption in various natural language processing tasks, including question answering and dialogue systems. However, a major drawback of LLMs is the issue of hallucination, where they generate unfaithful or inconsistent content that deviates from the input source, leading to severe consequences. In this paper, we propose a robust discriminator named RelD to effectively detect hallucination in LLMs' generated answers. RelD is trained on the constructed RelQA, a bilingual question-answering dialogue dataset along with answers generated by LLMs and a comprehensive set of metrics. Our experimental results demonstrate that the proposed RelD successfully detects …