Digital Commons Network^™

Spatially-Aware Speaker For Vision-And-Language Navigation Instruction Generation, Muraleekrishna Gopinathan, Martin Masek, Jumana Abu-Khalaf, David Suter

Research outputs 2022 to 2026

Embodied AI aims to develop robots that can understand and execute human language instructions, as well as communicate in natural languages. On this front, we study the task of generating highly detailed navigational instructions for the embodied robots to follow. Although recent studies have demonstrated significant leaps in the generation of step-by-step instructions from sequences of images, the generated instructions lack variety in terms of their referral to objects and landmarks. Existing speaker models learn strategies to evade the evaluation metrics and obtain higher scores even for low-quality sentences. In this work, we propose SAS (Spatially-Aware Speaker), an instruction generator …

Body-Oriented Gesture Generation System For Medical Interpreter Robots Based On Reinforcement Learning From Human Feedback, Tung Ngo, Emma Murphy, Conor Mcginn, Robert Ross

Conference papers

Medical interpreters are crucial in facilitating communication between healthcare stakeholders who speak different languages. Body-oriented gestures convey critical information essential for accurate and high-quality interpretation in healthcare settings. This study introduces a body-oriented gesture generation system designed for medical interpreter robots based on reinforcement learning from human feedback (RLHF). The system allows robots to interpret more naturally and improve over time through interactions with humans. By adopting a human-centered development approach, we tailor our system to address the actual needs of healthcare stakeholders.

Infusing Machine Learning And Computational Linguistics Into Clinical Notes, Funke V. Alabi, Onyeka Omose, Omotomilola Jegede

Mathematics & Statistics Faculty Publications

Entering free-form text notes into Electronic Health Records (EHR) systems takes a lot of time from clinicians. A large portion of this paper work is viewed as a burden, which cuts into the amount of time doctors spend with patients and increases the risk of burnout. We will see how machine learning and computational linguistics can be infused in the processing of taking clinical notes. We are presenting a new language modeling task that predicts the content of notes conditioned on historical data from a patient's medical record, such as patient demographics, lab results, medications, and previous notes, with the …

Robustsentembed: Robust Sentence Embeddings Using Adversarial Self-Supervised Contrastive Learning, Javad Rafiei Asl, Prajwal Panzade, Eduardo Blanco, Daniel Takabi, Zhipeng Cai

School of Cybersecurity Faculty Publications

Pre-trained language models (PLMs) have consistently demonstrated outstanding performance across a diverse spectrum of natural language processing tasks. Nevertheless, despite their success with unseen data, current PLM-based representations often exhibit poor robustness in adversarial settings. In this paper, we introduce RobustSentEmbed, a self-supervised sentence embedding framework designed to improve both generalization and robustness in diverse text representation tasks and against a diverse set of adversarial attacks. Through the generation of high-risk adversarial perturbations and their utilization in a novel objective function, RobustSentEmbed adeptly learns high-quality and robust sentence embeddings. Our experiments confirm the superiority of RobustSentEmbed over state-of-the-art representations. Specifically, …

A Chinese Power Text Classification Algorithm Based On Deep Active Learning, Song Deng, Qianliang Li, Renjie Dai, Siming Wei, Di Wu, Yi He, Xindong Wu

Computer Science Faculty Publications

The construction of knowledge graph is beneficial for grid production, electrical safety protection, fault diagnosis and traceability in an observable and controllable way. Highly-precision text classification algorithm is crucial to build a professional knowledge graph in power system. Unfortunately, there are a large number of poorly described and specialized texts in the power business system, and the amount of data containing valid labels in these texts is low. This will bring great challenges to improve the precision of text classification models. To offset the gap, we propose a classification algorithm for Chinese text in the power system based on deep …

Can Large Language Models Discern Evidence For Scientific Hypotheses? Case Studies In The Social Sciences, Sai Koneru, Jian Wu, Sarah Rajtmajer

Computer Science Faculty Publications

Hypothesis formulation and testing are central to empirical research. A strong hypothesis is a best guess based on existing evidence and informed by a comprehensive view of relevant literature. However, with exponential increase in the number of scientific articles published annually, manual aggregation and synthesis of evidence related to a given hypothesis is a challenge. Our work explores the ability of current large language models (LLMs) to discern evidence in support or refute of specific hypotheses based on the text of scientific abstracts. We share a novel dataset for the task of scientific hypothesis evidencing using community-driven annotations of studies …

Forecasting Crude Oil Price Using Multiple Factors, Hind Aldabagh, Xianrong Zheng, Mohammad Najand, Ravi Mukkamala

Computer Science Faculty Publications

In this paper, we predict crude oil price using various factors that may influence its price. The factors considered are physical market, financial, and trading market factors, including seven key factors and the dollar index. Firstly, we select the main factors that may greatly influence the prices. Then, we develop a hybrid model based on a convolutional neural network (CNN) and long short-term memory (LSTM) network to predict the prices. Lastly, we compare the CNN–LSTM model with other models, namely gradient boosting (GB), decision trees (DTs), random forests (RFs), neural networks (NNs), CNN, LSTM, and bidirectional LSTM (Bi–LSTM). The empirical …

Assessing Concepts, Procedures, And Cognitive Demand Of Chatgpt-Generated Mathematical Tasks, Bima Sapkota, Liza Bondurant

School of Mathematical & Statistical Sciences Faculty Publications

In November 2022, ChatGPT, an Artificial Intelligence (AI) large language model (LLM) capable of generating human-like responses, was launched. ChatGPT has a variety of promising applications in education, such as using it as thought-partner in generating curricular resources. However, scholars also recognize that the use of ChatGPT raises concerns, such as outputs that are inaccurate, nonsensical, or vague. We, two mathematics teacher educators, engaged in a collaborative self-study using qualitative descriptive approaches to investigate the procedures, concepts, and cognitive demand of ChatGPT-generated mathematical tasks focused on fraction multiplication using the area model approach. We found that the ChatGPT-generated tasks were …

Exploring International Educators' Learning About Local And Global Social Justice In A Virtual Community Of Practice, Bima Sapkota, Xuwei Luo, Muna Sapkota, Murat Akarsu, Emmanuel Deogratias, Daphne Fauber, Rose Mbewe, Fidelis Mumba, Ram Krishna Panthi, Jill Newton

School of Mathematical & Statistical Sciences Faculty Publications

In this chapter, the authors report themes that emerged when a cross-cultural team of researchers involved in a virtual international community of practice (Global Social Justice in Education-GSJE) investigated reflections on activities focused on social justice in local and global contexts. The findings suggested that the activities elicited GSJE community members' understandings of the complexities of social justice associated with naming practices, privilege, and the arts within their own and across contexts. The authors discuss implications of the activities to advance diverse educators' understanding of social justice in global and local contexts. They also unpack the opportunities and challenges that …

Hack24f: Llms And World Knowledge Problems, Chengtaon Lin, Joseph Lobo, Huuthanhvy Nguyen, Brian Santos, Ting Xu

Paul English Applied Artificial Intelligence (AI) Institute Publications

While AI “hallucinations” have been widely publicized by the media as shortcomings of Large Language Models (LLMs), there is much less awareness about shortcomings that derive from the potential lack of understanding of the world, from a potential lack of commonsense knowledge. We want to raise awareness among our student body about the potential fallibility of LLMs when they the problems that they need to solve require a basic understanding of the world. We hope that this awareness will prompt everyone to question more carefully the outputs generated by LLMs.

Beyond Binary: Revealing Variations In Islamophobic Content With Hierarchical Multi-Class Classification, Esraa Aldreabi, Khawlah M. Harahsheh, Mukul Dev Chhangani, Chung-Hao Chen, Jeremy Blackburn

Electrical & Computer Engineering Faculty Publications

In the digital age, the rise of Islamophobia-marked by an irrational fear or discrimination against Islam and Muslims-has emerged as a pressing issue, especially on social media platforms. In this paper we employs a multi-class classification system, moving beyond traditional binary models. We categorize Islamophobic content into three main classes and various subclasses, covering a range from subtle biases to explicit incitement. Comparative analysis of data from Reddit and Twitter illuminates the distinct prevalence and types of Islamophobic content specific to each platform. This paper deepens our understanding of digital Islamophobia and provides insights for crafting targeted online counter strategies. …

Toward Inclusivity: Rethinking Islamophobic Content Classification In The Digital Age, Esraa Aldreabi, Mukul Dev Chhangani, Khawlah M. Harahsheh, Justin M. Lee, Chung-Hao Chen

Electrical & Computer Engineering Faculty Publications

In this paper, we implement a comprehensive three-class system to categorize social media discussions about Islam and Muslims, enhancing the typical binary approach. These classes are: I) General Discourse About Islam and Muslims, II) Criticism of Islamic Teachings and Figures, and III) Comments Against Muslims. These categories are designed to balance the nuances of free speech while protecting diverse groups like Muslims, ex-Muslims, LGBTQ+ communities, and atheists. By utilizing machine learning and employing transformer-based models, we analyze the distribution and characteristics of these classes in social media content. Our findings reveal distinct patterns of user engagement with topics related to …

Shedding Light On Software Engineering-Specific Metaphors And Idioms, Mia Mohammad Imran, Preetha Chatterjee, Kostadin Damevski

Computer Science Faculty Research & Creative Works

Use of figurative language, such as metaphors and idioms, is common in our daily-life communications, and it can also be found in Software Engineering (SE) channels, such as comments on GitHub. Automatically interpreting figurative language is a challenging task, even with modern Large Language Models (LLMs), as it often involves subtle nuances. This is particularly true in the SE domain, where figurative language is frequently used to convey technical concepts, often bearing developer affect (e.g., 'spaghetti code). Surprisingly, there is a lack of studies on how figurative language in SE communications impacts the performance of automatic tools that focus on …

Eye-Gaze Guided Multi-Modal Alignment For Medical Representation Learning, Chong Ma, Hanqi Jiang, Wenting Chen, Yiwei Li, Zihao Wu, Xiaowei Yu, Zhengliang Liu, Lei Guo, Dajiang Zhu, Tuo Zhang, Dinggang Shen, Tianming Liu, Xiang Li

Computer Science Faculty Research & Creative Works

In the medical multi-modal frameworks, the alignment of cross-modality features presents a significant challenge. However, existing works have learned features that are implicitly aligned from the data, without considering the explicit relationships in the medical context. This data-reliance may lead to low generalization of the learned alignment relationships. In this work, we propose the Eye-gaze Guided Multi-modal Alignment (EGMA) framework to harness eye-gaze data for better alignment of medical visual and textual features. We explore the natural auxiliary role of radiologists' eye-gaze data in aligning medical images and text and introduce a novel approach by using eye-gaze data, collected synchronously …

Invasive Wild Pig (Sus Scrofa) Diets On Barrier Islands In The Southeastern United States, Vienna R. Canright, Antoinette J. Piaggio, James C. Beasley

United States Department of Agriculture Wildlife Services: Staff Publications

Background

Biological invasions are a leading cause of reductions in global biodiversity. Islands are particularly sensitive to invasions, which often result in cascading impacts throughout island communities. Wild pigs (Sus scrofa) are globally invasive and pose threats to numerous taxa and ecosystems, particularly for islands where they have contributed to declines of many endemic species. However, the impacts of wild pig diet on the flora and fauna remain understudied in many island systems.

Results

We used DNA metabarcoding of wild pig fecal samples to quantify the seasonal diet composition of wild pigs on three barrier islands in the …

Low-Resource Automatic Speech Recognition Domain Adaptation – A Case-Study In Aviation Maintenance, Nadine Amin, Tracy L. Yother, Julia Rayz

Journal of Aviation/Aerospace Education & Research

With timeliness and efficiency being critical in the aviation maintenance industry, the need has been growing for smart technological solutions that optimize and streamline the different underlying tasks (Bergkvist & Sabbagh, 2021). One such task is the technical documentation of the performed maintenance operations (Chandola et al., 2022). Instead of manual documentation, voice tools that transcribe spoken logbook entries allow technicians to document their work right away in a hands-free and time efficient manner. However, an accurate automatic speech recognition (ASR) model requires large training corpora (Siyaev & Jo, 2021a), which are lacking in the domain of aviation maintenance. In …

Effect Of Specific Data Variations On Automated Speaker Recognition, Ethan David Meighen

Graduate Theses, Dissertations, and Problem Reports (ETD)

Speaker recognition is not a new biometric modality but there are still many obstacles in the way in order for it to become as used as fingerprint recognition, facial recognition, and iris recognition. Many real-world environmental conditions, hardware device variations, and human behavior present serious challenges to the use of opportunistic voice or speaker samples for identification purposes. Non-idealities, identified as nuisance factors, include environmental noise, input device quality, length of utterance, sample rate variation, and unscripted data are common nuisance factors that can impact speaker recognition match score performance. The impact of the nuisance factors listed above were evaluated …

Assessing The Impact Of Chatbot-Human Personality Congruence On User Behavior: A Chatbot-Based Advising System Case, Mohammad Amin Kuhail, Mohamed Bahja, Ons Al-Shamaileh, Justin Thomas, Amina Alkazemi, Joao Negreiros

All Works

Chatbot personality has been demonstrated to influence user behavior, such as trust and intended engagement. However, previous research on chatbot-user personality congruence’s influence on user behavior is scant despite its significance in human-human conversations. This study explores the effect of chatbot-human personality trait congruence on user behavior in the context of a chatbot-based advising system. In this study, 54 college students interacted with chatbots with three different personalities (extraversion, agreeableness, and conscientiousness) and rated their trust, usage intention, and intended engagement with the chatbots. Additionally, 18 participants were interviewed to gain further insights into their perceptions and evaluations of the …

Digraph Enabled Digital Twin And Label-Encoding Machine Learning For Scada Network's Cyber Attack Analysis In Industry 5.0, Nabeel Al-Qirim, Anoud Bani-Hani, Munir Majdalawieh, Hussam Al Hamadi, Mohammad Kamrul Hasan

All Works

False-Data Injection Attack (FDIA), Remote-Tripping Command Injection (RTCI), and System Reconfiguration Attack (SRA) on SCADA (Supervisory Control and Data Acquisition) networks impact industry 5.0 enabled smart grid components such as intelligent-electronic-device (IED), circuit-breaker, network-switch, and power transmission lines. Since the SCADA-network-based cyber-attacking flow is not in digital-twin form, it is impossible to simulate the effects of the attack. Furthermore, the string nature of these affected components' data makes it challenging to incorporate into machine-learning-enabled intelligence (CTI) processes. To visualize the attacking flow of FDIA, RTCI, and SRA cyber-attacks on SCADA networks, this paper presents a novel "Digital Twin and Machine …

The Analyst’S Hierarchy Of Needs: Grounded Design Principles For Tailored Intelligence Analysis Tools, Antonio E. Girona, James C. Peter, Wenyuan Wang, R. Jordan Crouser

Computer Science: Faculty Publications

Intelligence analysis involves gathering, analyzing, and interpreting vast amounts of information from diverse sources to generate accurate and timely insights. Tailored tools hold great promise in providing individualized support, enhancing efficiency, and facilitating the identification of crucial intelligence gaps and trends where traditional tools fail. The effectiveness of tailored tools depends on an analyst’s unique needs and motivations, as well as the broader context in which they operate. This paper describes a series of focus discovery exercises that revealed a distinct hierarchy of needs for intelligence analysts. This reflection on the balance between competing needs is of particular value in …

Word Prediction Using Dynamic Skip Connections Along With Arabert And Lstm In Arabic Language, Ahad Almalki, Faris Kateb, Rayan Mosli

ASEAN Journal on Science and Technology for Development

Natural Language Generation (NLG) plays a crucial role in modern digital tools, including chatbots, virtual support, content suggestions, and tailored marketing, making bots more responsive and reducing the need for human staff. While there's much research on NLG for languages like English, languages like Arabic, Urdu, and Chinese still face challenges. This study examines Arabic NLG's unique aspects, dialects, and word variations. With around 420 million Arabic speakers globally, it's crucial to advance NLG for this language. We compared three models: Long Short-Term Memory (LSTM), a mix of Bidirectional Encoder Representations from Transformers (BERT) and LSTM, and a version that …

The Conceptual Review On The Effect Of Corporate Governance Monitoring Mechanisms On Tax Avoidance, Eveana Mosuin, Nor Balkish Zakaria, Yvonne Joseph Ason

ASEAN Journal on Science and Technology for Development

Taxes are the primary source of revenue for many nations in order to increase budget revenues and fund national development. Taxation serves multiple purposes because it can positively impact a nation's investment, education, social and economic development. However, tax authorities require assistance with the issue of tax non-compliance, which impedes tax administration and collection. One of the categories of tax non-compliance is tax avoidance, which is one of the company's strategies for legally reducing its tax burden by exploiting loopholes in tax regulations to minimise tax liability. Tax avoidance is when a company follows a particular tax strategy in the …

The Effect Of Nanomorphology As Quantified Via The K-Index On The Drug Delivery Properties Of Isocyanate-Derived Aerogels, Stephen Y. Owusu, A. B.M.Shaheen Ud Doulah, Vaibhav A. Edlabadkar, Kamden J. George, Chariklia Sotiriou-Leventis

Chemistry Faculty Research & Creative Works

This study explores the potential to predict the drug-loading and release profiles of aerogels based on their morphologies: a milestone in drug delivery research, which can help save time and cost in formulating new aerogel drug carriers and cut-down evaluation of the drug delivery capabilities of aerogels to a few experimental runs. Polyurea (PUA) and poly(isocyanurate-urethane) (PIR-PUR) aerogels were used as model systems, while 5-fluorouracil (5-FU) and paracetamol (PM) were used as model drugs. These model systems were chosen because they can be synthesized into different morphologies, which can be quantified by the so-called K-index (water contact angle divided by …

Cyberbullying Text Identification: A Deep Learning And Transformer-Based Language Modeling Approach, Khalid Saifullah, Muhammad Ibrahim Khan, Suhaima Jamal, Iqbal H. Sarker

Research outputs 2022 to 2026

In the contemporary digital age, social media platforms like Facebook, Twitter, and YouTube serve as vital channels for individuals to express ideas and connect with others. Despite fostering increased connectivity, these platforms have inadvertently given rise to negative behaviors, particularly cyberbullying. While extensive research has been conducted on high-resource languages such as English, there is a notable scarcity of resources for low-resource languages like Bengali, Arabic, Tamil, etc., particularly in terms of language modeling. This study addresses this gap by developing a cyberbullying text identification system called BullyFilterNeT tailored for social media texts, considering Bengali as a test case. The …

A Comparison Of Opennmt Sequence Model For Indonesian Automatic Question Generation, Yuniar Indrihapsari, Handaru Jati, N. Nurkhamid, Ratna Wardani, Pradana Setialana, Muhamad Izzudin Mahali, Danang Wijaya, Dhista Dwi Nur Ardiansyah, Satya Adhiyaksa Ardy, Maria Bernadetha Charlotta Wonda Tiala, Andi Hakim Al-Khawarizmi, Widya Ardiyanto

Elinvo (Electronics, Informatics, and Vocational Education)

Evaluation of learners is a crucial aspect of the educational system. However, creating evaluation instruments is a process that demands teachers' time and energy. The researcher developed the Indonesia Automatic Question Generator in this study using an architecture modified from past studies. The primary goals of this project are (1) to construct an AQG tool utilizing the OpenNMT series and (2) to analyze and compare the model's performance. As a data source, this study employs the SQuAD 2.0 dataset and numerous sequence techniques, including BiGRU, BiLSTM, and Transformer. The researcher trained the models using OpenNMT-py and Google Collaboratory. This approach …

Intelligent Security System In A Campus Building Using Rfid To Improve Security For Elevator Users, D. Desmira, Mustofa Abi Hamid, Lilis Noviawati, Tb. Ai Munandar, M. Martias, Didik Aribowo

Elinvo (Electronics, Informatics, and Vocational Education)

This study aims to develop an intelligent system of elevator access with RFID (Radio Frequency Identification) using fuzzy logic as an intelligent system, a safety controller of elevator doors that will open when the power goes out. This research made an Arduino Uno-based elevator access system with an RFID card using fuzzy logic as an intelligence system and markers for each floor. Each floor has an RFID card that contains the information data of each floor. This research uses research and development (R&D) methods with a waterfall development model. The research procedure consists of several stages, namely the requirement analysis …

Development Of An Android-Based Cultural Heritage Map App, Muhammad Irfan Luthfi, Ratna Wardani, Nisa Dwi Septiyanti

Elinvo (Electronics, Informatics, and Vocational Education)

This study aims to (1) develop an Android-based cultural heritage map app that can provide the location of cultural heritages in the Special Region in Yogyakarta along with its descriptions, images, and videos; and (2) understand the quality of the app based on standard software quality testing ISO / IEC 25010 on functional suitability, performance efficiency, and compatibility. The development method utilized a waterfall software development model that consists of communication, planning, modeling, construction, and deployment. The results of this study are 1) an Android-based cultural heritage map app that can provide the location of cultural heritages in the Special …

Cross-Language Tourism News Retrieval System Using Google Translate Api On Sebi Search Engine, H. Husni, Arif Muntasa, Sigit Susanto Putro, Zulfi Osman

Elinvo (Electronics, Informatics, and Vocational Education)

Cross-Language Information Retrieval (CLIR) is responsible for retrieving information stored in a language different from the language of the query provided by the user. Some translation methods commonly used in CLIR are Dictionary, Parallel corpora, Comparable corpora, Machine translator, Ontology, and Transitive-based. The query must be translated to the target language, followed by preprocessing and calculating the similarity between the query and all documents in the corpus. The problem is the time and accuracy of query translation. Moreover, the queries are not written as complete sentences according to certain language rules. Stemming, for example, every language has its own method. …

Design And Development Of Industrial Practice Monitoring And Assessment Systems Using Tsukamoto Fuzzy Logic, Tri Yuli Pahtoni, Fatchul Arifin

Elinvo (Electronics, Informatics, and Vocational Education)

Vocational high schools are given flexibility for their students to carry out direct learning in the industry as part of the practical education activities of implementing student skills. The implementation of industrial practice requires a special way to find out and monitor each student's activities so that the achievements of the implementation of industrial practice can be carried out properly. The implementation of industrial work practice assessment has several assessment criteria. These criteria include attendance, neatness, attitude, skills, and knowledge. The problems found in the assessment system are still done manually so that the effectiveness is minimal. This study aims …

Outsourcing Voting To Ai: Can Chatgpt Advise Index Funds On Proxy Voting Decisions?, Chen Wang

Fordham Journal of Corporate & Financial Law

Released in November 2022, Chat Generative Pre-training Transformer (“ChatGPT”), has risen rapidly to prominence, and its versatile capabilities have already been shown in a variety of fields. Due to ChatGPT’s advanced features, such as extensive pre-training on diverse data, strong generalization ability, fine-tuning capabilities, and improved reasoning, the use of AI in the legal industry could experience a significant transformation. Since small passive funds with low-cost business models generally lack the financial resources to make informed proxy voting decisions that align with their shareholders’ interests, this Article considers the use of ChatGPT to assist small investment funds, particularly small passive …