Artificial Intelligence and Robotics Commons^™

Pfedrag: A Personalized Federated Retrieval-Augmented Generation System With Depth-Adaptive Tiered Embedding Tuning, Hangyu He, Xin Yuan, Kai Wu, Ren Ping Liu, Wei Ni

Research outputs 2022 to 2026

Large Language Models (LLMs) can undergo hallucinations in specialized domains, and standard Retrieval-Augmented Generation (RAG) often falters due to general-purpose embeddings ill-suited for domain-specific terminology. Though domain-specific fine-tuning enhances retrieval, centralizing data introduces privacy risks. The use of federated learning (FL) can alleviate this to some extent, but faces challenges of data heterogeneity, poor personalization, and expensive training data generation. We propose pFedRAG, a novel Personalized Federated RAG framework, which enables efficient collaborative fine-tuning of embedding models to address these challenges. The key contribution is a new Depth-Adaptive Tiered Embedding (DATE) architecture, which comprises a Global Shared Layer, combined using …

Toward Embodied Navigation Through Vision And Language, Muraleekrishna Gopinathan

Theses: Doctorates and Masters

Embodied AI is a challenging but exciting field in which a robot learns to interact with human-living spaces to perform various tasks. This thesis studies the embodied navigation problem in which a robotic agent navigates in a previously unseen indoor environment based on a challenging task. In particular, the Vision-and-Language Navigation (VLN) task requires a robot to navigate based on a descriptive human-language instruction. This thesis aims to improve VLN agents on four key aspects - their understanding of the environment, training via additional data, correcting navigational errors, and predicting the layout of the environment for better planning.

First, we …

Gans And Synthetic Financial Data: Calculating Var*, David E. Allen, Leonard Mushunje, Shelton Peiris

Research outputs 2022 to 2026

Generative Adversarial Neural nets (GANs) are a new branch of machine learning techniques. A GAN learns to generate new data from the training data set. We examine the characteristics of the fake financial data using GANs trained on samples of daily S&P 500 and FTSE 100 index values. GANs feature two competing neural networks in a game theoretic context. The Generator net generates pseudo data that is presented to the discriminator net which then attempts to distinguish between the real and the fake data. This facilitates unsupervised learning on the dataset. The generative network generates data sets, while the discriminative …

On-Device Recommender Systems: A Comprehensive Survey, Hongzhi Yin, Liang Qu, Tong Chen, Wei Yuan, Ruiqi Zheng, Jing Long, Xin Xia, Yuhui Shi, Chengqi Zhang

Research outputs 2022 to 2026

Recommender systems have been widely deployed in various real-world applications to help users identify content of interest from massive amounts of information. Traditional recommender systems work by collecting user-item interaction data in a cloud-based data center and training a centralized model to perform the recommendation service. However, such cloud-based recommender systems (CloudRSs) inevitably suffer from excessive resource consumption, response latency, as well as privacy and security risks concerning both data and models. Recently, driven by the advances in storage, communication, and computation capabilities of edge devices, there has been a shift of focus from CloudRSs to on-device recommender systems (DeviceRSs), …

Novel Digital Twin Deployment Approaches: Local And Distributed Digital Twin, Shahid Rauf, Fazal Muhammad, Akhtar Badshah, Hisham Alasmary, Muhammad Waqas, Sheng Chen

Research outputs 2022 to 2026

The Digital Twin (DT) technology is considered as a backbone in the Industrial 4.0 revolution as it is playing a vital role in the digitization of various industries. A DT is a virtual representation of a physical entity, thus having the ability to simulate real data generated at physical space to optimize, estimate, control, monitor and forecast states/configurations. Despite enormous benefits, DT technology has several implementation challenges. Although deploying DT on edge or cloud platforms yields a plethora of services, its implementation in both spaces faces certain limitations. These limitations include latency, data communication overload, transmission energy consumption, privacy concerns, …

Panoscu: A Simulation-Based Dataset For Panoramic Indoor Scene Understanding, Mariia Khan, Yue Qiu, Yuren Cong, Jumana Abu-Khalaf, David Suter, Bodo Rosenhahn

Research outputs 2022 to 2026

Panoramic images offer a comprehensive spatial view that is crucial for indoor robotics tasks such as visual room rearrangement, where an agent must restore objects to their original positions or states. Unlike existing 2D scene change understanding datasets, which rely on single-view images, panoramic views capture richer spatial context, object relationships, and occlusions—making them better suited for embodied artificial intelligence (AI) applications. To address this, we introduce Panoramic Scene Change Understanding (PanoSCU), a dataset specifically designed to enhance the visual object rearrangement task. Our dataset comprises 5,300 panoramas generated in an embodied simulator, encompassing 48 common indoor object classes. PanoSCU …

A Confidence-Based Knowledge Integration Framework For Cross-Domain Table Question Answering, Yuankai Fan, Tonghui Ren, Can Huang, Beini Zheng, Yinan Jing, Zhenying He, Jinbao Li, Jianxin Li

Research outputs 2022 to 2026

Recent advancements in TableQA leverage sequence-to-sequence (Seq2seq) deep learning models to accurately respond to natural language queries. These models achieve this by converting the queries into SQL queries, using information drawn from one or more tables. However, Seq2seq models often produce uncertain (low-confidence) predictions when distributing probability mass across multiple outputs during a decoding step, frequently yielding translation errors. To tackle this problem, we present CKIF, a confidence-based knowledge integration framework that uses a two-stage deep-learning-based ranking technique to mitigate the low-confidence problem commonly associated with Seq2seq models for TableQA. The core idea of CKIF is to introduce a flexible …

Llm Potentiality And Awareness: A Position Paper From The Perspective Of Trustworthy And Responsible Ai Modeling, Iqbal H. Sarker

Research outputs 2022 to 2026

Large language models (LLMs) are an exciting breakthrough in the rapidly growing field of artificial intelligence (AI), offering unparalleled potential in a variety of application domains such as finance, business, healthcare, cybersecurity, and so on. However, concerns regarding their trustworthiness and ethical implications have become increasingly prominent as these models are considered black-box and continue to progress. This position paper explores the potentiality of LLM from diverse perspectives as well as the associated risk factors with awareness. Towards this, we highlight not only the technical challenges but also the ethical implications and societal impacts associated with LLM deployment emphasizing fairness, …

Exploring Post-Covid-19 Health Effects And Features With Advanced Machine Learning Techniques, Muhammad N. Islam, Md S. Islam, Nahid H. Shourav, Iftiaqur Rahman, Faiz A. Faisal, Md M. Islam, Iqbal H. Sarker

Research outputs 2022 to 2026

COVID-19 is an infectious respiratory disease that has had a significant impact, resulting in a range of outcomes including recovery, continued health issues, and the loss of life. Among those who have recovered, many experience negative health effects, particularly influenced by demographic factors such as gender and age, as well as physiological and neurological factors like sleep patterns, emotional states, anxiety, and memory. This research aims to explore various health factors affecting different demographic profiles and establish significant correlations among physiological and neurological factors in the post-COVID-19 state. To achieve these objectives, we have identified the post-COVID-19 health factors and …

Phypo: Priority-Based Hybrid Task Partitioning And Offloading In Mobile Computing Using Automated Machine Learning, Shehr Bano, Ghulam Abbas, Muhammad Bilal, Ziaul Haq Abbas, Zaiwar Ali, Muhammad Waqas

Research outputs 2022 to 2026

With the increasing demand for mobile computing, the requirement for intelligent resource management has also increased. Cloud computing lessens the energy consumption of user equipment, but it increases the latency of the system. Whereas edge computing reduces the latency along with the energy consumption, it has limited resources and cannot process bigger tasks. To resolve these issues, a Priority-based Hybrid task Partitioning and Offloading (PHyPO) scheme is introduced in this paper, which prioritizes the tasks with high time sensitivity and offloads them intelligently. It also calculates the optimal number of partitions a task can be divided into. The utility of …

Collectively Advancing Deep Learning For Animal Detection In Drone Imagery: Successes, Challenges, And Research Gaps, Daniel Axford, Ferdous Sohel, Mathew A. Vanderklift, Amanda J. Hodgson

Research outputs 2022 to 2026

Drones have emerged as a powerful tool in animal detection, significantly advancing wildlife monitoring, conservation, and management by capturing high-resolution, real-time imagery over areas often inaccessible or challenging for human observers to reach. However, manual analysis of drone imagery for animal detection is labour-intensive and time-consuming. The application of deep learning methods, particularly convolutional neural networks, in automating animal detection from drone imagery has the potential to revolutionise wildlife monitoring, conservation, and management protocols. This review provides a comprehensive overview of the increasing use and prospects of deep learning in animal detection using drone imagery. It explores successful applications of …

Fostering Trust Through User Interface Design In Multi-Drone Search And Rescue, Johanna Ahlskog, Maria Theresa Bahodi, Artur Lugmayr, Timothy Merritt

Research outputs 2022 to 2026

Unmanned Aerial Vehicles (UAVs), or drones, are increasingly used in search and rescue (SAR) missions, with pilots transitioning from manual control of single drones to more collaborative tasks orchestrating semi-autonomous fleets. Designing user interfaces to support UAV pilots effectively is crucial to improving the success of search missions. We developed two versions of a multi-drone SAR system prototype to simulate SAR missions and evaluated them with professional UAV SAR pilots in Sweden. Both versions showed the flight paths of the UAVs, yet in one version, a heatmap was overlayed to provide information from a lost person model. We evaluated situational …

Explainable Ai For Cybersecurity Automation, Intelligence And Trustworthiness In Digital Twin: Methods, Taxonomy, Challenges And Prospects, Iqbal H. Sarker, Helge Janicke, Ahmad Mohsin, Asif Gill, Leandros Maglaras

Research outputs 2022 to 2026

Digital twins (DTs) are an emerging digitalization technology with a huge impact on today's innovations in both industry and research. DTs can significantly enhance our society and quality of life through the virtualization of a real-world physical system, providing greater insights about their operations and assets, as well as enhancing their resilience through real-time monitoring and proactive maintenance. DTs also pose significant security risks, as intellectual property is encoded and more accessible, as well as their continued synchronization to their physical counterparts. The rapid proliferation and dynamism of cyber threats in today's digital environments motivate the development of automated and …

Do Realistic Avatars Make Virtual Reality Better? Examining Human-Like Avatars For Vr Social Interactions, Alan D. Fraser, Isabella Branson, Ross C. Hollett, Craig P. Speelman, Shane L. Rogers

Research outputs 2022 to 2026

No abstract provided.

Label-Free Surface-Enhanced Raman Spectroscopy Coupled With Machine Learning Algorithms In Pathogenic Microbial Identification: Current Trends, Challenges, And Perspectives, Jia Wei Tang, Quan Yuan, Xin Ru Wen, Muhammad Usman, Alfred Chin Yen Tay, Liang Wang

Research outputs 2022 to 2026

Infectious diseases caused by microbial pathogens remain a primary contributor to global health burdens. Prompt control and effective prevention of these pathogens are critical for public health and medical diagnostics. Conventional microbial detection methods suffer from high complexity, low sensitivity, and poor selectivity. Therefore, developing rapid and reliable methods for microbial pathogen detection has become imperative. Surface-enhanced Raman Spectroscopy (SERS), as an innovative non-invasive diagnostic technique, holds significant promise in pathogenic microorganism detection due to its rapid, reliable, and cost-effective advantages. This review comprehensively outlines the fundamental theories of Raman Spectroscopy (RS) with a focus on label-free SERS strategy, reporting …

Unveiling The Dynamics Of Ai Applications: A Review Of Reviews Using Scientometrics And Bertopic Modeling, Raghu Raman, Debidutta Pattnaik, Laurie Hughes, Prema Nedungadi

Research outputs 2022 to 2026

In a world that has rapidly transformed through the advent of artificial intelligence (AI), our systematic review, guided by the PRISMA protocol, investigates a decade of AI research, revealing insights into its evolution and impact. Our study, examining 3,767 articles, has drawn considerable attention, as evidenced by an impressive 63,577 citations, underscoring the scholarly community's profound engagement. Our study reveals a collaborative landscape with 18,189 contributing authors, reflecting a robust network of researchers advancing AI and machine learning applications. Review categories focus on systematic reviews and bibliometric analyses, indicating an increasing emphasis on comprehensive literature synthesis and quantitative analysis. The …

Application Of Multilayer Perceptron Artificial Neural Network (Mlp-Ann) Algorithm For Pm2.5 Mass Concentration Estimation During Open Biomass Burning Episodes In Thailand, P. Paluang, W. Thavorntam, W. Phairuang

Research outputs 2022 to 2026

Open biomass burning (OBB) is the main cause of air pollution in Northern Thailand, where PM2.5 concentrations exceed Thailand's air quality standards annually during the January–April (dry season). The air emissions from databases that detail the pollutants discharged into the atmosphere from specific sources of air pollution are crucial for monitoring air pollution. However, this data has been poorly studied in Thailand. This study estimated ground-level PM2.5 concentration in Northern Thailand using the Multilayer Perceptron Artificial Neural Networks (MLP-ANN) model, integrating the in-depth data as input variables. The 10-fold cross-validation approach was applied to validate the model's performance. The meteorological …

Multi-Aspect Rule-Based Ai: Methods, Taxonomy, Challenges And Directions Towards Automation, Intelligence And Transparent Cybersecurity Modeling For Critical Infrastructures, Iqbal H. Sarker, Helge Janicke, Mohamed A. Ferrag, Alsharif Abuadbba

Research outputs 2022 to 2026

Critical infrastructure (CI) typically refers to the essential physical and virtual systems, assets, and services that are vital for the functioning and well-being of a society, economy, or nation. However, the rapid proliferation and dynamism of today's cyber threats in digital environments may disrupt CI functionalities, which would have a debilitating impact on public safety, economic stability, and national security. This has led to much interest in effective cybersecurity solutions regarding automation and intelligent decision-making, where AI-based modeling is potentially significant. In this paper, we take into account “Rule-based AI” rather than other black-box solutions since model transparency, i.e., human …

Voice Synthesis Improvement By Machine Learning Of Natural Prosody, Joseph Kane, Michael N. Johnstone, Patryk Szewczyk

Research outputs 2022 to 2026

Since the advent of modern computing, researchers have striven to make the human–computer interface (HCI) as seamless as possible. Progress has been made on various fronts, e.g., the desktop metaphor (interface design) and natural language processing (input). One area receiving attention recently is voice activation and its corollary, computer-generated speech. Despite decades of research and development, most computer-generated voices remain easily identifiable as non-human. Prosody in speech has two primary components—intonation and rhythm—both often lacking in computer-generated voices. This research aims to enhance computer-generated text-to-speech algorithms by incorporating melodic and prosodic elements of human speech. This study explores a novel …

Scene Graph Generation: A Comprehensive Survey, Hongsheng Li, Guangming Zhu, Liang Zhang, Youliang Jiang, Yixuan Dang, Haoran Hou, Peiyi Shen, Xia Zhao, Syed A. A. Shah, Mohammed Bennamoun

Research outputs 2022 to 2026

Deep learning techniques have led to remarkable breakthroughs in the field of object detection and have spawned a lot of scene-understanding tasks in recent years. Scene graph has been the focus of research because of its powerful semantic representation and applications to scene understanding. Scene Graph Generation (SGG) refers to the task of automatically mapping an image or a video into a semantic structural scene graph, which requires the correct labeling of detected objects and their relationships. In this paper, a comprehensive survey of recent achievements is provided. This survey attempts to connect and systematize the existing visual relationship detection …

Spatially-Aware Speaker For Vision-And-Language Navigation Instruction Generation, Muraleekrishna Gopinathan, Martin Masek, Jumana Abu-Khalaf, David Suter

Research outputs 2022 to 2026

Embodied AI aims to develop robots that can understand and execute human language instructions, as well as communicate in natural languages. On this front, we study the task of generating highly detailed navigational instructions for the embodied robots to follow. Although recent studies have demonstrated significant leaps in the generation of step-by-step instructions from sequences of images, the generated instructions lack variety in terms of their referral to objects and landmarks. Existing speaker models learn strategies to evade the evaluation metrics and obtain higher scores even for low-quality sentences. In this work, we propose SAS (Spatially-Aware Speaker), an instruction generator …

Enhancing Water Safety: Exploring Recent Technological Approaches For Drowning Detection, Salman Jalalifar, Andrew Belford, Eila Erfani, Amir Razmjou, Rouzbeh Abbassi, Masoud Mohseni-Dargah, Mohsen Asadnia

Research outputs 2022 to 2026

Drowning poses a significant threat, resulting in unexpected injuries and fatalities. To promote water sports activities, it is crucial to develop surveillance systems that enhance safety around pools and waterways. This paper presents an overview of recent advancements in drowning detection, with a specific focus on image processing and sensor-based methods. Furthermore, the potential of artificial intelligence (AI), machine learning algorithms (MLAs), and robotics technology in this field is explored. The review examines the technological challenges, benefits, and drawbacks associated with these approaches. The findings reveal that image processing and sensor-based technologies are the most effective approaches for drowning detection …

Malware Detection With Artificial Intelligence: A Systematic Literature Review, Matthew G. Gaber, Mohiuddin Ahmed, Helge Janicke

Research outputs 2022 to 2026

In this survey, we review the key developments in the field of malware detection using AI and analyze core challenges. We systematically survey state-of-the-art methods across five critical aspects of building an accurate and robust AI-powered malware-detection model: malware sophistication, analysis techniques, malware repositories, feature selection, and machine learning vs. deep learning. The effectiveness of an AI model is dependent on the quality of the features it is trained with. In turn, the quality and authenticity of these features is dependent on the quality of the dataset and the suitability of the analysis tool. Static analysis is fast but is …

Cyberbullying Text Identification: A Deep Learning And Transformer-Based Language Modeling Approach, Khalid Saifullah, Muhammad Ibrahim Khan, Suhaima Jamal, Iqbal H. Sarker

Research outputs 2022 to 2026

In the contemporary digital age, social media platforms like Facebook, Twitter, and YouTube serve as vital channels for individuals to express ideas and connect with others. Despite fostering increased connectivity, these platforms have inadvertently given rise to negative behaviors, particularly cyberbullying. While extensive research has been conducted on high-resource languages such as English, there is a notable scarcity of resources for low-resource languages like Bengali, Arabic, Tamil, etc., particularly in terms of language modeling. This study addresses this gap by developing a cyberbullying text identification system called BullyFilterNeT tailored for social media texts, considering Bengali as a test case. The …

Infrared Ship Segmentation Based On Weakly-Supervised And Semi-Supervised Learning, Isa Ali Ibrahim, Abdallah Namoun, Sami Ullah, Hisham Alasmary, Muhammad Waqas, Iftekhar Ahmad

Research outputs 2022 to 2026

Existing fully-supervised semantic segmentation methods have achieved good performance. However, they all rely on high-quality pixel-level labels. To minimize the annotation costs, weakly-supervised methods or semi-supervised methods are proposed. When such methods are applied to the infrared ship image segmentation, inaccurate object localization occurs, leading to poor segmentation results. In this paper, we propose an infrared ship segmentation (ISS) method based on weakly-supervised and semi-supervised learning, aiming to improve the performance of ISS by combining the advantages of two learning methods. It uses only image-level labels and a minimal number of pixel-level labels to segment different classes of infrared ships. …

Ai-Analyst: An Ai-Assisted Sdlc Analysis Framework For Business Cost Optimization, Nuruzzaman Faruqui, Priyabrata Thatoi, Rohit Choudhary, Ivana Roncevic, Hamed Alqahtani, Iqbal H. Sarker, Shapla Khanam

Research outputs 2022 to 2026

Managing the System Development Lifecycle (SDLC) is a complex task because of its involvement in coordinating diverse activities, stakeholders, and resources while ensuring project goals are met efficiently. The complex nature of the SDLC process leaves plenty of scope for human error, which impacts the overall business cost. This paper introduces AI-Analyst, an AI-assisted framework developed using the transformer-based model with more than 150 million parameters to assist with SDLC management. It minimizes manual effort errors, optimizes resource allocation, and improves decision-making processes, resulting in substantial cost savings. The statistical analysis shows that it saves around 53.33% of costs in …

A Technical Perspective On Integrating Artificial Intelligence To Solid-State Welding, Sambath Yaknesh, Natarajan Rajamurugu, Prakash K. Babu, Saravanakumar Subramaniyan, Sher A. Khan, C. Ahamed Saleel, Mohammad Nur-E-Alam, Manzoore E. M. Soudagar

Research outputs 2022 to 2026

The implementation of artificial intelligence (AI) techniques in industrial applications, especially solid-state welding (SSW), has transformed modeling, optimization, forecasting, and controlling sophisticated systems. SSW is a better method for joining due to the least melting of material thus maintaining Nugget region integrity. This study investigates thoroughly how AI-based predictions have impacted SSW by looking at methods like Artificial Neural Networks (ANN), Fuzzy Logic (FL), Machine Learning (ML), Meta-Heuristic Algorithms, and Hybrid Methods (HM) as applied to Friction Stir Welding (FSW), Ultrasonic Welding (UW), and Diffusion Bonding (DB). Studies on Diffusion Bonding reveal that ANN and Generic Algorithms can predict outcomes …

A Systematic Collection Of Medical Image Datasets For Deep Learning, Johann Li, Guangming Zhu, Cong Hua, Mingtao Feng, Basheer Bennamoun, Ping Li, Xiaoyuan Lu, Juan Song, Peiyi Shen, Xu Xu, Lin Mei, Liang Zhang, Syed A. A. Shah, Mohammed Bennamoun

Research outputs 2022 to 2026

The astounding success made by artificial intelligence in healthcare and other fields proves that it can achieve human-like performance. However, success always comes with challenges. Deep learning algorithms are data dependent and require large datasets for training. Many junior researchers face a lack of data for a variety of reasons. Medical image acquisition, annotation, and analysis are costly, and their usage is constrained by ethical restrictions. They also require several other resources, such as professional equipment and expertise. That makes it difficult for novice and non-medical researchers to have access to medical data. Thus, as comprehensively as possible, this article …

Artificial Intelligence In Cardiology: An Australian Perspective, Biyanka Jaltotage, Abdul R. Ihdayhid, Nick S. R. Lan, Faraz Pathan, Sanjay Patel, Clare Arnott, Gemma Figtree, Leonard Kritharides, Syed M. S. Islam, Clara K. Chow, James M. Rankin, Stephen J. Nicholls, Girish Dwivedi

Research outputs 2022 to 2026

Significant advances have been made in artificial intelligence technology in recent years. Many health care applications have been investigated to assist clinicians and the technology is close to being integrated into routine clinical practice. The high prevalence of cardiac disease in Australia places overwhelming demands on the existing health care system, challenging its capacity to provide quality patient care. Artificial intelligence has emerged as a promising solution. This discussion paper provides an Australian perspective on the current state of artificial intelligence in cardiology, including the benefits and challenges of implementation. This paper highlights some current artificial intelligence applications in cardiology, …

Artificial Intelligence And Precision Health Through Lenses Of Ethics And Social Determinants Of Health: Protocol For A State-Of-The-Art Literature Review, Sarah Wamala-Andersson, Matt X. Richardson, Sara Landerdahl Stridsberg, Jillian Ryan, Felix Sukums, Yong-Shian Goh

Research outputs 2022 to 2026

Background: Precision health is a rapidly developing field, largely driven by the development of artificial intelligence (AI)–related solutions. AI facilitates complex analysis of numerous health data risk assessment, early detection of disease, and initiation of timely preventative health interventions that can be highly tailored to the individual. Despite such promise, ethical concerns arising from the rapid development and use of AI-related technologies have led to development of national and international frameworks to address responsible use of AI. Objective: We aimed to address research gaps and provide new knowledge regarding (1) examples of existing AI applications and what role they play …