Computer Engineering Commons^™

Meme Sentiment Analysis Enhanced With Multimodal Spatial Encoding And Face Embedding, Muzhaffar Hazman, Susan Mckeever, Josephine Griffith

Conference papers

Internet memes are characterised by the interspersing of text amongst visual elements. State-of-the-art multimodal meme classifiers do not account for the relative positions of these elements across the two modalities, despite the latent meaning associated with where text and visual elements are placed. Against two meme sentiment classification datasets, we systematically show performance gains from incorporating the spatial position of visual objects, faces, and text clusters extracted from memes. In addition, we also present facial embedding as an impactful enhancement to image representation in a multimodal meme classifier. Finally, we show that incorporating this spatial information allows our fully automated …

A Tutoring Framework To Support Computer Science Programmes In Higher Education, Emer Thornbury, Frances Sheridan, Pramod Pathak, Cristina Hava Muntean, Paul Stynes

Conference papers

Computing Support is the provision of academic supports such as individual tutoring and support classes to students studying computing at third level. Students can struggle with computing as it requires practice involving trial and error. This work proposes a research informed tutoring framework to support computer science students at third level. The tutoring framework combines three pillars; staff and training, pedagogies and activities. Support is put in place to help students develop technical and programming skills. Essential tutoring is provided for those who might otherwise drop out of college. The framework was applied to first and second-year undergraduate programmes and …

A Real-Time Machine Learning Framework For Smart Home-Based Yoga Teaching System, Jothika Sunney, Musfira Jilani, Pramod Pathak, Paul Stynes

Conference papers

Practicing yoga poses in a home-based environment has increased due to Covid19. Yoga poses without a trainer can be challenging, and incorrect yoga poses can cause muscle damage. Smart home-based yoga teaching systems may aid in performing accurate yoga poses. However, the challenge with such systems is the computational time required to detect yoga poses. This research proposes a real-time machine learning framework for teaching accurate yoga poses. It combines a pose estimation model, a pose classification model, and a real-time feedback mechanism. The dataset consists of five popular yoga poses namely the downdog pose, the tree pose, the goddess …

Work In Progress: A Virtual Educational Robotics Coding Club Framework To Improve K-6 Students Emotional Engagement In Stem, Kate Carmody, Julie Booth, Jospehine Bleach, Pramod Pathak, Paul Styles

Conference papers

The growing popularity and deployment of Internet of Things (IoT) devices has led to serious security concerns. The integration of a security operations center (SOC) becomes increasingly important in this situation to ensure the security of IoT devices. In this article, we will present a summary of IoT device security issues, their vulnerabilities, a review of current challenges to keep these devices secure, and discuss the role that SOC can bring in protecting IoT devices while considering the challenges encountered and the directions to consider when implementing a reliable SOC for IoT monitoring.

Robustness Of Image-Based Malware Classification Models Trained With Generative Adversarial Networks, Ciaran Reilly, Stephen O Shaughnessy, Christina Thorpe

Conference papers

As malware continues to evolve, deep learning models are increasingly used for malware detection and classification, including image based classification. However, adversarial attacks can be used to perturb images so as to evade detection by these models. This study investigates the effectiveness of training deep learning models with Generative Adversarial Network-generated data to improve their robustness against such attacks. Two image conversion methods, byte plot and space-filling curves, were used to represent the malware samples, and a ResNet-50 architecture was used to train models on the image datasets. The models were then tested against a projected gradient descent attack. It …

Queer In Ai: A Case Study In Community-Led Participatory Ai, Anaelia Ovalle, Arjun Subramonian, Ashwiin Singh, Claas Voelcker, Danica Sutherland, Davide Locatelli, Eva Breznik, Felip Klubicka, Hang Yuan, Hetvi J, Huan Zhang, Jaidev Shriram, Kruno Lehman, Luca Soldaini, Maarten Sap, Marc Peter Deisenroth, Maria Leonor Pacheco, Maria Ryskina, Martin Mundt, Melind Agarwal, Nyx Mclean, Pan Xu, A. Pranav, Raj Korpan, Ruchira Ray, Sarah Mathew, Sarthak Arora, S.T. John, Tanvi Anand, Vishakha Agrawal, William Agnew, Yanan Long, Zijie J. Wang, Zeerak Talat, Avijit Ghosh, Nathaniel Dennler, Michael Noseworthy, Sharvani Jha, Emi Baylor, Aditya Joshi, Natalia Y. Bilenko, Andrew Mcnamara, Raphael Gontijo-Lopes, Alex Markham, Evyn Dong, Jackie Kay, Manu Saraswat, Nikhil Vytla, Luke Stark

Conference papers

Queerness and queer people face an uncertain future in the face of ever more widely deployed and invasive artificial intelligence (AI). These technologies have caused numerous harms to queer people, including privacy violations, censoring and downranking queer content, exposing queer people and spaces to harassment by making them hypervisible, deadnaming and outing queer people. More broadly, they have violated core tenets of queerness by classifying and controlling queer identities. In response to this, the queer community in AI has organized Queer in AI, a global, decentralized, volunteer-run grassroots organization that employs intersectional and community-led participatory design to build an inclusive …

Bilstm−Bigru: A Fusion Deep Neural Network For Predicting Air Pollutant Concentration, Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-Phelan

Conference papers

Predicting air pollutant concentrations is an efficient way to prevent incidents by providing early warnings of harmful air pollutants. A precise prediction of air pollutant concentrations is an important factor in controlling and preventing air pollution. In this paper, we develop a bidirectional long-short-term memory and a bidirectional gated recurrent unit (BiLSTM−BiGRU) to predict PM 2.5 concentrations in a target city for different lead times. The BiLSTM extracts preliminary features, and the BiGRU further extracts deep features from air pollutant and meteorological data. The fully connected (FC) layer receives the output and makes an accurate prediction of the PM 2.5 …

Graph-Based Mutations For Music Generation, Maziar Kanani, Sean O'Leary, James Mcdermott

Conference papers

Our study aims to compare the effects of direct mutation and graphbased mutation on representations of music domain. We focus on short tunes from the Irish folk tradition, represented as integer sequences, and use a graph-based representation based on Pathway Assembly (a directed acyclic graph) and the Sequitur algorithm. We define multiple mutation operators to work directly on the sequences or on the graphs, hypothesizing that graph-based mutations will tend to preserve the pattern used per tune, while direct mutation of sequences will tend to destroy patterns, resulting in new generated tunes that are more complex. We perform experiments on …

Combinedeepnet: A Deep Network For Multistep Prediction Of Near-Surface Pm2.5 Concentration, Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-Phelan

Conference papers

PM2.5 is a type of air pollutant that can cause respiratory and cardiovascular problems. Precise PM2.5 ( μg/m3 ) concentration prediction may help reduce health concerns and provide early warnings. To better understand air pollution, a number of approaches have been presented for predicting PM2.5 concentrations. Previous research used deep learning models for hourly predictions of air pollutants due to their success in pattern recognition, however, these models were unsuitable for multisite, long-term predictions, particularly in regard to the correlation between pollutants and meteorological data. This article proposes the combine deep network (CombineDeepNet), which combines multiple deep networks, including a …

Actor-Centric Spatio-Temporal Feature Extraction For Action Recognition, Anil Kunchala, Bianca Schoen-Phelan, Mélanie Bouroche

Conference papers

Action understanding involves the recognition and detection of specific actions within videos. This crucial task in computer vision gained significant attention due to its multitude of applications across various domains. The current action detection models, inspired by 2D object detection methods, employ two-stage architectures. The first stage is to extract actor-centric video sub-clips, i.e. tubelets of individuals, and the second stage is to classify these tubelets using action recognition networks. The majority of these recognition models utilize a frame-level pre-trained 3D Convolutional Neural Networks (3D CNN) to extract spatio-temporal features of a given tubelet. This, however, results in suboptimal spatio-temporal …

Efficient Gpu Implementation Of Automatic Differentiation For Computational Fluid Dynamics, Mohammad Zubair, Desh Ranjan, Aaron Walden, Gabriel Nastac, Eric Nielsen, Boris Diskin, Marc Paterno, Samuel Jung, Joshua Hoke Davis

Computer Science Faculty Publications

Many scientific and engineering applications require repeated calculations of derivatives of output functions with respect to input parameters. Automatic Differentiation (AD) is a method that automates derivative calculations and can significantly speed up code development. In Computational Fluid Dynamics (CFD), derivatives of flux functions with respect to state variables (Jacobian) are needed for efficient solutions of the nonlinear governing equations. AD of flux functions on graphics processing units (GPUs) is challenging as flux computations involve many intermediate variables that create high register pressure and require significant memory traffic because of the need to store the derivatives. This paper presents a …

A Structure-Aware Generative Adversarial Network For Bilingual Lexicon Induction, Bocheng Han, Qian Tao, Lusi Li, Zhihao Xiong

Computer Science Faculty Publications

Bilingual lexicon induction (BLI) is the task of inducing word translations with a learned mapping function that aligns monolingual word embedding spaces in two different languages. However, most previous methods treat word embeddings as isolated entities and fail to jointly consider both the intra-space and inter-space topological relations between words. This limitation makes it challenging to align words from embedding spaces with distinct topological structures, especially when the assumption of isomorphism may not hold. To this end, we propose a novel approach called the Structure-Aware Generative Adversarial Network (SA-GAN) model to explicitly capture multiple topological structure information to achieve accurate …

Advancing Vehicular Communication Systems: An Evolution From Dsrc To 5g Nr C-V2x Technology For Enhanced Safety, Reliability, And Efficiency In Intelligent Transportation Systems, Mehnaz Tabassum

Dissertations, Master's Theses and Master's Reports

This work focuses on the evolution of connected vehicles communication technologies and performance evaluation of vehicular communication systems, specifically in the context of Cellular Vehicle-to-Everything (C-V2X) technology and the Third Generation Partnership Project (3GPP) specifications. The dissertation also discusses the evolution of vehicle communication systems from Dedicated Short-Range Communication (DSRC) through 5G technologies. It examines the motivation for this shift, which are the growing demand for transportation safety, low latency, high data rate, low energy use, and seamless inter connectivity. The research delves into the greater capabilities and improved performance that 5G offers for direct V2V communications by analyzing the …

Memory Optimizations For High-Throughput Computer Systems, Zhiyuan Lu

Dissertations, Master's Theses and Master's Reports

The emergence of new non-volatile memory (NVM) technology and deep neural network (DNN) inferences bring challenges related to off-chip memory access. Ensuring crash consistency leads to additional memory operations and exposes memory update operations on the critical execution path. DNN inference execution on some accelerators suffers from intensive off-chip memory access. The focus of this dissertation is to tackle the issues related to off-chip memory in these high performance computing systems.

The logging operations, required by the crash consistency, impose a significant performance overhead due to the extra memory access. To mitigate the persistence time of log requests, we introduce …

Benchmarking Model Predictive Control And Reinforcement Learning For Legged Robot Locomotion, Shivayogi Akki

Dissertations, Master's Theses and Master's Reports

This research delves into the realm of quadrupedal robotics, focusing on the comparative analysis of Model Predictive Control (MPC) and Reinforcement Learning (RL) as predominant control strategies. Through the comprehensive dataset compiled and the insights derived from this analysis, this research aims to serve as a valuable resource for the legged robotics community, guiding researchers and practitioners in the selection and implementation of control strategies. The ultimate goal is to contribute to the advancement of legged robot capabilities and facilitate their successful deployment in real-world applications.

In this study, we employ the Unitree Go1 quadrupedal robot as a testbed, subjecting …