Digital Commons Network^™

Laf: Labeling-Free Model Selection For Automated Deep Neural Network Reusing, Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Mike Papadakis, Yves Le Traon

Research Collection School Of Computing and Information Systems

pplying deep learning (DL) to science is a new trend in recent years, which leads DL engineering to become an important problem. Although training data preparation, model architecture design, and model training are the normal processes to build DL models, all of them are complex and costly. Therefore, reusing the open-sourced pre-trained model is a practical way to bypass this hurdle for developers. Given a specific task, developers can collect massive pre-trained deep neural networks from public sources for reusing. However, testing the performance (e.g., accuracy and robustness) of multiple deep neural networks (DNNs) and recommending which model should be …

Pro-Cap: Leveraging A Frozen Vision-Language Model For Hateful Meme Detection, Rui Cao, Ming Shan Hee, Adriel Kuek, Wen Haw Chong, Roy Ka-Wei Lee, Jing Jiang

Research Collection School Of Computing and Information Systems

Hateful meme detection is a challenging multimodal task that requires comprehension of both vision and language, as well as cross-modal interactions. Recent studies have tried to fine-tune pre-trained vision-language models (PVLMs) for this task. However, with increasing model sizes, it becomes important to leverage powerful PVLMs more efficiently, rather than simply fine-tuning them. Recently, researchers have attempted to convert meme images into textual captions and prompt language models for predictions. This approach has shown good performance but suffers from non-informative image captions. Considering the two factors mentioned above, we propose a probing-based captioning approach to leverage PVLMs in a zero-shot …

Matk: The Meme Analytical Tool Kit, Ming Shan Hee, Aditi Kumaresan, Nguyen Khoi Hoang, Nirmalendu Prakash, Rui Cao, Roy Ka-Wei Lee

Research Collection School Of Computing and Information Systems

The rise of social media platforms has brought about a new digital culture called memes. Memes, which combine visuals and text, can strongly influence public opinions on social and cultural issues. As a result, people have become interested in categorizing memes, leading to the development of various datasets and multimodal models that show promising results in this field. However, there is currently a lack of a single library that allows for the reproduction, evaluation, and comparison of these models using fair benchmarks and settings. To fill this gap, we introduce the Meme Analytical Tool Kit (MATK), an open-source toolkit specifically …

Revisiting Disentanglement And Fusion On Modality And Context In Conversational Multimodal Emotion Recognition, Bobo Li, Hao Fei, Lizi Liao, Yu Zhao, Chong Teng, Tat-Seng Chua, Donghong Ji, Fei Li

Research Collection School Of Computing and Information Systems

It has been a hot research topic to enable machines to understand human emotions in multimodal contexts under dialogue scenarios, which is tasked with multimodal emotion analysis in conversation (MM-ERC). MM-ERC has received consistent attention in recent years, where a diverse range of methods has been proposed for securing better task performance. Most existing works treat MM-ERC as a standard multimodal classification problem and perform multimodal feature disentanglement and fusion for maximizing feature utility. Yet after revisiting the characteristic of MM-ERC, we argue that both the feature multimodality and conversational contextualization should be properly modeled simultaneously during the feature disentanglement …

Partial Annotation-Based Video Moment Retrieval Via Iterative Learning, Wei Ji, Renjie Liang, Lizi Liao, Hao Fei, Fuli Feng

Research Collection School Of Computing and Information Systems

Given a descriptive language query, Video Moment Retrieval (VMR) aims to seek the corresponding semantic-consistent moment clip in the video, which is represented as a pair of the start and end timestamps. Although current methods have achieved satisfying performance, training these models heavily relies on the fully-annotated VMR datasets. Nonetheless, precise video temporal annotations are extremely labor-intensive and ambiguous due to the diverse preferences of different annotators.Although there are several works trying to explore weakly supervised VMR tasks with scattered annotated frames as labels, there is still much room to improve in terms of accuracy. Therefore, we design a new …

Heterogeneous Graph Neural Network With Multi-View Representation Learning, Zezhi Shao, Yongjun Xu, Wei Wei, Fei Wang, Zhao Zhang, Feida Zhu

Research Collection School Of Computing and Information Systems

In recent years, graph neural networks (GNNs)-based methods have been widely adopted for heterogeneous graph (HG) embedding, due to their power in effectively encoding rich information from a HG into the low-dimensional node embeddings. However, previous works usually easily fail to fully leverage the inherent heterogeneity and rich semantics contained in the complex local structures of HGs. On the one hand, most of the existing methods either inadequately model the local structure under specific semantics, or neglect the heterogeneity when aggregating information from the local structure. On the other hand, representations from multiple semantics are not comprehensively integrated to obtain …

Ppdf: A Privacy-Preserving Cloud-Based Data Distribution System With Filtering, Yudi Zhang, Willy Susilo, Fuchun Guo, Guomin Yang

Research Collection School Of Computing and Information Systems

Cloud computing has emerged as a popular choice for distributing data among both individuals and companies. Ciphertext-policy attribute-based encryption (CP-ABE) has been extensively used to provide data security and enable fine-grained access control. With this encryption technique, only users whose attributes satisfy the access policy can access the plaintext. In order to mitigate the computational overhead on users, particularly on lightweight devices, partial decryption has been introduced, where the cloud assists in performing the decryption computations without revealing sensitive information. However, in this process, the cloud obtains the user's attributes, thus infringing on the user's privacy. To address this issue, …

Spatial Data Management For Green Mobility, Christophe Claramunt, Christine Bassem, Demetrios Zeinalipour-Yazti, Baihua Zheng, Goce Trajcevski, Kristian Torp

Research Collection School Of Computing and Information Systems

While many countries are developing appropriate actions towards a greener future and moving towards adopting sustainable mobility activities, the real-time management and planning of innovative transportation facilities and services in urban environments still require the development of advanced mobile data management infrastructures. Novel green mobility solutions, such as electric, hybrid, solar and hydrogen vehicles, as well as public and gig-based transportation resources are very likely to reduce the carbon footprint. However, their successful implementation still needs efficient spatio-temporal data management resources and applications to provide a clear picture and demonstrate their effectiveness. This paper discusses the major data management challenges, …

Github Actions: The Impact On The Pull Request Process, Mairieli Wessel, Joseph Vargovich, Marco Gerosa, Christoph Treude

Research Collection School Of Computing and Information Systems

Software projects frequently use automation tools to perform repetitive activities in the distributed software development process. Recently, GitHub introduced GitHub Actions, a feature providing automated workflows for software projects. Understanding and anticipating the effects of adopting such technology is important for planning and management. Our research investigates how projects use GitHub Actions, what the developers discuss about them, and how project activity indicators change after their adoption. Our results indicate that 1,489 out of 5,000 most popular repositories (almost 30% of our sample) adopt GitHub Actions and that developers frequently ask for help implementing them. Our findings also suggest that …

Cgt-Gan: Clip-Guided Text Gan For Image Captioning, Jiarui Yu, Haoran Li, Yanbin Hao, Bin Zhu, Tong Xu, Xiangnan He

Research Collection School Of Computing and Information Systems

The large-scale visual-language pre-trained model, Contrastive Language-Image Pre-training (CLIP), has significantly improved image captioning for scenarios without human-annotated image-caption pairs. Recent advanced CLIP-based image captioning without human annotations follows a text-only training paradigm, i.e., reconstructing text from shared embedding space. Nevertheless, these approaches are limited by the training/inference gap or huge storage requirements for text embeddings. Given that it is trivial to obtain images in the real world, we propose CLIP-guided text GAN (CgT-GAN), which incorporates images into the training process to enable the model to "see" real visual modality. Particularly, we use adversarial training to teach CgT-GAN to mimic …

Metaformer Baselines For Vision, Weihao Yu, Chenyang Si, Pan Zhou, Mi Luo, Yichen Zhou, Jiashi Feng, Shuicheng Yan, Xinchao Wang

Research Collection School Of Computing and Information Systems

Abstract—MetaFormer, the abstracted architecture of Transformer, has been found to play a significant role in achieving competitive performance. In this paper, we further explore the capacity of MetaFormer, again, by migrating our focus away from the token mixer design: we introduce several baseline models under MetaFormer using the most basic or common mixers, and demonstrate their gratifying performance. We summarize our observations as follows: (1) MetaFormer ensures solid lower bound of performance. By merely adopting identity mapping as the token mixer, the MetaFormer model, termed IdentityFormer, achieves >80% accuracy on ImageNet-1K. (2) MetaFormer works well with arbitrary token mixers. When …

A Survey On Aspect-Based Sentiment Analysis: Tasks, Methods, And Challenges, Wenxuan Zhang, Xin Li, Yang Deng, Lidong Bing, Wai Lam

Research Collection School Of Computing and Information Systems

As an important fine-grained sentiment analysis problem, aspect-based sentiment analysis (ABSA), aiming to analyze and understand people's opinions at the aspect level, has been attracting considerable interest in the last decade. To handle ABSA in different scenarios, various tasks are introduced for analyzing different sentiment elements and their relations, including the aspect term, aspect category, opinion term, and sentiment polarity. Unlike early ABSA works focusing on a single sentiment element, many compound ABSA tasks involving multiple elements have been studied in recent years for capturing more complete aspect-level sentiment information. However, a systematic review of various ABSA tasks and their …

Leveraging Long Short-Term User Preference In Conversational Recommendation Via Multi-Agent Reinforcement Learning, Yang Deng, Yaliang Li, Bolin Ding, Wai Lam

Research Collection School Of Computing and Information Systems

Conversational recommender systems (CRS) endow traditional recommender systems with the capability of dynamically obtaining users’ short-term preferences for items and attributes through interactive dialogues. There are three core challenges for CRS, including the intelligent decisions for what attributes to ask, which items to recommend, and when to askor recommend, at each conversation turn. Previous methods mainly leverage reinforcement learning (RL) to learn conversational recommendation policies for solving one or two of these three decision-making problems in CRS with separated conversation and recommendation components. These approaches restrict the scalability and generality of CRS and fall short of preserving a stable training …

Rethinking Conversational Agents In The Era Of Large Language Models: Proactivity, Non-Collaborativity, And Beyond, Yang Deng, Wenqiang Lei, Minlie Huang, Tat-Seng Chua

Research Collection School Of Computing and Information Systems

Conversational systems are designed to offer human users social support or functional services through natural language interactions. Typical conversation researches mainly focus on the response-ability of the system, such as dialogue context understanding and response generation. In the era of large language models (LLMs), LLM-augmented conversational systems showcase exceptional capabilities of responding to user queries for different language tasks. However, as LLMs are trained to follow users' instructions, LLM-augmented conversational systems typically overlook the design of an essential property in intelligent conversations, i.e., goal awareness. In this tutorial, we will introduce the recent advances on the design of agent's awareness …

Multimodal Fashion Knowledge Extraction As Captioning, Yifei Yuan, Wenxuan Zhang, Yang Deng, Wai Lam

Research Collection School Of Computing and Information Systems

Social media plays a significant role in boosting the fashion industry, where a massive amount of fashion-related posts are generated every day. In order to obtain the rich fashion information from the posts, we study the task of social media fashion knowledge extraction. Fashion knowledge, which typically consists of the occasion, person attributes, and fashion item information, can be effectively represented as a set of tuples. Most previous studies on fashion knowledge extraction are based on the fashion product images without considering the rich text information in social media posts. Existing work on fashion knowledge extraction in social media is …

Osc-Co2: Coattention And Cosegmentation Framework For Plant State Change With Multiple Features, Rubi Quiñones, Ashok Samal, Sruti Das Choudhury, Francisco Muñoz-Arriola

School of Computing: Faculty Publications

Cosegmentation and coattention are extensions of traditional segmentation methods aimed at detecting a common object (or objects) in a group of images. Current cosegmentation and coattention methods are ineffective for objects, such as plants, that change their morphological state while being captured in different modalities and views. The Object State Change using Coattention-Cosegmentation (OSC-CO²) is an end-to-end unsupervised deep-learning framework that enhances traditional segmentation techniques, processing, analyzing, selecting, and combining suitable segmentation results that may contain most of our target object’s pixels, and then displaying a final segmented image. The framework leverages coattention-based convolutional neural networks (CNNs) and …

Depaul Digest

DePaul Magazine

College of Education Professor Jason Goulah fosters hope, happiness and global citizenship through DePaul’s Institute for Daisaku Ikeda Studies in Education. Associate Journalism Professor Jill Hopke shares how to talk about climate change. News briefs from DePaul’s 10 colleges and schools: Occupational Therapy Standardized Patient Program, Financial Planning Certificate program, Business Education in Technology and Analytics Hub, Racial Justice Initiative, Teacher Quality Partnership grant, Intimate Partner Violence and Brain Injury collaboration, School of Music Career Closet, Sports Photojournalism course, DePaul Migration Collaborative’s Solutions Lab, Inclusive Screenwriting courses. New appointments: School of Music Dean John Milbauer, College of Education Dean Jennifer …

Short History Of The Unl Digital Commons, Paul Royster

University of Nebraska-Lincoln Libraries: Presentations

From 2005 through 2023, the UNL Digital Commons grew to be a leading example of an institutional repository. This presentation reports on personnel, history, strategy, and outstanding examples of series or contributors.

Announcement about the session:

UNL Digital Commons began in 2005 and grew into America’s 3rd-largest and most-trafficked institutional repository. Approaching 100 million downloads and spreading UNL scholarship and branding across the globe, the UNL Digital Commons boasts works from a wide variety of affiliated faculty, researchers, and students. Their participation is opening the dissemination of scholarship in radical and fundamental ways. In this session, Paul Royster traces this …

Demo-Abstract: A Dtn System For Tracking Miners Using Gae-Lstm And Contact Graph Routing In An Underground Mine, Abhay Goyal, Sanjay Kumar Madria, Samuel Frimpong

Computer Science Faculty Research & Creative Works

Localization and prediction of movement of miners in underground mines have been a constant problem more so during a mine disaster. Due to the unavailability of GPS signals, the pillars are used as a method to locate these miners, and thus, location prediction is also carried out with reference to these pillars. In this work, we demon- strate a Delay-tolerant Network (DTN) system called Miner-Finder that leverages Machine Learning (ML) framework (GAE-LSTM) that works on edge devices (e.g., mobile phones, tablets) to predict the location of miners in an underground mine. The information such as speed, angle, time, nearest pillar …

Ship Registry And Flag State Obligations For The Plurinational State Of Bolivia : A Case Study For A Landlocked State, Marco Antonio Lucano Uzquiano

World Maritime University Dissertations

No abstract provided.

Artst: Arabic Text And Speech Transformer, Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, Hanan Al Darmaki

Natural Language Processing Faculty Publications

We present ArTST, a pre-trained Arabic text and speech transformer for supporting open-source speech technologies for the Arabic language. The model architecture follows the unified-modal framework, SpeechT5, that was recently released for English, and is focused on Modern Standard Arabic (MSA), with plans to extend the model for dialectal and code-switched Arabic in future editions. We pre-trained the model from scratch on MSA speech and text data, and fine-tuned it for the following tasks: Automatic Speech Recognition (ASR), Text-To-Speech synthesis (TTS), and spoken dialect identification. In our experiments comparing ArTST with SpeechT5, as well as with previously reported results in …

A Psychometric Analysis Of Natural Language Inference Using Transformer Language Models, Antonio Laverghetta Jr.

USF Tampa Graduate Theses and Dissertations

Large language models (LLMs) are poised to transform both academia and industry. But the excitement around these generative AIs has also been met with concern for the true extent of their capabilities. This dissertation helps to address these questions by examining the capabilities of LLMs using the tools of psychometrics. We focus on analyzing the capabilities of LLMs on the task of natural language inference (NLI), a foundational benchmark often used to evaluate new models. We demonstrate that LLMs can reliably predict the psychometric properties of NLI items were those items administered to humans. Through a series of experiments, we …

Yet Another Model For Arabic Dialect Identification, Ajinkya Kulkarni, Hanan Al Darmaki

Natural Language Processing Faculty Publications

In this paper, we describe a spoken Arabic dialect identification (ADI) model for Arabic that consistently outperforms previously published results on two benchmark datasets: ADI-5 and ADI-17. We explore two architectural variations: ResNet and ECAPA-TDNN, coupled with two types of acoustic features: MFCCs and features exratected from the pre-trained self-supervised model UniSpeech-SAT Large, as well as a fusion of all four variants. We find that individually, ECAPA-TDNN network outperforms ResNet, and models with UniSpeech-SAT features outperform models with MFCCs by a large margin. Furthermore, a fusion of all four variants consistently outperforms individual models. Our best models outperform previously reported …

Improving Human-Automation Collaboration In Motion Planning, Torin J. Adamson

Computer Science ETDs

Human-automation collaboration is becoming a part of everyday life as AI helps us drive, make decisions, and solve a variety of other tasks. However, safe and effective collaboration systems depend on factors in trust, communication, and more. Existing studies to explore these are typically carried out in laboratory settings, providing robust data under tight environmental control. However, human behavior evolves over time, driven by external factors that cannot be fully captured in single participation sessions. These factors form the "human context", contextualizing the behavioral data for a more complete understanding. In this thesis, video game adaptations upon conventional subject studies …

Hprop: Hierarchical Privacy-Preserving Route Planning For Smart Cities, Francis Tiausas, Keiichi Yasumoto, Jose Paolo Talusan, Hayato Yamana, Hirozumi Yamaguchi, Shameek Bhattacharjee, Abhishek Dubey, Sajal K. Das

Computer Science Faculty Research & Creative Works

Route Planning Systems (RPS) are a core component of autonomous personal transport systems essential for safe and efficient navigation of dynamic urban environments with the support of edge-based smart city infrastructure, but they also raise concerns about user route privacy in the context of both privately owned and commercial vehicles. Numerous high-profile data breaches in recent years have fortunately motivated research on privacy preserving RPS, but most of them are rendered impractical by greatly increased communication and processing overhead. We address this by proposing an approach called Hierarchical Privacy-Preserving Route Planning (HPRoP), which divides and distributes the route-planning task across …

Dtitd: An Intelligent Insider Threat Detection Framework Based On Digital Twin And Self-Attention Based Deep Learning Models, Zhi Qiang Wang, Abdulmotaleb El Saddik

Computer Vision Faculty Publications

Recent statistics and studies show that the loss generated by insider threats is much higher than that generated by external attacks. More and more organizations are investing in or purchasing insider threat detection systems to prevent insider risks. However, the accurate and timely detection of insider threats faces significant challenges. In this study, we proposed an intelligent insider threat detection framework based on Digital Twins and self-attentions based deep learning models. First, this paper introduces insider threats and the challenges in detecting them. Then this paper presents recent related works on solving insider threat detection problems and their limitations. Next, …

Exploring Network Security Educator Knowledge, Jennifer B. Chauvot, Deniz Gurkan, Cathy Horn

Journal of Cybersecurity Education, Research and Practice

It is critical for nations to have trained professionals in network security who can safeguard hardware, information systems, and electronic data. Network security education is a key knowledge unit of the National Centers of Academic Excellence in Cybersecurity and various information systems security curricula at the master's and bachelor's levels in higher education. Network security units are components of computer science curricula in high school contexts as well. Educators who teach these concepts play a significant role in developing a skilled workforce of network security experts for both governmental and non-governmental organizations. Understanding the necessary knowledge and skills of network …

Cybersecurity Challenges And Awareness Of The Multi-Generational Learners In Nepal, Raj Kumar Dhungana, Lina Gurung Dr, Hem Poudyal

Journal of Cybersecurity Education, Research and Practice

Increased exposure to technologies has lately emerged as one of the everyday realities of digital natives, especially K-12 students, and teachers, the digital immigrants. Protection from cybersecurity risks in digital learning spaces is a human right, but students are increasingly exposed to high-risk cyberspace without time to cope with cybersecurity risks. This study, using a survey (N-891 students and 157 teachers) and in-depth interviews (27 students and 14 teachers), described the students' cybersecurity-related experiences and challenges in Nepal. This study revealed that the school’s cybersecurity support system is poor and teachers has very low awareness and competencies to protect students …

Adapting The Adapters For Code-Switching In Multilingual Asr, Atharva Kulkarni, Ajinkya Kulkarni, Miguel Couceiro, Hanan Al Darmaki

Natural Language Processing Faculty Publications

Recently, large pre-trained multilingual speech models have shown potential in scaling Automatic Speech Recognition (ASR) to many low-resource languages. Some of these models employ language adapters in their formulation, which helps to improve monolingual performance and avoids some of the drawbacks of multi-lingual modeling on resource-rich languages. However, this formulation restricts the usability of these models on code-switched speech, where two languages are mixed together in the same utterance. In this work, we propose ways to effectively fine-tune such models on code-switched speech, by assimilating information from both language adapters at each language adaptation point in the network. We also …

Editorial For "Automated Breast Density Assessment In Mri Using Deep Learning And Radiomics: Strategies For Reducing Inter-Observer Variability"., Pegah Khosravi

Publications and Research

No abstract provided.