Digital Commons Network^™

Unified Generative And Discriminative Training For Multi-Modal Large Language Models, Wei Chow, Juncheng Li, Kaihang Pan, Qifan Yu, Hao Fei, Zhiqi Ge, Shuai Yang, Siliang Teng, Hanwang Zhang, Qianru Sun

Research Collection School Of Computing and Information Systems

In recent times, Vision-Language Models (VLMs) have been trained under two predominant paradigms. Generative training has enabled Multimodal Large Language Models (MLLMs) to tackle various complex tasks, yet issues such as hallucinations and weak object discrimination persist. Discriminative training, exemplified by models like CLIP, excels in zero-shot image-text classification and retrieval, yet struggles with complex scenarios requiring fine-grained semantic differentiation. This paper addresses these challenges by proposing a unified approach that integrates the strengths of both paradigms. Considering interleaved image-text sequences as the general format of input samples, we introduce a structure-induced training strategy that imposes semantic relationships between input …

Generative Semi-Supervised Graph Anomaly Detection, Hezhe Qiao, Qingsong Wen, Xiaoli Li, Ee-Peng Lim, Guansong Pang

Research Collection School Of Computing and Information Systems

This work considers a practical semi-supervised graph anomaly detection (GAD) scenario, where part of the nodes in a graph are known to be normal, contrasting to the extensively explored unsupervised setting with a fully unlabeled graph. We reveal that having access to the normal nodes, even just a small percentage of normal nodes, helps enhance the detection performance of existing unsupervised GAD methods when they are adapted to the semi-supervised setting. However, their utilization of these normal nodes is limited. In this paper we propose a novel Generative GAD approach (namely GGAD) for the semi-supervised scenario to better exploit the …

Inverse Factorized Soft Q-Learning For Cooperative Multi-Agent Imitation Learning, The Viet Bui, Tien Mai, Thanh Nguyen

Research Collection School Of Computing and Information Systems

This paper concerns imitation learning (IL) in cooperative multi-agent systems.The learning problem under consideration poses several challenges, characterized by high-dimensional state and action spaces and intricate inter-agent dependencies. In a single-agent setting, IL was shown to be done efficiently via an inverse soft-Q learning process. However, extending this framework to a multi-agent context introduces the need to simultaneously learn both local value functions to capture local observations and individual actions, and a joint value function for exploiting centralized learning.In this work, we introduce a new multi-agent IL algorithm designed to address these challenges. Our approach enables thecentralized learning by leveraging …

A Community-Driven Framework To Evaluate Factors Influencing Nft Buyers, Yi Meng Lau, Ping Fan Ke

Research Collection School Of Computing and Information Systems

This study examines the pivotal role of community-driven social dynamics in shaping non-fungible tokens (NFT) buyers’ purchasing intentions and the increasing social significance of digital assets. Building on prior research in online word-of-mouth, we propose a comprehensive framework that integrates technological, personal, social, economic, and political factors influencing NFT purchases. The framework emphasizes the importance of service providers, personalization, perceived value, and associated risks, while also providing insights into how online word-of-mouth and community dynamics impact these aspects. As a preliminary exploration, this research lays the foundation for future empirical studies to validate the framework and assess its applicability across …

Trustworthy Web3 Domains: A Framework For Digital Identity Verification, Yi Meng Lau, Ping Fan Ke

Research Collection School Of Computing and Information Systems

As decentralized applications evolve, digital identities represented through Web3 domain names gained prominence. This study addresses the challenges of establishing trust in Web3 domain names. The decentralized nature of Web3 introduces complexities in verifying domain name authenticity, making them targets for malicious activities such as cybersquatting and phishing. We propose a comprehensive framework that enhances traditional identification, authentication, and authorization processes by incorporating technological and social trust elements. This framework enables organizations and users to systematically assess the trustworthiness of Web3 domain names, offering a structured approach to managing digital identities in decentralized environments.

Towards General Conceptual Model Editing Via Adversarial Representation Engineering, Yihao Zhang, Zeming Wei, Jun Sun, Meng Sun

Research Collection School Of Computing and Information Systems

Since the rapid development of Large Language Models (LLMs) has achieved remarkable success, understanding and rectifying their internal complex mechanisms has become an urgent issue. Recent research has attempted to interpret their behaviors through the lens of inner representation. However, developing practical and efficient methods for applying these representations for general and flexible model editing remains challenging. In this work, we explore how to leverage insights from representation engineering to guide the editing of LLMs by deploying a representation sensor as an editing oracle. We first identify the importance of a robust and reliable sensor during editing, then propose an …

Question-Attentive Review-Level Explanation For Neural Rating Regression, Trung Hoang Le, Hady Wirawan Lauw

Research Collection School Of Computing and Information Systems

Recommendation explanations help to improve their acceptance by end users. Explanations come in many different forms. One that is of interest here is presenting an existing review of the recommended item as the explanation. The challenge is in selecting a suitable review, which is customarily addressed by assessing the relative importance or “attention” of each review to the recommendation objective. Our focus is improving review-level explanation by leveraging additional information in the form of questions and answers (QA). The proposed framework employs QA in an attention mechanism that aligns reviews to various QAs of an item and assesses their contribution …

Custom Permission Misconfigurations In Android: A Large-Scale Security Analysis, Rui Li, Wenrui Diao, Debin Gao

Research Collection School Of Computing and Information Systems

Android’s popularity is due to its openness and vast app ecosystem. Global developers can use Android Studio and rich Android APIs to create their apps. Within this ecosystem, Android permissions play a crucial role in managing access to resources, with system permissions controlled by system apps and custom permissions declared by third-party apps. However, the security of custom permissions has not received enough attention from the mobile security community, resulting in a lack of thorough evaluation of security practices for app developers using custom permissions. This study systematically evaluated the misconfiguration of custom permissions by Android app developers. It is …

Replay-And-Forget-Free Graph Class-Incremental Learning: A Task Profiling And Prompting Approach, Chaoxi Niu, Guansong Pang, Ling Chen, Bing Liu

Research Collection School Of Computing and Information Systems

Class-incremental learning (CIL) aims to continually learn a sequence of tasks, with each task consisting of a set of unique classes. Graph CIL (GCIL) follows the same setting but needs to deal with graph tasks (e.g., node classification in a graph). The key characteristic of CIL lies in the absence of task identifiers (IDs) during inference, which causes a significant challenge in separating classes from different tasks (i.e., inter-task class separation). Being able to accurately predict the task IDs can help address this issue, but it is a challenging problem. In this paper, we show theoretically that accurate task ID …

Generative Artificial Intelligence In Business Higher Education: A Focus Group Study, Xuenan Huo, Keng Siau

Research Collection School Of Computing and Information Systems

This research investigates the opportunities and challenges of integrating generative artificial intelligence (GenAI) into business higher education, drawing insights from an asynchronous focus group research study with doctoral students who serve dual roles as both learners and educators. Key opportunities identified through thematic analysis include knowledge acquisition, intelligent co-ideation, supportive augmentation, and personalized learning. Challenges identified include AI trustworthiness, cognitive dependency, human value, policy and instruction, assessment integrity, and identity management. This study clarifies GenAI’s specific role in business education and provides practical insights for effectively integrating GenAI to enhance learning outcomes and address emerging challenges. An analysis theory on …

Meta-Entrepreneurship: An Analysis Theory On Integrating Generative Ai, Agentic Ai, And Metaverse For Entrepreneurship, Yingying Zhang, Keng Siau

Research Collection School Of Computing and Information Systems

Metaverse entrepreneurship has emerged as an innovative topic alongside the development of generative AI, agentic AI and metaverse. This study conceptualizes meta-entrepreneurship as a novel form of entrepreneurial activity that enables value creation within virtual and physical realms and proposes an analytical theoretical framework based on a systematic literature review, observations, and focus group study. Our framework is structured around three layers (infrastructure, content, and experience) and two domains (metaverse-based operational domain and AI-based production domain), aims to conceptualize “what is meta-entrepreneurship” and identify new possibilities. The research highlights the multifaceted impact of meta-entrepreneurship on individuals, corporations, industries, societies, and …

Shield-U: Safeguarding Traffic Sign Recognition Against Perturbation Attacks, Shengmin Xu, Jianfei Sun, Hangcheng Cao, Yulan Gao, Ziyang He, Cong Wu

Research Collection School Of Computing and Information Systems

Traffic sign recognition systems are crucial for the navigation and situation awareness of autonomous vehicles. They leverage deep learning technologies to swiftly and accurately identify traffic signs, even in the most challenging traffic environments. However, security researchers have uncovered a critical vulnerability in these systems: learning-based TSRs are particularly susceptible to physical-world perturbation attacks. Through subtle modifications (i.e., attaching well-designed patches on traffic signs), attackers can deceive the recognition system into making erroneous judgments, which can further lead to serious traffic accidents. Although several defense mechanisms have been proposed to enhance the security of sign recognition systems, these solutions generally …

Towards Privacy-Aware Iot Communications: Delegable, Revocable, And Efficient, Pengfei Wu, Jianfei Sun, Guomin Yang, Robert H. Deng

Research Collection School Of Computing and Information Systems

The Internet of Things (IoT) is widely recognized for its potential to enhance efficiency and productivity across various industries. However, its increasing prevalence has also made it a more attractive target for cybercriminals. While many advanced cryptographic solutions have been developed to secure IoT, some practical security and privacy issues such as self-sovereign delegation, flexible revocation, and lightweight access remain inadequately addressed in existing solutions. In this paper, we propose PLIC, a Privacy-aware Lightweight IoT Communication scheme, which not only enables any authorized user to flexibly delegate their lightweight access privileges to other delegatees, such that they can also access …

Ohss: Optimizing Homomorphic Secret Sharing To Support Fast Matrix Multiplication, Shuguang Zhang, Jianli Bai, Kun Tu, Ziyue Yin, Chan Liu

Research Collection School Of Computing and Information Systems

Homomorphic Secret Sharing (HSS) has evolved as a state-of-the-art methodology for achieving secure two-party computation, synthesizing the advantages of secret sharing and homomorphic encryption. This amalgamation ensures minimal computational and communicational overhead, making it particularly adept at arithmetic operations. However, HSS faces challenges in scalability and efficiency when confronted with extensive matrix operations, including both matrix-vector and matrix-matrix multiplications, which are fundamental in numerous privacy-preserving computations, notably within the realm of privacy-preserving machine learning. In this research, we introduce Optimized Homomorphic Secret Sharing (OHSS), a refined version of HSS, crafted to address these limitations. Our contributions include enhancements to the …

Gtree: Gpu-Friendly Privacy-Preserving Decision Tree Training And Inference, Qifan Wang, Shujie Cui, Lei Zhou, Ye Dong, Jianli Bai, Yun Sing Koh, Giovanni Russello

Research Collection School Of Computing and Information Systems

Outsourcing Decision tree (DT) training and inference to cloud platforms raises privacy concerns. Recent Secure Multi-Party Computation (MPC)-based methods are hindered by heavy overhead. Few recent studies explored GPUs to improve MPC-protected deep learning, yet integrating GPUs into MPC-protected DT with massive data-dependent operations remains challenging, raising question: can MPC-protected DT training and inference fully leverage GPUs for optimal performance?We present GTree, the first scheme that exploits GPU to accelerate MPC-protected secure DT training and inference. GTree is built across 3 parties who jointly perform DT training and inference with GPUs. GTree is secure against semi-honest adversaries, ensuring that no …

Lilac: Log Parsing Using Llms With Adaptive Parsing Cache, Zhihan Jiang, Jinyang Liu, Zhuangbin Chen, Yichen Li, Junjie Huang, Yintong Huo, Pinjia He, Jiazhen Gu, R. Michael Lyu

Research Collection School Of Computing and Information Systems

Log parsing transforms log messages into structured formats, serving as the prerequisite step for various log analysis tasks. Although a variety of log parsing approaches have been proposed, their performance on complicated log data remains compromised due to the use of human-crafted rules or learning-based models with limited training data. The recent emergence of powerful large language models (LLMs) demonstrates their vast pre-trained knowledge related to code and logging, making it promising to apply LLMs for log parsing. However, their lack of specialized log parsing capabilities currently hinders their parsing accuracy. Moreover, the inherent inconsistent answers, as well as the …

Agchain: A Blockchain-Based Gateway For Trustworthy App Delegation From Mobile App Markets, Mengjie Chen, Xiao Yi, Daoyuan Wu, Jianliang Xu, Yingjiu Li, Debin Gao

Research Collection School Of Computing and Information Systems

The popularity of smartphones has led to the growth of mobile app markets, creating a need for enhanced transparency, global access, and secure downloading. This paper introduces AGChain, a blockchain-based gateway that enables trustworthy app delegation within existing markets. AGChain ensures that markets can continue providing services while users benefit from permanent, distributed, and secure app delegation. During its development, we address two key challenges: significantly reducing smart contract gas costs and enabling fully distributed IPFS-based file storage. Additionally, we tackle three system issues related to security and sustainability. We have implemented a prototype of AGChain on Ethereum and Polygon …

Automating Maritime Risk Data Collection And Identification Leveraging Large Language Models, Donghao Huang, Xiuju Fu, Xiaofeng Yin, Haibo Pen, Zhaoxia Wang

Research Collection School Of Computing and Information Systems

Maritime risk research is crucial yet challenging for improving safety, efficiency, and sustainability in maritime operations. This paper presents an innovative method for automating the collection and identification of risk data related to global maritime risks from news sources, addressing the limitations of traditional manual methods. To evaluate the proposed method, different learning-based models, including conventional machine learning approaches and advanced Large Language Models (LLMs) such as GPT-4 and LLaMA-3.1, are comprehensively studied for comparison. In addition, not only do we use popular evaluation metrics to assess the proposed method, but we also introduce a new evaluation metric, called the …

Editorial For The Special Issue Of The Metaverse, Fiona Fui-Hoon Nah, Gert-Jan De Vreede, Lakshmi Goel, Eric Lim, Shu Schiller, Chee-Wee Tan

Research Collection School Of Computing and Information Systems

The metaverse is laying the groundwork for more accessible and immersive experiences by blending the physical and virtual worlds into a unified space where people can interact, create, and connect in entirely new ways. It holds the potential to revolutionize how we work, socialize, and learn, which in turn gives rise to unprecedented opportunities for innovation. In this special issue, we present four articles that depict the current state of research in metaverse, the key themes and theoretical underpinnings within this space, as well as emerging directions for future work. This special issue delivers valuable insights for both researchers and …

Virtual Conferencing Fatigue: Look‑Alike Avatar And Facial Attractiveness, Yuxin Liu, Keng Siau, Xueqing Wang, Yang Yang

Research Collection School Of Computing and Information Systems

The rapid evolution of avatar-related technologies provides extensive opportunities for diverse avatar applications in various areas. This study aims to investigate the innovative use of avatars to mitigate virtual conferencing fatigue, which refers to the physical and mental exhaustion from the inappropriate use of virtual conferencing applications. Grounded in Self-Awareness Theory, the research compares the impact of using real faces and user-look-alike avatars on virtual conferencing fatigue, delving into its underlying factors. In addition, the study examines the role of facial attractiveness enhancement on virtual conferencing fatigue. Laboratory experiments with a 2-by-2 between-subject design are employed to test hypotheses. The …

De-Identification Is Not Enough: A Comparison Between De-Identified And Synthetic Clinical Notes

Faculty, Staff and Student Publications

For sharing privacy-sensitive data, de-identification is commonly regarded as adequate for safeguarding privacy. Synthetic data is also being considered as a privacy-preserving alternative. Recent successes with numerical and tabular data generative models and the breakthroughs in large generative language models raise the question of whether synthetically generated clinical notes could be a viable alternative to real notes for research purposes. In this work, we demonstrated that (i) de-identification of real clinical notes does not protect records against a membership inference attack, (ii) proposed a novel approach to generate synthetic clinical notes using the current state-of-the-art large language models, (iii) evaluated …

Habit Coach: Customising Rag-Based Chatbots To Support Behavior Change, Arian Fooroogh Mand Arabi, Cansu Koyuturk, Michael O'Mahony, Raffaella Calati, Dimitri Ognibene

Conference papers

This paper presents the iterative development of Habit Coach, a GPT-based chatbot designed to support users in habit change through personalized interaction. Employing a user-centered design approach, we developed the chatbot using a Retrieval-Augmented Generation (RAG) system, which enables behavior personalization without retraining the underlying language model (GPT-4). The system leverages document retrieval and specialized prompts to tailor interactions, drawing from Cognitive Behavioral Therapy (CBT) and narrative therapy techniques. A key challenge in the development process was the difficulty of translating declarative knowledge into effective interaction behaviors. In the initial phase, the chatbot was provided with declarative knowledge about CBT …

Developing Linguistic Patterns To Mitigate Inherent Human Bias In Offensive Language Detection, Toygar Tanyel, Besher Alkurdi, Serkan Ayvaz

Turkish Journal of Electrical Engineering and Computer Sciences

With the proliferation of social media, there has been a sharp increase in offensive content, particularly targeting vulnerable groups, exacerbating social problems such as hatred, racism, and sexism. Detecting offensive language use is crucial to prevent offensive language from being widely shared on social media. However, the accurate detection of irony, implication, and various forms of hate speech on social media remains a challenge. Natural language-based deep learning models require extensive training with large, comprehensive, and labeled datasets. Unfortunately, manually creating such datasets is both costly and error-prone. Additionally, the presence of human-bias in offensive language datasets is a major …

A Surface-Based Approach For 3d Approximate Convex Decomposition, Onat Zeybek Kuşkonmaz, Yusuf Sahi̇lli̇oğlu

Turkish Journal of Electrical Engineering and Computer Sciences

Approximate convex decomposition enables the simplification of complex shapes into manageable convex components. In this work, we propose a novel surface-based method to achieve this which leads to efficient computation times and sufficiently convex results while avoiding over-approximating the input model. We start approximation using mesh simplification. Then we iterate over the surface polygons of the mesh and divide them into convex groups. We utilize planar and angular equations to determine suitable neighboring polygons for inclusion in forming convex groups. To ensure our method outputs a sufficient result for a wide range of input shapes, we run multiple iterations of …

A Systematic Review Of The Effects Of Ai-Assisted Moderation On Individuals And Groups, Zehui Yu, Lukas Otto, Dennis Assenmacher, Claudia Wagner

Human-Machine Communication

This review paper provides a conceptualization of AI-assisted content moderation with various degrees of autonomy and summarizes experimental evidence for how different levels of automation in content moderation and related losses of autonomy affect individuals and groups. Our results show that current research predominantly focuses on individuallevel effects, necessitating a shift toward understanding the impact on groups. The study highlights gaps in exploring different levels of AI-assisted moderation interventions and misalignments of different conceptualizations that make comparing research results difficult. The discussion underscores the prevailing emphasis on harmful content removal and advocates for investigating more constructive moderation techniques, emphasizing the …

Development Of A Web-Based Information System For Student Leave Permission At Dar Al-Raudhah Islamic Boarding School: Iso Quality Standards Analysis, Bonita Destiana, Priyanto Priyanto, Rahmatul Irfan, Muhammad Gus Khamim, Muhammad Yusuf Ridlo, Muhammad Iqbal

Elinvo (Electronics, Informatics, and Vocational Education)

Dar Al-Raudhah Entrepreneur, Islamic Boarding School, has adopted digital technology by upgrading hardware and software also investing in reliable internet infrastructure. However, this school still faces issues with students’ leave permission process due to reliance on manual bookkeeping and Excel, which leads to potential errors. Based on those problems, this research aims to create a web-based student leave permission system called SIPERSAN. The SIPERSAN system was developed with a Waterfall development model, which includes requirements analysis, design, implementation, testing, and deployment. The database is managed with MySQL, and the system is developed using PHP with the Laravel framework. Based on …

Relationship-Influenced Cyber Hygiene (Rich) In Community Banks, Monte L. Ward

USF Tampa Graduate Theses and Dissertations

This dissertation is a research study that introduces the theoretical model of Relationship-Influenced Cyber Hygiene (RICH) through its investigation of the phenomenon the researcher experienced: how the interpersonal relationship between top management and cybersecurity personnel within smaller community banks influences the cyber hygiene of top management. Due to the value proposition of community banks to their clients and the limited budget of smaller institutions for investment in cybersecurity controls and initiates, community banks need additional strategies to achieve stronger cyber hygiene, especially as it relates the greatest weakness in most cybersecurity infrastructures—the human element. This study identifies and addresses a …

Designing Customized Loss Functions For Training Deep Neural Networks, Ali Pourramezan Fard

Electronic Theses and Dissertations

This dissertation explores the critical role of loss functions in enhancing the predictive performance of deep machine learning models. Loss functions are an integral element of all the ongoing advances we witness daily in this domain. I design custom loss functions and their impacts on various machine learning tasks, particularly in computer vision.

In the first stage of my research, I aim to improve the prediction performance of deep learning models by providing them with more precise feedback associated with task requirements. This led me to create the concept of assistive loss functions. My first proposed loss function, inspired by …

An Overview Of Generative Ai Initiatives At Minnesota State University, Mankato (So Far), Evan Rusch, Nat Gustafson-Sundell

Library Services Publications

At Minnesota State University, Mankato, we’ve undertaken several experiments and initiatives focused on Generative Artificial Intelligence. We provided several examples at the Generative AI in Libraries (GAIL) conference. For this presentation, we provided a revised and expanded overview of our initiatives for the Northern Ohio Technical Services Librarians (NOTSL) Fall General Meeting. We explained license-related restrictions on uses of AI. We discussed the limitations of the retrieval-augmented generation tools currently available in the library. We summarized how we’ve tested ChatBots to support licensing and we showed how we’ve tried to use AI to improve data visualization for collections outreach. We …

Exploring Customer Behavior Through Emotion Detection, Darlington Omoifo

Master's Theses

This thesis investigates the potential of emotion detection, gaze tracking, and hand movement analysis as tools to enhance understanding of customer behavior in retail environments. In retail, subtle indicators such as facial expressions, gaze direction, and hand gestures provide valuable insights into customer preferences and decision-making processes. The study leverages real-time video processing with a multithreaded approach on a Raspberry Pi-powered system embedded in a gift cart equipped with dual cameras. Using FER for emotion detection and a custom model for gaze and hand tracking, this research identifies customer engagement patterns, focusing on how emotions like happiness correlate with purchasing …