Computer Sciences Commons^™

Defense-To-Attack: Bypassing Weak Defenses Enables Stronger Jailbreaks In Vision-Language Models, Yunhan Zhao, Xiang Zheng, Yige Li, Xingjun Ma

Research Collection School Of Computing and Information Systems

Despite their superb capabilities, Vision-Language Models (VLMs) have been shown to be vulnerable to jailbreak attacks. While recent jailbreaks have achieved notable progress, their effectiveness and efficiency can still be improved. In this work, we reveal an interesting phenomenon: incorporating weak defense cues into the attack pipeline can significantly enhance both the effectiveness and efficiency of jailbreaks on VLMs. Building on this insight, we propose Defense2Attack, a novel jailbreak method that bypasses the safety guardrails of VLMs by leveraging defensive patterns to guide jailbreak prompt construction. Specifically, Defense2Attack consists of three key components: (1) a visual optimizer that embeds universal …

Semantic Shields: Automating Critical Infrastructure Defense Via Nlp-Driven Ransomware Profiling, Henry Trowbridge, Ian Zalcberg, Ryan Schley, Carter Yagemann, Natasha Phan, Srikar Maduposu, Vimal Buck

Military Cyber Affairs

Ransomware poses a growing threat to critical infrastructure, where successful attacks can disrupt operational technology (OT) and industrial control systems (ICS) with significant public safety consequences. However, attributing ransomware incidents to specific threat actors remains challenging due to ransomware-as-a-service ecosystems, actor rebranding, and the obfuscation of traditional indicators of compromise. This paper presents Semantic Shields, an NLP-driven attribution framework that leverages BERT-generated semantic embeddings and DBSCAN clustering to profile ransomware actors through the linguistic characteristics of ransom notes. Using a dataset of 295 ransom notes from 189 distinct threat groups, the framework achieved an 87.2% true positive clustering rate and …

Closing The Interpretability Gap: Explainable Ml-Based Malware Detection For Defensive Cyberspace Operations, Tashi Stirewalt, Sean Hodgson, Puumaaya Tahiru, Assefaw Gebremedhin

Military Cyber Affairs

This paper presents an end-to-end, explainable malware triage pipeline designed for defense-oriented cyber operations. It combines high-performance static detection methods with analyst-centered interpretability. Utilizing the EMBER 2024 Windows PE subset, we train and evaluate four classifiers and select LightGBM as the production model based on its predictive performance, inference efficiency, and compatibility with exact tree-based attribution. The deployed system consists of four sequential components: PE feature extraction, malware probability scoring, dual explainability (using SHAP and LIME), and large language model (LLM) report generation, all integrated within a Flask web interface. On a temporal test set of 1,080,000 samples, LightGBM achieves …

From Data To Decision-Making: The Role Of Local Digital Twins In Cross-Domain Management Within Municipalities – A Research-In-Progress Study In Veenendaal, Diana M.E. Boekman, Koen Smit, Guido Ongena, Rob Peters

Communications of the IIMA

Municipalities are facing increasingly complex, interconnected challenges in areas like housing, climate adaptation, mobility, and social policy. Local Digital Twins (LDTs) are seen as a promising tool to make this complexity more understandable and support decision-making. At the same time, both literature and practice show that few initiatives get past the pilot phase, even though getting through that phase is essential for successful long-term adoption.

This paper presents a research-in-progress study on the development and application of an implementation method for LDT technology within the municipality of Veenendaal, based on human values rather than driven by technological possibilities. Based on …

Learning 1-Bit Lidar-Based Localization With Auxiliary Objective, Kaijie Yin, Zhiyuan Zhang, Tian Gao, Wentao Zhu, Cheng-Zhong Xu, Hui Kong

Research Collection School Of Computing and Information Systems

6-DoF LiDAR-based localization is a fundamental capability for autonomous systems operating in large-scale outdoor environments. Many deep-learning-based localization methods have achieved promising performance so far. However, as one of the always-on modules competing for limited on-board computational resources, the localization module is expected to consume only a small portion of the overall compute budget. Most existing learning-based methods are still too heavy for this purpose. In contrast, binary neural networks (BNNs) offer an appealing solution, but the 1-bit compression causes severe information loss and performance drop. In this paper, we address this challenge by proposing Binarized LiDAR-based Localization (BiLoc), the …

Generalized Logit Adjustment: Improved Fine-Tuning By Mitigating Label Bias In Zero-Shot Vision Models, Beier Zhu, Qianru Sun, Xun Yang, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Foundation models like CLIP allow zero-shot transfer on various tasks without additional training data. Yet, the zero-shot performance is less competitive than a fully supervised one. Thus, fine-tuning and ensembling are also commonly adopted to better fit the downstream tasks. However, we argue that such prior work has overlooked the inherent biases in foundation models. Due to the highly imbalanced Web-scale training set, foundation models are inevitably skewed toward frequent semantics, and thus the subsequent fine-tuning or ensembling is still biased. In this study, we systematically examine the biases in foundation models and demonstrate the efficacy of our proposed Generalized …

Neural Symphony Of Flow Experience: Evidence For High-Dimensional Metastable Dynamics, Abdelrahman B. M. Eldaly, Kris Zhangguang Kang, Fiona Fui-Hoon Nah, Leanne Lai-Hang Chan, Keng Siau, Xiao Fan Liu, Richard Huskey, Langtao Chen, Tejaswini Yelamanchili, Rene Weber

Research Collection School Of Computing and Information Systems

Flow, an optimal experience characterized by deep immersion and engagement in an activity, has been extensively studied in behavioral research. However, its neural dynamic mechanism remains poorly understood. In a within-subject video gaming experiment, we captured neural activity underlying flow, boredom, and anxiety using a 64-channel electroencephalogram (EEG) system. Compared to boredom and anxiety, flow exhibits the highest global functional connectivity, metastability, and dimensionality of dynamic functional connectivity patterns, suggesting that flow is a highly adaptable process that is supported by high-dimensional neural dynamics. Unlike previous studies that focused on identifying static or localized brain activity, we examine the neural …

Restoring Linguistic Grounding In Vla Models Via Train-Free Attention Recalibration, Ninghao Zhang, Bin Zhu, Shijie Zhou, Jingjing Chen

Research Collection School Of Computing and Information Systems

Vision-Language-Action (VLA) models enable robots to perform manipulation tasks directly from natural language instructions and are increasingly viewed as a foundation for generalist robotic policies. However, their reliability under Out-Of-Distribution (OOD) instructions remains underexplored. In this paper, we reveal a critical failure mode in which VLA policies continue executing visually plausible actions even when the language instruction contradicts the scene. We refer to this phenomenon as linguistic blindness, where VLA policies prioritize visual priors over instruction semantics during action generation. To systematically analyze this issue, we introduce ICBench, a diagnostic benchmark constructed from the LIBERO dataset that probes language–action coupling …

Ai And The Environment: Solutions For Advancing Technology Safely, Kaitlynn Baker

Discovery Day - Daytona Beach

Since 2022, the world of Artificial Intelligence (AI) has boomed. AI went from a special and rare entity to a commonly used resource available to all through web sites, and phone apps. AI has benefitted everyday activities by making office, class, and personal tasks easier through grammar help, informational citations, and as someone to bounce ideas off of. Additionally, many companies have begun utilizing AI to improve customer service and experience, and train workers more efficiently, therefore, saving thousands of dollars. Despite the benefits humans reap from its use, AI has been harming our environment at growing rates. Data centers …

How Ai Influences The Design Process Of Unmanned Underwater Vehicles’ (Uuvs) 3d Sonar System, Eden Tsouklaris, Abriella Smith, Brianna Broderick, Carissa Aumack, Victoria Cornaro

Discovery Day - Daytona Beach

With the exponential growth of Artificial Intelligence (AI), user interface (UI) designers have explored using AI to shorten design time. This study assessed the effectiveness of UIs designed with AI programs versus manual methods for an Unmanned Underwater Vehicle (UUV) control system. Participants were tasked with designing an interface that would allow submarine operators to monitor and coordinate three UUVs repairing a severed underwater communication cable at a depth of 2,000 meters. The scenario presented several operational challenges (zero visibility, sonar-only perception, data latency, and potential system degradation), requiring participants' designs to maintain spatial awareness and support remote repair tasks. …

Real Bullets, Plastic Guns: Evaluating The Strength Of 3-D Printed Gun Parts, Maria Latenia Mayol

Student Theses

Privately made firearms (PMFs), often referred to as “ghost guns,” are firearms manufactured or assembled by individuals rather than federally licensed manufacturers. Although the terms are frequently used interchangeably, “ghost gun” more specifically describes an unserialized firearm, whereas PMFs include a broader range of firearms produced through nontraditional manufacturing methods. PMFs may be entirely 3-D printed, assembled from partially completed firearm kits, or constructed by integrating additively manufactured components with commercially manufactured firearm parts. The increasing accessibility of additive manufacturing and widespread dissemination of computer-aided design files have raised concerns about concealment, regulation, and forensic evasion, particularly when factory-manufactured components …

"The First Web Novel At 30: The Collection And The Creative Process", Robert Arellano, Scott Rettberg

ELO (un)supervised 2026

Summer 2026 marks the 30th anniversary of Sunshine '69, recognized as the first novelistic hypertext fiction published on the web. While the full work remains accessible online—an "(un)supervised" preservation achievement in itself—the archive remains split between boxes and memory. This conversation between the work's creator and a major scholar in electronic literature documents both specific preservation challenges and systemic patterns in what the field chooses to preserve.

Topics include: figuring out web-born composition before established methodologies existed; the three decades of technical decisions that kept a 1996 work alive through format obsolescence and server migrations; and what gets lost …

Research On Image Feature Analysis Of Intangible Cultural Heritage Brocade Integrating Multi-Scale Visual Perception, Ruiyang Yuan, Hao Wang, Shu Zhou, Hui Zhu, Jingwen Qiu

Journal of Scientific Information Research

[Purpose/significance] Addressing the challenges posed by the complex semantic characteristics of intangible cultural heritage brocade imagery, the difficulty in extracting their profound connotations, and the inadequate utilisation of multi-scale features by traditional deep learning models, this paper aims to explore a method for analysing the characteristics of intangible cultural heritage brocade images that integrates multi-scale visual features. [Method/process] This paper constructs a multi-scale feature analysis framework for intangible cultural heritage brocade images (ICH_BC), integrating convolutional neural networks with Transformer architectures. The framework employs ResNet to extract local texture and detail features from brocade images, utilises VIT to capture global structural …

Activity Transition Graph Generation: How Far Are We?, Jiakun Liu, Peixin Zhang, Han Hu, Yonghui Liu, Wei Minn, Ferdian Thung, Shahar Maoz, Eran Toch, Debin Gao, David Lo

Research Collection School Of Computing and Information Systems

Android applications (i.e., apps) are indispensable nowadays and are getting bigger and bigger with an increasing number offunctionalities. To understand how to access functionalities in an app, prior studies proposed tools to model the transitionsbetween functionalities with the activity transition graph (ATG). ATG is an important data structure and has been used forvarious Android app analyses, including app design, understanding, and testing. However, there is no benchmarking work onATG generation. It is still unclear whether the transitions identified by tools are correct and how many transitions are missed.To fill this gap, we manually identified all transitions in 98 applications to …

Chi Meta-Project Ecosystem Overview - Spring 2026, David B. Smith

Publications and Research

This paper offers a high-level account of the Center for Holistic Integration’s (CHI) meta-project ecosystem as visualized in the included system map. CHI provides an organizational structure framed around persistent meta-projects that support and extend individual initiatives across curriculum, scholarly and applied research, infrastructure, artistic production, AI development, cultural inquiry, and external partnerships. Rather than presenting the map as a static inventory of projects, the paper examines how its core domains function as living systems through which knowledge, tools, documentation, participants, and collaborations can accumulate over time. It also considers how CHI-mediated connectivity, institutional integration, and external funding allow the …

Tranx-Adapter: Bridging Artifacts And Semantics Within Mllms For Robust Ai-Generated Image Detection, Wenbin Wang, Yuge Huang, Jianqing Xu, Yue Yu, Jiangtao Yan, Shouhong Ding, Pan Zhou, Yong Luo

Research Collection School Of Computing and Information Systems

Rapid advances in AI-generated image (AIGI) technology enable highly realistic synthesis, threatening public information integrity and security. Recent studies have demonstrated that incorporating texture-level artifact features alongside semantic features into multimodal large language models (MLLMs) can enhance their AIGI detection capability. However, our preliminary analyses reveal that artifact features exhibit high intra-feature similarity, leading to an almost uniform attention map after the softmax operation. This phenomenon causes attention dilution, thereby hindering effective fusion between semantic and artifact features. To overcome this limitation, we propose a lightweight fusion adapter, TranX-Adapter, which integrates a Task-aware Optimal-Transport Fusion that leverages the Jensen-Shannon divergence …

Dual-Diffusional Generative Fashion Recommendation, Mingzhe Yu, Lei Wu, Qianru Sun, Yunshan Ma

Research Collection School Of Computing and Information Systems

Personalized generative recommender systems have emerged as a promising solution for fashion recommendation. However, existing methods primarily rely on implicit visual embeddings from historical interactions, which often contain preference-irrelevant information and result in insufficient user behavior modeling. Moreover, these models typically generate only item images, providing limited interpretability. To address these limitations, we propose DualFashion, a Dual-Diffusional Generative Fashion Recommendation Architecture that jointly models image and text modalities for personalized and explainable recommendation. DualFashion adopts a dual-diffusion Transformer with image and text branches, where structured attribute-level captions and visual outfit information are jointly used as conditioning signals to model user …

Oscbench: Benchmarking Object State Change In Text-To-Video Generation, Xianjing Han, Bin Zhu, Shiqi Hu, Franklin Mingzhe Li, Patrick Carrington, Roger Zimmermann, Jingjing Chen

Research Collection School Of Computing and Information Systems

Text-to-video (T2V) generation models have made rapid progress in producing visually high-quality and temporally coherent videos. However, existing benchmarks primarily focus on perceptual quality, text–video alignment, or physical plausibility, leaving a critical aspect of action understanding largely unexplored: object state change (OSC) explicitly specified in the text prompt. OSC refers to the transformation of an object’s state induced by an action, such as peeling a potato or slicing a lemon. In this paper, we introduce OSCBench, a benchmark specifically designed to assess OSC performance in T2V models. OSCBench is constructed from instructional cooking data and systematically organizes action–object interactions into …

Breadquest: Enhancing Roguelike Accessibility Through Procedural Generation And Thematic Design, Hahns Pena

Computer Science and Software Engineering

BreadQuest is a top-down roguelike dungeon crawler with a whimsical dessert theme that aims to make the genre more accessible while preserving strategic depth and replayability. Players explore procedurally generated dungeons, fight pastry-themed enemies, and collect bakery-inspired items that support a flavor-elemental combat system, with each run offering unique layouts, encounters, and rewards. Built in Unity with a modular, data-driven architecture, the game uses procedural generation techniques like Binary Space Partitioning, Voronoi diagrams, and Perlin noise to create varied and replayable levels. The project emphasizes approachable gameplay, cultural dessert inspiration, and replayability, with success evaluated through playtesting and player feedback.

Ai Interview Helper: A Tool For Assisting Search And Rescue Long-Profile Interviews, Dylan P. Starink

Master's Theses

In Search and Rescue (SAR) operations, time pressure and limited interviewer experience can lead to missed opportunities when interviewing a missing person’s friends and family. This thesis presents a real-time, end-to-end system that provides context-aware follow-up question suggestions as interviews unfold. Leveraging large language models (LLMs) and agentic design patterns, the system is intended to support interviewers by helping them identify relevant follow-up questions and pursue potentially overlooked lines of inquiry.

The system was evaluated through three mock interviews with two SAR interviewer participants across two events. Given the limited sample size, the results provide early insights into the feasibility …

Co-Designing With Autistic Livestreamers: Care, Constraints, And Trade-Offs In Livestreaming, Terrance Mok, Anthony Tang, Lora Oehlberg

Research Collection School Of Computing and Information Systems

Autistic livestreamers use platforms like Twitch for social connection, self-expression, and community, but these spaces also impose ongoing social and emotional demands. Prior work has documented these experiences, but less is known about what autistic creators themselves envision for the tools and platforms they use. We address this gap through a Research through Design (RtD) co-design study with three autistic Twitch streamers, using speculative artefacts as discussion prompts to explore how participants reasoned about potential livestreaming technologies. Across three co-design activities, we identify three overarching tensions shaping autistic streaming practice: Expression versus Misinterpretation and Harm; Public Participation versus Control and …

Enhancing Pointing Gestures Of Non-Hmd Users In Asymmetric Collocated Mixed Reality Collaboration, Nam-Dang Vo, Van-Vinh Thai, Anthony Tang, Khanh-Duy Le

Research Collection School Of Computing and Information Systems

A common collocated group setting in mixed-reality (MR) collaboration is a person wearing a MR headset (HMD user) and presenting MR contents to audiences who are not provided with such specialized devices (Non-HMD users). In this setting, while Non-HMD users can view the MR environment shown on a large physical display, it still remains challenging for the HMD user to interpret their pointing gesture when they spatially refer to objects in the MR environment. To address this, we designed and evaluated two pointing techniques—SCREEN and SCREEN+SPACE—that support Non-HMD users in referring to MR content. Screen pointing allows users to refer …

A Pruning-Based Question-Answering For Interactive Video Search: A Simple Baseline, Yu Tong Cheng, Phuong Anh Nguyen, Chong-Wah Ngo

Research Collection School Of Computing and Information Systems

There are various factors affecting the performance of video search. An imprecise query will enlarge search space and reduce the discriminative power of ranking functions. This problem is further exacerbated by the presence of numerous visually or semantically similar videos in large datasets. Consequently, users need to painstakingly browse through many highly similar candidates to locate the search target, leading to increased cognitive load and inefficient searching. Ideally, engaging users through interactive questioning to resolve uncertainties in the search process is an effective strategy for progressively narrowing down the search space. However, despite rapid advances in deep learning, generating informative …

Sam3-Litetext: An Anatomical Study Of The Sam3 Text Encoder For Efficient Vision-Language Segmentation, Chengxi Zeng, Yuxuan Jiang, Ge Gao, Shuai Wang, Duolikun Danier, Bin Zhu, Stevan Rudinac, David Bull, Fan Zhang

Research Collection School Of Computing and Information Systems

Vision-language segmentation models such as SAM3 enable flexible, prompt-driven visual grounding, but inherit large, general-purpose text encoders originally designed for open-ended language understanding. In practice, segmentation prompts are short, structured, and semantically constrained, leading to substantial over-provisioning in text encoder capacity and persistent computational and memory overhead. In this paper, we perform a large-scale anatomical analysis of text prompting in vision–language segmentation, covering 404,796 real prompts across multiple benchmarks. Our analysis reveals severe redundancy: most context windows are underutilized, vocabulary usage is highly sparse, and text embeddings lie on a low-dimensional manifold despite high-dimensional representations. Motivated by these findings, we …

Not Too Early, Not All At Once: Design Tensions In Ai-Mediated Self-Disclosure In Online Dating, Pei-Hua Tsai, Tianyi Zhang, Emran Bin Elias Poh, Anthony Tang, Yung-Ju Chang

Research Collection School Of Computing and Information Systems

Online dating relies on self-disclosure, yet initial conversations are fragile: users must navigate uncertainty around timing, boundaries, and reciprocity with little shared context. While advances in AI raise the possibility of mediating disclosure, how such support might reshape the experience of early-stage relational disclosure remains underexplored. We conducted 29 semi-structured interviews to examine how daters envision AI-mediated self-disclosure in online dating. Our findings surface recurring design tensions rather than simple opportunities or risks. Participants welcomed guidance that could pace disclosure, support reflection, and reduce social awkwardness, but stressed preserving agency and authorship. They valued interpretive assistance for sense-making of ambiguous …

Videocreator: An Agentic System For Multi-Turn Video Production, Zhengyang Liang, Yan Shu, Cathal Gurrin, Nicu Sebe, Lizi Liao

Research Collection School Of Computing and Information Systems

Recent advances in video generation models enable visually compelling single clips. However, real-world video creation is inherently continuous and iterative: creators refine content over multiple rounds while maintaining narrative, style, and entity consistency. Existing standalone generators are largely stateless and lack memory of previously generated segments, making it difficult to produce a coherent and consistent video project. To address this gap, we present VideoCreator, a unified video agent that integrates generation and understanding with a project-level memory system. VideoCreator leverages understanding capabilities to perform fine-grained analysis of newly produced content and uses persistent memory to retain and reuse prior context …

“From Remembering To Shaping”: Narrating Shared Experiences By Co-Designing Cultural Heritage Artifacts In Collaborative Vr, Yushang Yang, Fanxu Meng, Fiona Fui-Hoon Nah, L. C. Ray

Research Collection School Of Computing and Information Systems

The ways people remember and recall places reveal an invisible aspect of cultural heritage (CH), reflecting how individuals and communities relate to these places. Heritage is communal, emerging through collaboratively constructed narratives rather than individual records. To probe how people may share collective memories, we designed an immersive two-person workflow for collaboratively co-designing 3D artifacts and environments in virtual heritage locations, using Generative AI (GenAI) to instantiate these intangible memories. Observations of the co-creation process revealed that participants merged prompts and model placements when negotiating different perspectives. They used spatial operations to compose scenes, and also to express personal and …

History To Future: Evolving Agent With Experience And Thought For Zero-Shot Vision-And-Language Navigation, Guangzhao Dai, Shuo Wang, Zihan Wang, Guo-Sen Xie, Yang Yang, Jinshan Pan, Qianru Sun, Xiangbo Shu

Research Collection School Of Computing and Information Systems

Vision-and-Language Navigation in Continuous Environment (VLN-CE) requires an agent to follow language instructions to navigate the target destination. With the advancement of large language models (LLMs), recent efforts have explored adapting them for zero-shot VLN-CE, offering a promising solution in addressing the drawbacks of poor generalization in the training-based paradigm. However, existing LLM-based works primarily perform naive reasoning for decision-making and lack feedback, e.g., reviewing historical errors and predicting future potentials. Consequently, it may suffer from continuous failure for those initial error tasks. In this paper, we rethink LLM-based zero-shot VLN-CE and propose a new paradigm, named EvoNav, to improve …

Happycal: Designing Text And Image-Based Supports For Savouring Positive Work Experiences, Molly Stewart, Minghao Cai, Anthony Tang, Sam Liu, Chris Mosunic, Sowmya Somanath

Research Collection School Of Computing and Information Systems

Savouring positive work experiences can promote positive affect and well-being at work, yet there is limited guidance on how digital applications can support workers to engage in savouring. We developed HappyCal, a work-focused savouring application offering two forms of savouring support: text-based, a common modality in workplace reflection tools, and images, a largely unexplored approach in work-related savouring. We conducted an exploratory qualitative study where participants (N=36) used HappyCal over five days and engaged in savouring through either a text-only modality (n=17) or text input paired with image output (n=19). We found that (1) participants in both groups reported heightened …

Perceptual And Geometric Advances In Crowd Simulation, Bilas Talukdar

Dissertations

Simulating realistic crowd motion remains a fundamental challenge in computer graphics and multi-agent systems, as it requires modeling both physically plausible interactions and perceptually natural behaviors. Existing crowd simulation methods typically employ simplified geometric abstractions, most commonly circular agent representations, and model navigation using either analytical interaction formulations (e.g., force, velocity, or constraint-based methods) or learned policies derived through reinforcement learning. Despite their effectiveness, these approaches often overlook detailed geometric structure and do not explicitly account for perceptual realism. This dissertation addresses these challenges by improving the realism of virtual crowd simulation through two key advancements: perceptual preference learning and …