Jigsaw: Edge-Based Streaming Perception Over Spatially Overlapped Multi-Camera Deployments,
2024
Singapore Management University
Jigsaw: Edge-Based Streaming Perception Over Spatially Overlapped Multi-Camera Deployments, Ila Gokarn, Yigong Hu, Tarek Abdelzaher, Archan Misra
Research Collection School Of Computing and Information Systems
We present JIGSAW, a novel system that performs edge-based streaming perception over multiple video streams, while additionally factoring in the redundancy offered by the spatial overlap often exhibited in urban, multi-camera deployments. To assure high streaming throughput, JIGSAW extracts and spatially multiplexes multiple regions-of-interest from different camera frames into a smaller canvas frame. Moreover, to ensure that perception stays abreast of evolving object kinematics, JIGSAW includes a utility-based weighted scheduler to preferentially prioritize and even skip object-specific tiles extracted from an incoming stream of camera frames. Using the CityflowV2 traffic surveillance dataset, we show that JIGSAW can simultaneously process 25 …
An Exploratory Study Of Conventional Machine Learning And Large Language Models For Sentiment Analysis,
2024
Singapore Management University
An Exploratory Study Of Conventional Machine Learning And Large Language Models For Sentiment Analysis, Cui Zou, Jingyuan Cai, Langtao Chen, Fiona Fui-Hoon Nah
Research Collection School Of Computing and Information Systems
Sentiment analysis is the use of natural language processing to identify affective states and determine people’s opinions in various analytical applications such as customer reviews and social media analyses. Large language models (LLMs) such as GPT-4o demonstrate impressive performance in text generation tasks. Despite numerous studies in the extant literature, few have compared the performance of conventional machine learning models with LLMs for sentiment analysis. This study aims to fill this gap by conducting an evaluation of these models using a balanced dataset of 2,000 IMDb movie reviews. Our study shows that GPT-4o achieves the highest performance, while GPT-3.5 and …
A Computational Aesthetic Design Science Study On Online Video Based On Triple-Dimensional Multimodal Analysis,
2024
Singapore Management University
A Computational Aesthetic Design Science Study On Online Video Based On Triple-Dimensional Multimodal Analysis, Zhangguang Kang, Fiona Fui-Hoon Nah, Keng Siau
Research Collection School Of Computing and Information Systems
Computational video aesthetic prediction refers to using models that automatically evaluate the features of videos to produce their aesthetic scores. Current video aesthetic prediction models are designed based on bimodal frameworks. To address their limitations, we developed the Triple-Dimensional Multimodal Temporal Video Aesthetic neural network (TMTVA-net) model. The Long Short-Term Memory (LSTM) forms the conceptual foundation for the design framework. In the multimodal transformer layer, we employed two distinct transformers: the multimodal transformer and the feature transformer, enabling the acquisition of modality-specific patterns and representational features uniquely adapted to each modality. The fusion layer has also been redesigned to compute …
Understanding And Fighting Scams: Media, Language, Appeals And Effects,
2024
City University of Hong Kong
Understanding And Fighting Scams: Media, Language, Appeals And Effects, Shuhua Zhou, Xiao Fan Liu, Fiona Fui-Hoon Nah, S. Harrison, X. Zhang, S. Zhen, D. Yeung, J. Hsiao, R. Lc, A. Chan, X. Wang, C. Jiang, F. Lin, J. Li, A. Wong, L. Chan, B. George, P. Li
Research Collection School Of Computing and Information Systems
Scams are fraudulent activities aiming to deceive individuals into relinquishing money, property, or rights, and they have proliferated in the context of widespread misinformation and disinformation. In this paper, we propose strategies and a research plan to address key questions about the exploitation of new communication technologies by scammers, the prevalence and nature of different scam types, and the language characteristics and appeals used in scamming content. We aim to develop a comprehensive taxonomy of scams and identify factors that contribute to their persuasiveness. Additionally, we propose the use of advanced technologies, including artificial intelligence, physiological measures, and brain mapping, …
Hierarchical Damage Correlations For Old Photo Restoration,
2024
Singapore Management University
Hierarchical Damage Correlations For Old Photo Restoration, Weiwei Cai, Xuemiao Xu, Jiajia Xu, Huaidong Zhang, Haoxin Yang, Kun Zhang, Shengfeng He
Research Collection School Of Computing and Information Systems
Restoring old photographs can preserve cherished memories. Previous methods handled diverse damages within the same network structure, which proved impractical. In addition, these methods cannot exploit correlations among artifacts, especially in scratches versus patch-misses issues. Hence, a tailored network is particularly crucial. In light of this, we propose a unified framework consisting of two key components: ScratchNet and PatchNet. In detail, ScratchNet employs the parallel Multi-scale Partial Convolution Module to effectively repair scratches, learning from multi-scale local receptive fields. In contrast, the patch-misses necessitate the network to emphasize global information. To this end, we incorporate a transformer-based encoder and decoder …
Evilscreen Attack: Smart Tv Hijacking Via Multi-Channel Remote Control Mimicry,
2024
Singapore Management University
Evilscreen Attack: Smart Tv Hijacking Via Multi-Channel Remote Control Mimicry, Yiwei Zhang, Siqi Ma, Tiancheng Chen, Juanru Li, Robert H. Deng, Elisa Bertino
Research Collection School Of Computing and Information Systems
Modern smart TVs often communicate with their remote controls (including the smartphone simulated ones) using multiple wireless channels (e.g., Infrared, Bluetooth, and Wi-Fi). However, this multi-channel remote control communication introduces a new attack surface. An inherent security flaw is that remote controls of most smart TVs are designed to work in a benign environment rather than an adversarial one, and thus wireless communications between a smart TV and its remote controls are not strongly protected. Attackers can leverage such a flaw to abuse the remote control communication and compromise smart TV systems. In this paper, we propose EvilScreen, a novel …
Creating And Delivering Audio Descriptions For Videos,
2024
Singapore Management University
Creating And Delivering Audio Descriptions For Videos, Rosiana Natalie
Dissertations and Theses Collection (Open Access)
Despite anti-discrimination regulations mandating the provision of audio descriptions (ADs), the majority of online video content remains inaccessible to blind and low-vision (BLV) individuals. This is because these ADs are either absent or fail to adequately address the diverse and unique needs of the audience. Traditionally, content creators have relied on professionals to author ADs. However, this gold standard may not be accessible for some content creators because this method is still costly and has a long turnaround time. Moreover, when ADs are available, they tend to be static and unalterable, failing to cater to the unique preferences of BLV …
Creative Insights Into Motion: Enhancing Human Activity Understanding With 3d Data Visualization And Annotation,
2024
Chapman University
Creative Insights Into Motion: Enhancing Human Activity Understanding With 3d Data Visualization And Annotation, Isaac Browen, Hector M. Camarillo-Abad, Franceli L. Cibrian, Trudi Di Qi
Engineering Faculty Articles and Research
This paper presents a novel 3D system for human motion analysis - Motion Data Visualization and Annotation (MoViAn). Designed to provide a comprehensive visual representation of 3D human motion data, MoViAn incorporates detailed visualization of gaze direction, hand movements, and object interactions, alongside an interactive interface for efficient data annotation. A user study involving eight participants indicates that MoViAn enables users to thoroughly explore and annotate human motion data, with System Usability Scale (SUS) results demonstrating a satisfactory usability level. The contribution of this paper lies in the development of an interactive and usable data analytics tool aimed at deepening …
Technology And Homelessness: How Website Design And Blockchain Technology Could Impact The Unhoused,
2024
University of Denver
Technology And Homelessness: How Website Design And Blockchain Technology Could Impact The Unhoused, Casey Pratt
Undergraduate Theses, Capstones, and Recitals
Although technology could be used to combat inequality, it is instead increasing it. This paper discusses how the unhoused population suffers at the hand of technological inequality despite being relatively offline. It presents theories on how this would change if we reapproached how technology is used to assist the unhoused. It suggests implementing blockchain as a resource as well as modifying the websites built to assist in accessing benefits. Employees at shelters are interviewed for this paper about their experiences with using digital resources to rehouse and restabilize the vulnerable. They are asked how the sites can be improved for …
Curating Familiarity Within The Unfamiliar: Exploring Non-Native Mobile App Experiences To Create Cross-Cultural Design Frameworks,
2024
Dartmouth College
Curating Familiarity Within The Unfamiliar: Exploring Non-Native Mobile App Experiences To Create Cross-Cultural Design Frameworks, Hanna Hong
Computer Science Senior Theses
Global mobility and markets are expanding, and as a result, countries are becoming less and less monocultural. With multiple cultural affinity groups to cater towards, companies often will deploy different versions of a website or app based on the country a user is accessing it from. This strategy of catering to geographic location results in a lack of accommodation for people living within a culture that is different from their native one. In order to increase accessibility and equal ease-of-use for all audiences, designers should understand and work towards the needs of a multicultural user base. This study investigates how …
Balancing Darkness And Visibility: An Algorithmic Approach To Light Placement In Low-Light, Ray-Traced Scenes,
2024
California Polytechnic State University, San Luis Obispo
Balancing Darkness And Visibility: An Algorithmic Approach To Light Placement In Low-Light, Ray-Traced Scenes, Briana Kuo
Master's Theses
In recent years, digital media has seen incredible advancements in rendering visually stunning computer graphics scenes. Photo-realistic games, animated films, and more leave viewers blown away by the sheer beauty of their graphics. However, challenges arise when depicting dark scenes, often resulting in visual monotony and difficulty in comprehension due to insufficient detail within the scene. In order to enhance readability and visual interest of a scene, additional, artificial lights can be placed throughout a scene to enhance the aesthetic. These lights, however, must be strategically placed in order to retain an essence of darkness and maintain the delicate balance …
Violet: Visual Analytics For Explainable Quantum Neural Networks,
2024
Singapore Management University
Violet: Visual Analytics For Explainable Quantum Neural Networks, Shaolun Ruan, Zhiding Liang, Qiang Guan, Paul Robert Griffin, Xiaolin Wen, Yanna Lin, Yong Wang
Research Collection School Of Computing and Information Systems
With the rapid development of Quantum Machine Learning, quantum neural networks (QNN) have experienced great advancement in the past few years, harnessing the advantages of quantum computing to significantly speed up classical machine learning tasks. Despite their increasing popularity, the quantum neural network is quite counter-intuitive and difficult to understand, due to their unique quantum-specific layers (e.g., data encoding and measurement) in their architecture. It prevents QNN users and researchers from effectively understanding its inner workings and exploring the model training status. To fill the research gap, we propose VIOLET , a novel visual analytics approach to improve the explainability …
Inceptionnext: When Inception Meets Convnext,
2024
Singapore Management University
Inceptionnext: When Inception Meets Convnext, Weihao Yu, Pan Zhou, Shuicheng Yan, Xinchao Wang
Research Collection School Of Computing and Information Systems
Inspired by the long-range modeling ability of ViTs, large-kernel convolutions are widely studied and adopted recently to enlarge the receptive field and improve model performance, like the remarkable work ConvNeXt which employs 7×7 depthwise convolution. Although such depthwise operator only consumes a few FLOPs, it largely harms the model efficiency on powerful computing devices due to the high memory access costs. For example, ConvNeXtT has similar FLOPs with ResNet-50 but only achieves ∼ 60% throughputs when trained on A100 GPUs with full precision. Although reducing the kernel size of ConvNeXt can improve speed, it results in significant performance degradation, which …
Let’S Think Outside The Box: Exploring Leap-Of-Thought In Large Language Models With Multimodal Humor Generation,
2024
Singapore Management University
Let’S Think Outside The Box: Exploring Leap-Of-Thought In Large Language Models With Multimodal Humor Generation, Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, Pan Zhou
Research Collection School Of Computing and Information Systems
Chain-of-Thought (CoT) [2, 3] guides large language models (LLMs) to reason step-by-step, and can motivate their logical reasoning ability. While effective for logical tasks, CoT is not conducive to creative problem-solving which often requires out-of-box thoughts and is crucial for innovation advancements. In this paper, we explore the Leap-of-Thought (LoT) abilities within LLMs — a nonsequential, creative paradigm involving strong associations and knowledge leaps. To this end, we study LLMs on the popular Oogiri game which needs participants to have good creativity and strong associative thinking for responding unexpectedly and humorously to the given image, text, or both, and thus …
Few-Shot Learner Parameterization By Diffusion Time-Steps,
2024
Singapore Management University
Few-Shot Learner Parameterization By Diffusion Time-Steps, Zhongqi Yue, Pan Zhou, Richang Hong, Hanwang Zhang, Sun Qianru
Research Collection School Of Computing and Information Systems
Even when using large multi-modal foundation models, few-shot learning is still challenging—if there is no proper inductive bias, it is nearly impossible to keep the nuanced class attributes while removing the visually prominent attributes that spuriously correlate with class labels. To this end, we find an inductive bias that the time-steps of a Diffusion Model (DM) can isolate the nuanced class attributes, i.e., as the forward diffusion adds noise to an image at each time-step, nuanced attributes are usually lost at an earlier time-step than the spurious attributes that are visually prominent. Building on this, we propose Time-step Few-shot (TiF) …
Efficient Cross-Modal Video Retrieval With Meta-Optimized Frames,
2024
Singapore Management University
Efficient Cross-Modal Video Retrieval With Meta-Optimized Frames, Ning Han, Xun Yang, Ee-Peng Lim, Hao Chen, Qianru Sun
Research Collection School Of Computing and Information Systems
Cross-modal video retrieval aims to retrieve semantically relevant videos when given a textual query, and is one of the fundamental multimedia tasks. Most top-performing methods primarily leverage Vision Transformer (ViT) to extract video features [1]-[3]. However, they suffer from the high computational complexity of ViT, especially when encoding long videos. A common and simple solution is to uniformly sample a small number (e.g., 4 or 8) of frames from the target video (instead of using the whole video) as ViT inputs. The number of frames has a strong influence on the performance of ViT, e.g., using 8 frames yields better …
Diffusion Time-Step Curriculum For One Image To 3d Generation,
2024
Singapore Management University
Diffusion Time-Step Curriculum For One Image To 3d Generation, Xuanyu Yi, Zike Wu, Qingshan Xu, Pan Zhou, Joo Hwee Lim, Hanwang Zhang
Research Collection School Of Computing and Information Systems
Score distillation sampling (SDS) has been widely adopted to overcome the absence of unseen views in reconstructing 3D objects from a single image. It leverages pretrained 2D diffusion models as teacher to guide the reconstruction of student 3D models. Despite their remarkable success, SDS-based methods often encounter geometric artifacts and texture saturation. We find out the crux is the overlooked indiscriminate treatment of diffusion time-steps during optimization: it unreasonably treats the studentteacher knowledge distillation to be equal at all time-steps and thus entangles coarse-grained and fine-grained modeling. Therefore, we propose the Diffusion Time-step Curriculum one-image-to-3D pipeline (DTC123), which involves both …
Jollygesture: Exploring Dual-Purpose Gestures In Vr Presentations,
2024
Singapore Management University
Jollygesture: Exploring Dual-Purpose Gestures In Vr Presentations, Gun Woo Warren Park, Anthony Tang, Fanny Chevalier
Research Collection School Of Computing and Information Systems
Virtual reality (VR) offers new opportunities for presenters to use expressive body language to engage their audience. Yet, most VR presentation systems have adopted control mechanisms that mimic those found in face-to-face presentation systems. We explore the use of gestures that have dual-purpose: first, for the audience, a communicative purpose; second, for the presenter, a control purpose to alter content in slides. To support presenters, we provide guidance on what gestures are available and their effects. We realize our design approach in JollyGesture, a VR technology probe that recognizes dual-purpose gestures in a presentation scenario. We evaluate our approach through …
Posmlp-Video: Spatial And Temporal Relative Position Encoding For Efficient Video Recognition,
2024
Singapore Management University
Posmlp-Video: Spatial And Temporal Relative Position Encoding For Efficient Video Recognition, Yanbin Hao, Diansong Zhou, Zhicai Wang, Chong-Wah Ngo, Xiangnan He, Meng Wang
Research Collection School Of Computing and Information Systems
In recent years, vision Transformers and MLPs have demonstrated remarkable performance in image understanding tasks. However, their inherently dense computational operators, such as self-attention and token-mixing layers, pose significant challenges when applied to spatio-temporal video data. To address this gap, we propose PosMLP-Video, a lightweight yet powerful MLP-like backbone for video recognition. Instead of dense operators, we use efficient relative positional encoding (RPE) to build pairwise token relations, leveraging small-sized parameterized relative position biases to obtain each relation score. Specifically, to enable spatio-temporal modeling, we extend the image PosMLP’s positional gating unit to temporal, spatial, and spatio-temporal variants, namely PoTGU, …
Open-Vocabulary Video Anomaly Detection,
2024
Singapore Management University
Open-Vocabulary Video Anomaly Detection, Peng Wu, Xuerong Zhou, Guansong Pang, Yujia Sun, Jing Liu, Peng Wang, Yanning Zhang
Research Collection School Of Computing and Information Systems
Current video anomaly detection (VAD) approaches with weak supervisions are inherently limited to a closed-set setting and may struggle in open-world applications where there can be anomaly categories in the test data unseen during training. A few recent studies attempt to tackle a more realistic setting, open-set VAD, which aims to de-tect unseen anomalies given seen anomalies and normal videos. However, such a setting focuses on predicting frame anomaly scores, having no ability to recognize the specific categories of anomalies, despite the fact that this ability is essential for building more informed video surveillance systems. This paper takes a step …
