Open Access. Powered by Scholars. Published by Universities.®

Graphics and Human Computer Interfaces Commons™

Open Access. Powered by Scholars. Published by Universities.®

Singapore Management University

Discipline
Keyword
Publication Year
Publication
Publication Type

Articles 301 - 330 of 959

Full-Text Articles in Graphics and Human Computer Interfaces

Towards A Smaller Student: Capacity Dynamic Distillation For Efficient Image Retrieval, Yi Xie, Huaidong Zhang, Xuemiao Xu, Jianqing Zhu, Shengfeng He Jun 2023

Towards A Smaller Student: Capacity Dynamic Distillation For Efficient Image Retrieval, Yi Xie, Huaidong Zhang, Xuemiao Xu, Jianqing Zhu, Shengfeng He

Research Collection School Of Computing and Information Systems

Previous Knowledge Distillation based efficient image retrieval methods employ a lightweight network as the student model for fast inference. However, the lightweight student model lacks adequate representation capacity for effective knowledge imitation during the most critical early training period, causing final performance degeneration. To tackle this issue, we propose a Capacity Dynamic Distillation framework, which constructs a student model with editable representation capacity. Specifically, the employed student model is initially a heavy model to fruitfully learn distilled knowledge in the early training epochs, and the student model is gradually compressed during the training. To dynamically adjust the model capacity, our …

Go to article

Generalizing Graph Neural Networks Across Graphs, Time, And Tasks, Zhihao Wen Jun 2023

Generalizing Graph Neural Networks Across Graphs, Time, And Tasks, Zhihao Wen

Dissertations and Theses Collection (Open Access)

Graph-structured data are ubiquitous across numerous real-world contexts, encompassing social networks, commercial graphs, bibliographic networks, and biological systems. Delving into the analysis of these graphs can yield significant understanding pertaining to their corresponding application fields.Graph representation learning offers a potent solution to graph analytics challenges by transforming a graph into a low-dimensional space while preserving its information to the greatest extent possible. This conversion into low-dimensional vectors enables the efficient computation of subsequent graph algorithms. The majority of prior research has concentrated on deriving node representations from a single, static graph. However, numerous real-world situations demand rapid generation of representations …

Go to article

Venus: A Geometrical Representation For Quantum State Visualization, Shaolun Ruan, Ribo Yuan, Qiang Guan, Yanna Lin, Ying Mao, Weiwen Jiang, Zhepeng Wang, Wei Xu, Yong Wang Jun 2023

Venus: A Geometrical Representation For Quantum State Visualization, Shaolun Ruan, Ribo Yuan, Qiang Guan, Yanna Lin, Ying Mao, Weiwen Jiang, Zhepeng Wang, Wei Xu, Yong Wang

Research Collection School Of Computing and Information Systems

Visualizations have played a crucial role in helping quantum computing users explore quantum states in various quantum computing applications. Among them, Bloch Sphere is the widely-used visualization for showing quantum states, which leverages angles to represent quantum amplitudes. However, it cannot support the visualization of quantum entanglement and superposition, the two essential properties of quantum computing. To address this issue, we propose VENUS, a novel visualization for quantum state representation. By explicitly correlating 2D geometric shapes based on the math foundation of quantum computing characteristics, VENUS effectively represents quantum amplitudes of both the single qubit and two qubits for quantum …

Go to article

Class-Incremental Exemplar Compression For Class-Incremental Learning, Zilin Luo, Yaoyao Liu, Bernt Schiele, Qianru Sun Jun 2023

Class-Incremental Exemplar Compression For Class-Incremental Learning, Zilin Luo, Yaoyao Liu, Bernt Schiele, Qianru Sun

Research Collection School Of Computing and Information Systems

Exemplar-based class-incremental learning (CIL) finetunes the model with all samples of new classes but few-shot exemplars of old classes in each incremental phase, where the "few-shot" abides by the limited memory budget. In this paper, we break this "few-shot" limit based on a simple yet surprisingly effective idea: compressing exemplars by downsampling non-discriminative pixels and saving "many-shot" compressed exemplars in the memory. Without needing any manual annotation, we achieve this compression by generating 0-1 masks on discriminative pixels from class activation maps (CAM). We propose an adaptive mask generation model called class-incremental masking (CIM) to explicitly resolve two difficulties of …

Go to article

Extracting Class Activation Maps From Non-Discriminative Features As Well, Zhaozheng Chen, Qianru Sun Jun 2023

Extracting Class Activation Maps From Non-Discriminative Features As Well, Zhaozheng Chen, Qianru Sun

Research Collection School Of Computing and Information Systems

Extracting class activation maps (CAM) from a classification model often results in poor coverage on foreground objects, i.e., only the discriminative region (e.g., the “head” of “sheep”) is recognized and the rest (e.g., the “leg” of “sheep”) mistakenly as background. The crux behind is that the weight of the classifier (used to compute CAM) captures only the discriminative features of objects. We tackle this by introducing a new computation method for CAM that explicitly captures non-discriminative features as well, thereby expanding CAM to cover whole objects. Specifically, we omit the last pooling layer of the classification model, and perform clustering …

Go to article

Freestyle Layout-To-Image Synthesis, Han Xue, Zhiwu Huang, Qianru Sun, Li Song, Wenjun Zhang Jun 2023

Freestyle Layout-To-Image Synthesis, Han Xue, Zhiwu Huang, Qianru Sun, Li Song, Wenjun Zhang

Research Collection School Of Computing and Information Systems

Typical layout-to-image synthesis (LIS) models generate images for a close set of semantic classes, e.g., 182 common objects in COCO-Stuff. In this work, we explore the freestyle capability of the model, i.e., how far can it generate unseen semantics (e.g., classes, attributes, and styles) onto a given layout, and call the task Freestyle LIS (FLIS). Thanks to the development of large-scale pre-trained language-image models, a number of discriminative models (e.g., image classification and object detection) trained on limited base classes are empowered with the ability of unseen class prediction. Inspired by this, we opt to leverage large-scale pre-trained text-to-image diffusion …

Go to article

Semantic Scene Completion With Cleaner Self, Fengyun Wang, Dong Zhang, Hanwang Zhang, Jinhui Tang, Qianru Sun Jun 2023

Semantic Scene Completion With Cleaner Self, Fengyun Wang, Dong Zhang, Hanwang Zhang, Jinhui Tang, Qianru Sun

Research Collection School Of Computing and Information Systems

Semantic Scene Completion (SSC) transforms an image of single-view depth and/or RGB 2D pixels into 3D voxels, each of whose semantic labels are predicted. SSC is a well-known ill-posed problem as the prediction model has to “imagine” what is behind the visible surface, which is usually represented by Truncated Signed Distance Function (TSDF). Due to the sensory imperfection of the depth camera, most existing methods based on the noisy TSDF estimated from depth values suffer from 1) incomplete volumetric predictions and 2) confused semantic labels. To this end, we use the ground-truth 3D voxels to generate a perfect visible surface, …

Go to article

Unbiased Multiple Instance Learning For Weakly Supervised Video Anomaly Detection, Hui Lyu, Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, Hanwang Zhang Jun 2023

Unbiased Multiple Instance Learning For Weakly Supervised Video Anomaly Detection, Hui Lyu, Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Weakly Supervised Video Anomaly Detection (WSVAD) is challenging because the binary anomaly label is only given on the video level, but the output requires snippet-level predictions. So, Multiple Instance Learning (MIL) is prevailing in WSVAD. However, MIL is notoriously known to suffer from many false alarms because the snippet-level detector is easily biased towards the abnormal snippets with simple context, confused by the normality with the same bias, and missing the anomaly with a different pattern. To this end, we propose a new MIL framework: Unbiased MIL (UMIL), to learn unbiased anomaly features that improve WSVAD. At each MIL training …

Go to article

Evading Deepfake Detectors Via Adversarial Statistical Consistency, Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Jianjun Zhao Jun 2023

Evading Deepfake Detectors Via Adversarial Statistical Consistency, Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Jianjun Zhao

Research Collection School Of Computing and Information Systems

In recent years, as various realistic face forgery techniques known as DeepFake improves by leaps and bounds, more and more DeepFake detection techniques have been proposed. These methods typically rely on detecting statistical differences between natural (i.e., real) and DeepFake-generated images in both spatial and frequency domains. In this work, we propose to explicitly minimize the statistical differences to evade state-of-the-art DeepFake detectors. To this end, we propose a statistical consistency attack (StatAttack) against DeepFake detectors, which contains two main parts. First, we select several statistical-sensitive natural degradations (i.e., exposure, blur, and noise) and add them to the fake images …

Go to article

Ifundit: Visual Profiling Of Fund Investment Styles, Rong Zhang, Bon Kyung Ku, Yong Wang, Xuanwu Yue, Siyuan Liu, Ke Li, Huamin Qu Jun 2023

Ifundit: Visual Profiling Of Fund Investment Styles, Rong Zhang, Bon Kyung Ku, Yong Wang, Xuanwu Yue, Siyuan Liu, Ke Li, Huamin Qu

Research Collection School Of Computing and Information Systems

Mutual funds are becoming increasingly popular with the emergence of Internet finance. Clear profiling of a fund's investment style is crucial for fund managers to evaluate their investment strategies, and for investors to understand their investment. However, it is challenging to profile a fund's investment style as it requires a comprehensive analysis of complex multi-dimensional temporal data. In addition, different fund managers and investors have different focuses when analysing a fund's investment style. To address the issue, we propose iFUNDit, an interactive visual analytic system for fund investment style analysis. The system decomposes a fund's critical features into performance attributes …

Go to article

Gnnlens: A Visual Analytics Approach For Prediction Error Diagnosis Of Graph Neural Networks., Zhihua Jin, Yong Wang, Qianwen Wang, Yao Ming, Tengfei Ma, Huamin Qu Jun 2023

Gnnlens: A Visual Analytics Approach For Prediction Error Diagnosis Of Graph Neural Networks., Zhihua Jin, Yong Wang, Qianwen Wang, Yao Ming, Tengfei Ma, Huamin Qu

Research Collection School Of Computing and Information Systems

Graph Neural Networks (GNNs) aim to extend deep learning techniques to graph data and have achieved significant progress in graph analysis tasks (e.g., node classification) in recent years. However, similar to other deep neural networks like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), GNNs behave like a black box with their details hidden from model developers and users. It is therefore difficult to diagnose possible errors of GNNs. Despite many visual analytics studies being done on CNNs and RNNs, little research has addressed the challenges for GNNs. This paper fills the research gap with an interactive visual analysis …

Go to article

Document Graph Representation Learning, Ce Zhang May 2023

Document Graph Representation Learning, Ce Zhang

Dissertations and Theses Collection (Open Access)

Much of the data on the Web can be represented in a graph structure, ranging from social and biological to academic and Web page graphs, etc. Graph analysis recently attracts escalating research attention due to its importance and wide applicability. Diverse problems could be formulated as graph tasks, such as text classification and information retrieval. As the primary information is the inherent structure of the graph itself, one promising direction known as the graph representation learning problem is to learn the representation of each node, which could in turn fuel tasks such as node classification, node clustering, and link prediction. …

Go to article

Liloc: Enabling Precise 3d Localization In Dynamic Indoor Environments Using Lidars, Darshana Rathnayake, Meera Radhakrishnan, Inseok Hwang, Archan Misra May 2023

Liloc: Enabling Precise 3d Localization In Dynamic Indoor Environments Using Lidars, Darshana Rathnayake, Meera Radhakrishnan, Inseok Hwang, Archan Misra

Research Collection School Of Computing and Information Systems

We present LiLoc, a system for precise 3D localization and tracking of mobile IoT devices (e.g., robots) in indoor environments using multi-perspective LiDAR sensing. The key differentiators in our work are: (a) First, unlike traditional localization approaches, our approach is robust to dynamically changing environmental conditions (e.g., varying crowd levels, object placement/layout changes); (b) Second, unlike prior work on visual and 3D SLAM, LiLoc is not dependent on a pre-built static map of the environment and instead works by utilizing dynamically updated point clouds captured from both infrastructural-mounted LiDARs and LiDARs equipped on individual mobile IoT devices. To achieve fine-grained, …

Go to article

Compositional Prompt Tuning With Motion Cues For Open-Vocabulary Video Relation Detection, Kaifeng Gao, Long Chen, Hanwang Zhang, Jun Xiao, Qianru Sun May 2023

Compositional Prompt Tuning With Motion Cues For Open-Vocabulary Video Relation Detection, Kaifeng Gao, Long Chen, Hanwang Zhang, Jun Xiao, Qianru Sun

Research Collection School Of Computing and Information Systems

Prompt tuning with large-scale pretrained vision-language models empowers open-vocabulary prediction trained on limited base categories, e.g., object classification and detection. In this paper, we propose compositional prompt tuning with motion cues: an extended prompt tuning paradigm for compositional predictions of video data. In particular, we present Relation Prompt (RePro) for Open-vocabulary Video Visual Relation Detection (Open-VidVRD), where conventional prompt tuning is easily biased to certain subject-object combinations and motion patterns. To this end, RePro addresses the two technical challenges of Open-VidVRD: 1) the prompt tokens should respect the two different semantic roles of subject and object, and 2) the tuning …

Go to article

Mando-Hgt: Heterogeneous Graph Transformers For Smart Contract Vulnerability Detection, Huu Hoang Nguyen, Nhat Minh Nguyen, Chunyao Xie, Zahra Ahmadi, Daniel Kudendo, Thanh-Nam Doan, Lingxiao Jiang May 2023

Mando-Hgt: Heterogeneous Graph Transformers For Smart Contract Vulnerability Detection, Huu Hoang Nguyen, Nhat Minh Nguyen, Chunyao Xie, Zahra Ahmadi, Daniel Kudendo, Thanh-Nam Doan, Lingxiao Jiang

Research Collection School Of Computing and Information Systems

Smart contracts in blockchains have been increasingly used for high-value business applications. It is essential to check smart contracts' reliability before and after deployment. Although various program analysis and deep learning techniques have been proposed to detect vulnerabilities in either Ethereum smart contract source code or bytecode, their detection accuracy and scalability are still limited. This paper presents a novel framework named MANDO-HGT for detecting smart contract vulnerabilities. Given Ethereum smart contracts, either in source code or bytecode form, and vulnerable or clean, MANDO-HGT custom-builds heterogeneous contract graphs (HCGs) to represent control-flow and/or function-call information of the code. It then …

Go to article

Chronos: Time-Aware Zero-Shot Identification Of Libraries From Vulnerability Reports, Yunbo Lyu, Thanh Le Cong, Hong Jin Kang, Ratnadira Widyasari, Zhipeng Zhao, Xuan-Bach Dinh Le, Ming Li, David Lo May 2023

Chronos: Time-Aware Zero-Shot Identification Of Libraries From Vulnerability Reports, Yunbo Lyu, Thanh Le Cong, Hong Jin Kang, Ratnadira Widyasari, Zhipeng Zhao, Xuan-Bach Dinh Le, Ming Li, David Lo

Research Collection School Of Computing and Information Systems

Tools that alert developers about library vulnerabilities depend on accurate, up-to-date vulnerability databases which are maintained by security researchers. These databases record the libraries related to each vulnerability. However, the vulnerability reports may not explicitly list every library and human analysis is required to determine all the relevant libraries. Human analysis may be slow and expensive, which motivates the need for automated approaches. Researchers and practitioners have proposed to automatically identify libraries from vulnerability reports using extreme multi-label learning (XML). While state-of-the-art XML techniques showed promising performance, their experimental settings do not practically fit what happens in reality. Previous studies …

Go to article

Diffseer: Difference-Based Dynamic Weighted Graph Visualization, Xiaolin Wen, Yong Wang, Meixuan Wu, Fengjie Wang, Xuanwu Yue, Qiaomu Shen, Yuxin Ma, Min Zhu May 2023

Diffseer: Difference-Based Dynamic Weighted Graph Visualization, Xiaolin Wen, Yong Wang, Meixuan Wu, Fengjie Wang, Xuanwu Yue, Qiaomu Shen, Yuxin Ma, Min Zhu

Research Collection School Of Computing and Information Systems

Existing dynamic weighted graph visualization approaches rely on users’ mental comparison to perceive temporal evolution of dynamic weighted graphs, hindering users from effectively analyzing changes across multiple timeslices. We propose DiffSeer, a novel approach for dynamic weighted graph visualization by explicitly visualizing the differences of graph structures (e.g., edge weight differences) between adjacent timeslices. Specifically, we present a novel nested matrix design that overviews the graph structure differences over a time period as well as shows graph structure details in the timeslices of user interest. By collectively considering the overall temporal evolution and structure details in each timeslice, an optimization-based …

Go to article

Lpt: Long-Tailed Prompt Tuning For Image Classification, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo May 2023

Lpt: Long-Tailed Prompt Tuning For Image Classification, Bowen Dong, Pan Zhou, Shuicheng Yan, Wangmeng Zuo

Research Collection School Of Computing and Information Systems

For long-tailed classification tasks, most works often pretrain a big model on a large-scale (unlabeled) dataset, and then fine-tune the whole pretrained model for adapting to long-tailed data. Though promising, fine-tuning the whole pretrained model tends to suffer from high cost in computation and deployment of different models for different tasks, as well as weakened generalization capability for overfitting to certain features of long-tailed data. To alleviate these issues, we propose an effective Long-tailed Prompt Tuning (LPT) method for long-tailed classification tasks. LPT introduces several trainable prompts into a frozen pretrained model to adapt it to long-tailed data. For better …

Go to article

Towards Understanding Why Mask Reconstruction Pretraining Helps In Downstream Tasks, Jiachun Pan, Pan Zhou, Shuicheng Yan May 2023

Towards Understanding Why Mask Reconstruction Pretraining Helps In Downstream Tasks, Jiachun Pan, Pan Zhou, Shuicheng Yan

Research Collection School Of Computing and Information Systems

For unsupervised pretraining, mask-reconstruction pretraining (MRP) approaches, e.g. MAE (He et al., 2021) and data2vec (Baevski et al., 2022), randomly mask input patches and then reconstruct the pixels or semantic features of these masked patches via an auto-encoder. Then for a downstream task, supervised fine-tuning the pretrained encoder remarkably surpasses the conventional “supervised learning" (SL) trained from scratch. However, it is still unclear 1) how MRP performs semantic feature learning in the pretraining phase and 2) why it helps in downstream tasks. To solve these problems, we first theoretically show that on an auto-encoder of a two/one-layered convolution encoder/decoder, MRP …

Go to article

Graph Neural Point Process For Temporal Interaction Prediction, Wenwen Xia, Yuchen Li, Shengdong Li May 2023

Graph Neural Point Process For Temporal Interaction Prediction, Wenwen Xia, Yuchen Li, Shengdong Li

Research Collection School Of Computing and Information Systems

Temporal graphs are ubiquitous data structures in many scenarios, including social networks, user-item interaction networks, etc. In this paper, we focus on predicting the exact time of the next interaction, given a node pair on a temporal graph. This novel problem can support interesting applications, such as time-sensitive items recommendation, congestion prediction on road networks, and many others. We present Graph Neural Point Process (GNPP) to tackle this problem. GNPP relies on the graph neural message passing and the temporal point process framework. Most previous graph neural models only utilize the chronological order of observed events and ignore exact timestamps. …

Go to article

Supporting Novices Author Audio Descriptions Via Automatic Feedback, Rosiana Natalie, Joshua Shi-Hao Tseng, Hernisa Kacorri, Kotaro Hara Apr 2023

Supporting Novices Author Audio Descriptions Via Automatic Feedback, Rosiana Natalie, Joshua Shi-Hao Tseng, Hernisa Kacorri, Kotaro Hara

Research Collection School Of Computing and Information Systems

Audio descriptions (AD) make videos accessible to those who cannot see them. But many videos lack AD and remain inaccessible as traditional approaches involve expensive professional production. We aim to lower production costs by involving novices in this process. We present an AD authoring system that supports novices to write scene descriptions (SD)—textual descriptions of video scenes—and convert them into AD via text-to-speech. The system combines video scene recognition and natural language processing to review novice-written SD and feeds back what to mention automatically. To assess the effectiveness of this automatic feedback in supporting novices, we recruited 60 participants to …

Go to article

Open-Set Domain Adaptation By Deconfounding Domain Gaps, Xin Zhao, Shengsheng Wang, Qianru Sun Apr 2023

Open-Set Domain Adaptation By Deconfounding Domain Gaps, Xin Zhao, Shengsheng Wang, Qianru Sun

Research Collection School Of Computing and Information Systems

Open-Set Domain Adaptation (OSDA) aims to adapt the model trained on a source domain to the recognition tasks in a target domain while shielding any distractions caused by open-set classes, i.e., the classes “unknown” to the source model. Compared to standard DA, the key of OSDA lies in the separation between known and unknown classes. Existing OSDA methods often fail the separation because of overlooking the confounders (i.e., the domain gaps), which means their recognition of “unknown classes” is not because of class semantics but domain difference (e.g., styles and contexts). We address this issue by explicitly deconfounding domain gaps …

Go to article

Stargazer: An Interactive Camera Robot For Capturing How-To Videos Based On Subtle Instructor Cues, Jiannan Li, Mauricio Sousa, Karthik Mahadevan, Bryan Wang, Paula Akemi Aoyagui, Nicole Yu, Angela Yang, Ravin Balakrishnan, Anthony Tang, Tovi Grossman Apr 2023

Stargazer: An Interactive Camera Robot For Capturing How-To Videos Based On Subtle Instructor Cues, Jiannan Li, Mauricio Sousa, Karthik Mahadevan, Bryan Wang, Paula Akemi Aoyagui, Nicole Yu, Angela Yang, Ravin Balakrishnan, Anthony Tang, Tovi Grossman

Research Collection School Of Computing and Information Systems

Live and pre-recorded video tutorials are an effective means for teaching physical skills such as cooking or prototyping electronics. A dedicated cameraperson following an instructor’s activities can improve production quality. However, instructors who do not have access to a cameraperson’s help often have to work within the constraints of static cameras. We present Stargazer, a novel approach for assisting with tutorial content creation with a camera robot that autonomously tracks regions of interest based on instructor actions to capture dynamic shots. Instructors can adjust the camera behaviors of Stargazer with subtle cues, including gestures and speech, allowing them to fluidly …

Go to article

Subgraph Centralization: A Necessary Step For Graph Anomaly Detection, Zhong Zhuang, Kai Ming Ting, Guansong Pang, Shuaibin Song Apr 2023

Subgraph Centralization: A Necessary Step For Graph Anomaly Detection, Zhong Zhuang, Kai Ming Ting, Guansong Pang, Shuaibin Song

Research Collection School Of Computing and Information Systems

Abstract Graph anomaly detection has attracted a lot of interest recently. Despite their successes, existing detectors have at least two of the three weaknesses: (a) high computational cost which limits them to small-scale networks only; (b) existing treatment of subgraphs produces suboptimal detection accuracy; and (c) unable to provide an explanation as to why a node is anomalous, once it is identified. We identify that the root cause of these weaknesses is a lack of a proper treatment for subgraphs. A treatment called Subgraph Centralization for graph anomaly detection is proposed to address all the above weaknesses. Its importance is …

Go to article

Investigating Noticeable Hand Redirection In Virtual Reality Using Physiological And Interaction Data, Martin Feick, Kora Regitz, Anthony Tang, Jungbluth Mar 2023

Investigating Noticeable Hand Redirection In Virtual Reality Using Physiological And Interaction Data, Martin Feick, Kora Regitz, Anthony Tang, Jungbluth

Research Collection School Of Computing and Information Systems

Hand redirection is effective so long as the introduced offsets are not noticeably disruptive to users. In this work we investigate the use of physiological and interaction data to detect movement discrepancies between a user's real and virtual hand, pushing towards a novel approach to identify discrepancies which are too large and therefore can be noticed. We ran a study with 22 participants, collecting EEG, ECG, EDA, RSP, and interaction data. Our results suggest that EEG and interaction data can be reliably used to detect visuo-motor discrepancies, whereas ECG and RSP seem to suffer from inconsistencies. Our findings also show …

Go to article

Towards A Design Space For Storytelling On The Fashion Technology Runway, Sydney Pratte, Anthony Tang, Shannon Hoover, Maria Elena Hoover, Matt Laprarie, Catherine Larose, Lora Oehlberg Mar 2023

Towards A Design Space For Storytelling On The Fashion Technology Runway, Sydney Pratte, Anthony Tang, Shannon Hoover, Maria Elena Hoover, Matt Laprarie, Catherine Larose, Lora Oehlberg

Research Collection School Of Computing and Information Systems

Fashion is driven by a narrative, i.e. a story or idea that the designer wants to convey to the audience. Fashion-tech now adds another dimension to this narrative through dynamically changing aspects of the garments. Many factors of presentation in a runway show affect how fashion-tech garments communicate a story to the audience. In this pictorial, we review a set of twenty-eight storytelling fashion-tech garments. We identify, catalogue, and categorize the factors designers used to convey stories to the audience from the runway. The design space consists of three levels: (1) the artifact-level, (2) the viewer-level, and (3) the context-level. …

Go to article

Wearables For In-Situ Monitoring Of Cognitive States: Challenges And Opportunities, Meeralakshmi Radhakrishnan, Thivya Kandappu, Manoj Gulati, Archan Misra Mar 2023

Wearables For In-Situ Monitoring Of Cognitive States: Challenges And Opportunities, Meeralakshmi Radhakrishnan, Thivya Kandappu, Manoj Gulati, Archan Misra

Research Collection School Of Computing and Information Systems

We propose using wrist and ear-based sensing, via multiple novel and complementary modalities, to unobtrusively infer activity-aware, complex cognitive and affective states (such as confusion, boredom, and recall failure) of individuals. While state-of-the-art wearable devices are predominantly used (a) independently, with limited coordination among multiple devices, and (b) to capture macro-level physical activity and physiological state, we seek to expand the ambit of unobtrusive wearable sensing to capture the cognitive states while performing commonplace physical activities. Such states typically manifest via fine-grained, almost unobservable, microscopic head, face, and eye movements. We identify some of these fine-grained physical markers that serve …

Go to article

Concept-Oriented Transformers For Visual Sentiment Analysis, Quoc Tuan Truong, Hady Wirawan Lauw Mar 2023

Concept-Oriented Transformers For Visual Sentiment Analysis, Quoc Tuan Truong, Hady Wirawan Lauw

Research Collection School Of Computing and Information Systems

In the richly multimedia Web, detecting sentiment signals expressed in images would support multiple applications, e.g., measuring customer satisfaction from online reviews, analyzing trends and opinions from social media. Given an image, visual sentiment analysis aims at recognizing positive or negative sentiment, and occasionally neutral sentiment as well. A nascent yet promising direction is Transformer-based models applied to image data, whereby Vision Transformer (ViT) establishes remarkable performance on largescale vision benchmarks. In addition to investigating the fitness of ViT for visual sentiment analysis, we further incorporate concept orientation into the self-attention mechanism, which is the core component of Transformer. The …

Go to article

Wearables For In-Situ Monitoring Of Cognitive States: Challenges And Opportunities, Meera Radhakrishnan, Thivya Kandappu, Manoj Gulati, Archan Misra Mar 2023

Wearables For In-Situ Monitoring Of Cognitive States: Challenges And Opportunities, Meera Radhakrishnan, Thivya Kandappu, Manoj Gulati, Archan Misra

Research Collection School Of Computing and Information Systems

We propose using wrist and ear-based sensing, via multiple novel and complementary modalities, to unobtrusively infer activity-aware, complex cognitive and affective states (such as confusion, boredom, and recall failure) of individuals. While state-of-the-art wearable devices are predominantly used (a) independently, with limited coordination among multiple devices, and (b) to capture macro-level physical activity and physiological state, we seek to expand the ambit of unobtrusive wearable sensing to capture the cognitive states while performing commonplace physical activities. Such states typically manifest via fine-grained, almost unobservable, microscopic head, face, and eye movements. We identify some of these fine-grained physical markers that serve …

Go to article

Investigating Guardian Awareness Techniques To Promote Safety In Virtual Reality, Sixuan Wu, Jiannan Li, Maurício Sousa, Tovi Grossman Mar 2023

Investigating Guardian Awareness Techniques To Promote Safety In Virtual Reality, Sixuan Wu, Jiannan Li, Maurício Sousa, Tovi Grossman

Research Collection School Of Computing and Information Systems

Virtual Reality (VR) can completely immerse users in a virtual world and provide little awareness of bystanders in the surrounding physical environment. Current technologies use predefined guardian area visualizations to set safety boundaries for VR interactions. However, bystanders cannot perceive these boundaries and may collide with VR users if they accidentally enter guardian areas. In this paper, we investigate four awareness techniques on mobile phones and smartwatches to help bystanders avoid invading guardian areas. These techniques include augmented reality boundary overlays and visual, auditory, and haptic alerts indicating bystanders' distance from guardians. Our findings suggest that the proposed techniques effectively …

Go to article

First Prev Next Last