Graphics and Human Computer Interfaces Commons^™

Synthesizing Multi-Person And Rare Pose Images For Human Pose Estimation, Liuqing Zhao, Zichen Tian, Zou Peng, Richang Hong, Qianru Sun

Research Collection School Of Computing and Information Systems

Human pose estimation (HPE) models underperform in recognizing rare poses because they suffer from data imbalance problems (i.e., there are few image samples for rare poses) in their training datasets. From a data perspective, the most intuitive solution is to synthesize data for rare poses. Specifically, the rule-based methods apply manual manipulations (such as Cutout and GridMask) to the existing data, so the limited diversity of the data constrains the model. An alternative method is to learn the underlying data distribution via deep generative models (such as ControlNet and HumanSD) and then sample “new data” from the distribution. This works …

Accessmenu: Enhancing Usability Of Online Restaurant Menus For Screen Reader Users, Nithiya Venkatraman, Akshay Kolgar Nayak, Suyog Dahal, Yash Prakash, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Online food ordering has become commonplace due to its convenience. The wide variety of culinary choices, combined with fast and economical door-delivery services, encourages more people to order food online. To facilitate this process, food vendors, including restaurants, often provide full menus on their websites, typically in visual formats such as images or PDFs. While this is convenient for sighted users, blind and visually impaired (BVI) individuals face significant challenges accessing these visual menus with their screen reader assistive technology. An interview study with 12 BVI screen reader users revealed that present assistive tools do not adequately satisfy the needs …

Automation To Autonomy: Temporal Dynamics Of Trust And Visual Attention Allocation Did Not Evolve, Tetsuya Sato, Eric Chancey, Yusuke Yamani

Psychology Faculty Publications

Emerging work environments are expected to implement autonomy that performs various functions without human input. Previous works has shown that trust in automation is negatively correlated with visual attention allocation, indicating that trust is a dynamic construct. Moreover, trust in automation and trust in autonomy appears to evolve in similar ways. However, recent work has demonstrated differences between trust in automation and trust in autonomy within Kaber’s (2018) theoretical framework (Sato et al., 2023b). Yet, it is uncertain whether the development of trust and visual attention allocation differs between automation and autonomy. The present study examined the temporal dynamics of …

The Interacting Roles Of Attention Allocation And Trust In Highly Automated Aam Environments, Yusuke Yamani

Psychology Faculty Publications

[First slide]

Mechanisms of attentive visual processing

• Attention control
• Visual search
• Eye movement
• Aging and individual differences

Limits of human performance in applied environment

• Complex displays
• Machine operation
• Surface transportation
• Advanced air mobility
• Nuclear operation

Methods to ameliorate human cognitive performance

• Human-machine interface
• Human autonomy/AI teaming
• Human-systems integration
• Training

Human Perception Of Ai Capabilities At Classifying Perturbed Roadway Signs, Katherine R. Garcia, Jing Chen, Yanru Xiao, Scott Mishler, Cong Wang, Bin Hu

Computer Science Faculty Publications

Artificial Intelligence (AI) is crucial to numerous functions required for driving automation systems, including the computer vision techniques used to detect the roadway environment and make real-time decisions. However, the images used as inputs to the AI system may be maliciously perturbed, or manipulated, causing the AI system to make an incorrect classification. In this study, we examined humans’ perception of the AI’s computer vision capability of classifying various road sign images, including the original images, images with two different types of malicious attacks, and images that are scrambled randomly at the pixel level. Our results showed that participants rated …

A Real-Time Approach To Capture Ambient And Focal Attention In Visual Search, Gavindya Jayawardena, Yasith Jayawardana, Yasasi Abeysinghe, Bhanuka Mahanama, Sampath Jayarathna, Jacek Gwizdka

Computer Science Faculty Publications

During visual search, individuals’ attention shifts between ambient and focal states in response to task demands and stimuli. The ambient/focal coefficient K is a statistically validated measure of these states, computed offline from fixation duration and saccade amplitude data. While current methods compute K offline, real-time computation could enable applications such as monitoring user attention, creating attention-adaptive user interfaces, and optimizing graphics rendering. However, real-time computation of K requires stable estimates for the parameters of fixation duration and saccade amplitude distributions. Since these distributions are heavy-tailed, the real-time estimates exhibit high variance and slow convergence. To overcome this, we propose …

Optiselect And Enshap: Integrating Machine Learning And Game Theory For Ischemic Stroke Prediction, Pritam Chakraborty, Anjan Bandyopadhyay, Sricheta Parul, Sujata Swain, Partha Sarathy Banerjee, Tapas Si, Hong Qin, Saurav Mallik

Computer Science Faculty Publications

Stroke analysis using game theory and machine learning techniques. The study investigates the use of the Shapley value in predictive ischemic brain stroke analysis. Initially, preference algorithms identify the most important features in various machine learning models, including logistic regression, K-nearest neighbor, decision tree, support vector machine (linear kernel), support vector machine ( RBF kernel), neural networks, etc. For each sample, the top 3, 4, and 5 features are evaluated and selected to evaluate their performance. The Shapley value method was used to rank the models using their best four features based on their predictive capabilities. As a result, better-performing …

Insights In Adaptation: Examining Self-Reflection Strategies Of Job Seekers With Visual Impairments In India, Akshay Kolgar Nayak, Yash Prakash, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Significant changes in the digital employment landscape, driven by rapid technological advancements and the COVID-19 pandemic, have introduced new opportunities for blind and visually impaired (BVI) individuals in developing countries like India. However, a significant portion of the BVI population in India remains unemployed despite extensive accessibility advancements and job search interventions. Therefore, we conducted semi-structured interviews with 20 BVI persons who were either pursuing or recently sought employment in the digital industry. Our findings reveal that despite gaining digital literacy and extensive training, BVI individuals struggle to meet industry requirements for fulfilling job openings. While they engage in self-reflection …

Adapting Online Customer Reviews For Blind Users: A Case Study Of Restaurant Reviews, Mohan Sunkara, Akshay Kolgar Nayak, Sandeep Kalari, Yash Prakash, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Online reviews have become an integral aspect of consumer decision-making on e-commerce websites, especially in the restaurant industry. Unlike sighted users who can visually skim through the reviews, perusing reviews remains challenging for blind users, who rely on screen reader assistive technology that supports predominantly one-dimensional narration of content via keyboard shortcuts. In an interview study, we uncovered numerous pain points of blind screen reader users with online restaurant reviews, notably, the listening fatigue and frustration after going through only the first few reviews. To address these issues, we developed QuickCue assistive tool that performs aspect-focused sentiment-driven summarization to reorganize …

An Optimized Generalized Multi-Color Point Implicit Solver For Intel Gpus Using Oneapi Esimd, Joseph Wassell, Mohammad Zubair, Aaron Walden, Gabriel Nastac, Eric Nielsen, Timothée Ewart

Computer Science Faculty Publications

This paper presents an efficient implementation of a linear-solver kernel relevant to FUN3D, a suite of computational fluid dynamics software developed at NASA’s Langley Research Center. The linear solver is optimized for a range of block sizes commonly used in FUN3D. The implementation targets Aurora, the Argonne Leadership Computing Facility’s (ALCF) exascale machine featuring Intel Data Center Max 1550 GPUs. The linear solver’s performance is memory bandwidth-bound due to its low arithmetic intensity. The primary performance challenges stem from variable matrix row lengths and indirect memory access patterns inherent in unstructured-grid applications. Variable block sizes introduce additional complexity through differing …

Surveying The Role Of Visual Analytics In Human-Machine Teaming, Naga Datha Saikiran Battula

Theses

Humans and machines both possess their unique capabilities and have their strengths and weaknesses, which can be complementary to one another and allow them to achieve a common goal. Teaming in the modern era involves text prompts, voice commands, gesture recognition, touch interfaces, and the latest visualization techniques that allow parties/agents to interact. Communication through visualization plays a vital role in allowing robust insights to be gained through a glance. Using visualization as a medium between humans and machines can increase the communication bandwidth. Human-machine teaming has witnessed much progress, with many theories and practical examples emerging. In the report, …

La Creatividad En Peligro: Como La Inteligencia Artificial Es Un Reto Para Los Artistas., Nathaly Cisneros

Capstones

Los artistas digitales han creado obras maestras que nos han dejado sin aliento con sus pinceles digitales, lápices y pinturas. Desde retratos que parecen saltar de la pantalla hasta paisajes que nos transportan a mundos desconocidos, su arte ha sido una fuente constante de inspiración.

Pero en los últimos años, una nueva fuerza ha comenzado a cambiar el juego. La inteligencia artificial ha estado avanzando a pasos agigantados y ahora se perfila como una amenaza para el futuro de los artistas digitales. ¿Qué significa esto para el arte y la creatividad?

Link: https://docs.google.com/document/d/1xe8UxDMekX_SwiIppyt_JppK8M-lB-YWNWGyeyShlJM/edit?usp=sharing

Real-Time Feedback-Driven Framework For Automated Cybersickness Mitigation, Md Jahirul Islam

Master's Theses

As technologies are becoming more advanced day by day, the embracement of virtual reality (VR) technology among users is also increasing in daily activities for various purposes, and subsequently, the barrier between the real and virtual world is fading. Despite the versatile uses, cybersickness (CS) is a major problem which is induced among users due to the immersive VR experience. There is a plethora of research findings and methods to measure the users’ CS such as virtual reality sickness questionnaire (VRSQ), simulator sickness questionnaire (SSQ), fast motion scale questionnaire (FMS), and others. Recently, machine learning approaches have also been adopted …

On The Benefits Of Directness In Virtual Characters For Motivational Interviews, Michael O'Mahony, Cathy Ennis, Robert Ross

Conference papers

Understanding the factors influencing successful engagement with Embodied Conversational Agents (ECAs) remains a significant challenge. This understanding could be used to personalise agents to users to improve interactions. Some studies have shown that simulating personalities in healthcare agents can improve effectiveness and engagement. However, it is not yet well understood how variations of agent personality can be leveraged to improve user engagement with Motivational Interviewing (MI) ECAs. Specifically how the balance between agent warmth and directness can be controlled in an MI agent to improve likeability and engagement. We conducted an online Wizard-of-Oz (WoZ) mediated study of two variants of …

Visualization Of Paleocurrents On A Web Application Using Gplates, Anjan Sapkota

MS in Computer Science Theses

Paleocurrents are flow directions derived from features of sedimentary rocks that reveal the direction of the current of wind or water that deposited the sediment. In 2015, Brand et al. created a global database of paleocurrents, which contains over 1,000,000 measurements worldwide: North America, South America, Australia, Great Britain, parts of Western Europe, China, Africa are fairly well represented; Antarctica, Eastern Europe, and Asia are modestly represented and Russia is poorly represented. The contribution of this thesis is a web application that uses the GPlates’ Application Programming Interface (API) to visualize global paleocurrents through time in an interactive way based …

Using Llms To Establish Implicit User Sentiment Of Software Desirability, Sherri Weitl-Harms, John D. Hastings, Jonah Lum

Research & Publications

This study explores the use of LLMs for providing quantitative zero-shot sentiment analysis of implicit software desirability, addressing a critical challenge in product evaluation where traditional review scores, though convenient, fail to capture the richness of qualitative user feedback. Innovations include establishing a method that 1) works with qualitative user experience data without the need for explicit review scores, 2) focuses on implicit user satisfaction, and 3) provides scaled numerical sentiment analysis, offering a more nuanced understanding of user sentiment, instead of simply classifying sentiment as positive, neutral, or negative.

Data is collected using the Microsoft Product Desirability Toolkit (PDT), …

Real-Time Motion Augmentation And Synthesis For Animating The Hands And Eyes Of Virtual Humans And Avatars, Ryan Canales

All Dissertations

Virtual Reality (VR) enables users to interact within virtual worlds via an embodied virtual representation of themselves called an “avatar”. Because avatars are essential for immersive experiences, it is important to consider how altering or augmenting avatar motion affects virtual experiences. This dissertation aims to improve virtual experiences by addressing some of the many challenges in animating avatars and virtual humans.

In our first study, we addressed the lack of tactile feedback during virtual grasping by using visual feedback techniques. We augmented the avatar’s hand motion to remain outside virtual objects (“outer hand”) even when the user’s hand penetrated them. …

Shifting Perspectives With Procedural Modeling Techniques And Vertex Animated Textures For Real-Time Interactive Morphing, Stephanie Schulze

All Theses

As humans experience reality, they intake external stimuli using sensory receptors to process information, forming a perception. Perceptions are subjective and shape each individual’s reality. Naturally then, by changing one’s perceptions, their experience of reality is altered, for better or worse. To emphasize the idea that perception is malleable and encourage a mindset of questioning alternative ways to look at a situation, an interactive experience in Unreal Engine is developed where a dull cityscape morphs and transforms into a surreal oversized nature scene. Procedural modeling techniques are used to create a variety of morphing assets, while Vertex Animated Textures are …

Innovations In Full-Stack Web Development: Front-End To Back-End, Yassine Chahid, Patrick Slattery

Publications and Research

This research explores emerging technologies within full-stack web development and their potential impact on current front-end and back-end solutions. Both areas employ crucial technologies that determine how end-users access information and navigate web services. Front-end solutions include HTML, JavaScript, and CSS which shape user interaction on websites. Back-end solutions use technologies such as SQL and PHP for the foundation of data processing, retrieval, and storage. The research method involves examining official documentation for these technologies to better understand their key components and to understand how their use in cyberspace has changed. The research will observe several high-traffic websites and domains …

Mvgamba : Unify 3d Content Generation As State Space Sequence Modeling, Xuanyu Yi, Zike Wu, Qiuhong Shen, Qingshan Xu, Pan Zhou, Joo-Hwee Lim, Shuicheng Yan, Xinchao Wang, Hanwang Zhang

Research Collection School Of Computing and Information Systems

Recent 3D large reconstruction models (LRMs) can generate high-quality 3D content in sub-seconds by integrating multi-view diffusion models with scalable multi-view reconstructors. Current works further leverage 3D Gaussian Splatting as 3D representation for improved visual quality and rendering efficiency. However, we observe that existing Gaussian reconstruction models often suffer from multi-view inconsistency and blurred textures. We attribute this to the compromise of multi-view information propagation in favor of adopting powerful yet computationally intensive architectures (e.g., Transformers). To address this issue, we introduce MVGamba, a general and lightweight Gaussian reconstruction model featuring a multi-view Gaussian reconstructor based on the RNN-like State …