Computer Engineering Commons^™

Enhancing Virtual Reality Cockpit Procedure Training With Haptic Feedback, Christian K. Jaedicke

Doctoral Dissertations and Master's Theses

Pilot training is time-consuming and costly, and not all phases require full-flight simulators or actual aircraft. Virtual reality (VR) offers a potential means to reduce cost and duration while supporting practical cockpit procedure training. While prior research in the VR domain suggests that providing haptic feedback can support training of muscular control and increase perceived immersion, it remains unclear whether haptic feedback improves spatial cognition in an unfamiliar VR environment. This study examined whether haptic feedback and cockpit complexity affected response time and position error under limited-visibility, time-constrained conditions in a VR cockpit environment (i.e., a quasi-blindfold cockpit check).

Grounded …

Machine Learning For Real-Time Body Movement Classification Using Eeg And Vr Technologies, Aiden H. Behler, Robin Ghosh

Undergraduate Research

This project investigates the feasibility of real-time full-body movement classification using electroencephalography (EEG) integrated with virtual reality (VR) technologies. The primary objective is to develop and evaluate machine learning models for predicting human body movements using EEG data alone, with the long-term goal of reducing or eliminating reliance on wearable motion trackers. Currently, several machine learning algorithms have been tested, but classification accuracy remains modest, indicating the complexity of the task. Ongoing work focuses on optimizing preprocessing, feature selection, and model architectures to improve performance. The system architecture combines synchronized neural and motion data collected within a VR environment. EEG …

Beyond Permissions: A Privacy And Security Analysis Of Sensor And Network Activity On Xr Devices Using Xrmonitor, Gayatri Sravya Siripurapu

Computer Science and Engineering Theses - Archive

Standalone virtual reality headsets integrate multiple sensors and cameras whose runtime behavior is not transparently documented, creating potential privacy and security risks that are difficult to audit using existing tools. The relationship between software feature activation and the underlying hardware engaged at runtime remains opaque, limiting the ability of users and researchers to assess what data is being collected and when.

This thesis presents XRMonitor, a unified framework for monitoring and analyzing sensor activation behavior and network traffic on Meta Quest devices. The framework is motivated by the lack of transparency in how standalone XR devices engage their hardware sensors …

Interoperability Model And Application Of Military Training System For Combination Of Virtuality And Reality, Jianxing Gong, Hai Hu, Haihui Ren, Ruixiang Wu

Journal of System Simulation

Abstract: With the development of AI technology, VR technology, and combat simulation technology, in order to achieve the practical training effect of "how to fight and how to train soldiers", virtual and real training has become a widely popular military training mode. It has become a trend to integrate digital systems, virtual equipment, semi-physical models, physical models, and other heterogeneous systems to carry out training in the same training environment. Therefore, it is necessary to study the interoperability model and application of training systems for the combination of virtuality and reality. This paper proposed the definition of interoperability between virtuality …

Llm-Powered Question Answering For Object States In Virtual Reality, Shiyi Ding

Master's Theses

Recent advances in large language models (LLMs) and multimodal large language models (MLLMs) enable natural language–based querying in virtual reality (VR). However, VR environments are highly localized, personalized, and dynamic, making it challenging for general-purpose models to answer environment-specific queries or reason about subtle object state changes. To address these challenges, this thesis develops two systems for 3D question answering in VR.

First, we present RAG-VR, the first retrieval-augmented 3D question-answering system designed for VR. RAG-VR augments an LLM with external knowledge retrieved from a localized knowledge database and includes a pipeline for extracting environmental and user-related information. To improve …

Face Off: Evaluating Virtual Human Expressions And Non-Tracking Control Methods In Vr, J K Sangeeth Chandran, Marisa Llorens Salvador, Cathy Ennis

Conference papers

Social virtual reality (VR) applications have become more ubiquitous in recent years; central to this is the communication pipeline, how users perceive virtual human facial expressions, and how they control them in real time, especially when using VR devices without face-tracking. We investigated both aspects in a set of experiments. Firstly, we compared the perception of virtual human emotions on a traditional 2D screen and in VR. In a second experiment, we used a validated set of stimuli to compare three different control methods for manipulating an avatar’s facial expressions in VR. These control methods utilize non-tracking control techniques, which …

L'Utilisation Des Technologies Numériques Dans Les Musées Contemporains Au Maroc : Les Défis De La Préservation Culturelle, De L'Accès Et De La Justice Dans L'Utilisation De La Réalité Virtuelle Et Augmentée, Fadma Fangach

Dirassat

This study investigates the impact of immersive technologies, particularly Virtual Reality (VR) and Augmented Reality (AR), on visitor experiences at two distinct museums: the Museum of Modern and Contemporary Art Mohamed the Sixth and the Exhibition and Museum of the Prophet’s Biography and Islamic Civilization. Employing a qualitative approach and a comparative case study methodology, the research examines the employment of VR and AR within each museum, focusing on their respective modes of projection and exhibition. While the Museum of Modern and Contemporary Art Mohamed the Sixth uses VR to establish a virtual space online, the Exhibition and Museum of …

Indigenous Technology Futurisms: Reclaiming Mackinac Island In Virtual Reality, Madeline A. Gupta

Library Map Prize

In Indigenous Technology Futurisms: Reclaiming Mackinac Island in Virtual Reality, Madeline A. Gupta introduces a place-based digital storytelling project that uses interactive mapping as the foundation for cultural reconnection and community wellness. Centered on Mackinac Island—ancestral land of the Anishinaabe people—the project features a custom-designed map by an Anishinaabe artist, allowing users to navigate a VR website by selecting specific locations across northern Michigan. Each map point opens into a spatial video experience paired with traditional audio content, including oral histories, poetry, and songs.

The project reframes digital cartography through an Indigenous lens, challenging Western colonial mapping practices that …

Rodcast Interaction: A Novel Technique For Dense Virtual Reality Environments, Nevzat U. Demirseren

UNF Graduate Theses and Dissertations

Virtual Reality (VR) technologies continue to grow in popularity and application versatility, yet effective interaction within complex dense environments remains as a critical challenge. In particular, users with low level of VR experience often face decreased accuracy and dissatisfaction selecting occluded objects. A variety of interaction techniques to select and manipulate objects exist, but there is a research gap in understanding what kinds of techniques support users in dense environments. This study evaluates the user performance and preference in such environments. Three interaction techniques are examined in this study: Go-Go Hand, Flower Cone, and a proposed technique called RodCast Interaction. …

Virtual Control: A Comparison Of Methods For Hand-Tracking Implementation, Nathan Roberts

Honors Program: Senior Projects (Public)

This thesis examines the design philosophy of modern virtual reality applications that utilize hand-tracking as a primary form of user input. The analysis presented hopes to provide ideas for future implementations of this technology so that more immersive experiences are developed. This analysis starts with the discussion of a modern example of successful hand-tracking implementation, then comparing that implementation to a recent senior design project. This comparison is primarily based on each experience’s ability to create interactivity and immediacy. Interactivity is the degree to which the user can quickly and reliably make changes to their virtual environment, while immediacy is …

Integrating Authentication Schemes In Augmented And Virtual Reality Classrooms, Naheem Noah

Electronic Theses and Dissertations

Augmented Reality and Virtual Reality (AR/VR) technologies are revolutionizing educational experiences, but their widespread adoption hinges on addressing critical security and usability challenges, particularly in the domain of user authentication. This research presents an investigation into the security landscape of AR/VR and explores a graphical authentication scheme called “Things” that enhances both security and usability in immersive learning environments. Through a systematic evaluation of popular AR/VR devices and applications, potential vulnerabilities and limitations were identified, such as high usage of pin/passwords which are susceptible to shoulder-surfing attacks, lack of multi-factor authentication, and unclear data-sharing practices. A review of existing knowledge-based …

Toward Intuitive 3d Interactions In Virtual Reality: A Deep Learning- Based Dual-Hand Gesture Recognition Approach, Trudi Di Qi, Franceli L. Cibrian, Meghna Raswan, Tyler Kay, Hector M. Camarillo-Abad, Yuxin Wen

Engineering Faculty Articles and Research

Dual-hand gesture recognition is crucial for intuitive 3D interactions in virtual reality (VR), allowing the user to interact with virtual objects naturally through gestures using both handheld controllers. While deep learning and sensor-based technology have proven effective in recognizing single-hand gestures for 3D interactions, research on dual-hand gesture recognition for VR interactions is still underexplored. In this work, we introduce CWT-CNN-TCN, a novel deep learning model that combines a 2D Convolution Neural Network (CNN) with Continuous Wavelet Transformation (CWT) and a Temporal Convolution Network (TCN). This model can simultaneously extract features from the time-frequency domain and capture long-term dependencies using …

Cloud Computing Integration Into Mixed-Reality: Physical To Abstraction, Yassine Chahid, Patrick Slattery

Publications and Research

This research evaluates the progression of cloud computing and mixed-reality technologies, and to identify how these technologies influence advancements in the latter. Both cloud computing and mixed reality have significantly impacted the IT field and the services available to the public and various institutions. Cloud computing provides a valuable way to process information or allocate computational resources on otherwise limited hardware. Augmented or virtual reality hardware would greatly benefit from this by offloading resource-intensive tasks to other machines. The research methodology involves analyzing essential components of both innovations, divided into multiple categories. These components range from physical, hardware-based elements to …

Empathy, Education, And Awareness: A Vr Hackathon’S Approach To Tackling Climate Change, Noor Alqallaf, Dalia W. Elnagar, Sherif G. Aly, Khalil I. Elkhodary, Rami Ghannam

Faculty Journal Articles

Climate change education is crucial for fostering informed and engaged future generations. However, traditional pedagogies often fail to engage learners fully and provide real-world, experiential learning. This paper presents a novel approach to climate change education through a three-day virtual reality (VR) hackathon. The hackathon focused on four United Nations (UN) Sustainable Development Goals (SDGs)—Quality Education, Affordable and Clean Energy, Sustainable Cities and Communities, and Climate Action. Using VR technology and game design software, engineering students worked in teams. They competed against each other in designing immersive environments that demonstrated their understanding of these SDGs and climate change. Our goal …

Bringing "Virtual" To "Reality": Enhancing Security And Usability On Vr System And Applications, Huadi Zhu

Computer Science and Engineering Dissertations - Archive

With the rapid advancements in computer science, electronics, optics, and related fields, virtual reality (VR) gradually penetrates into our daily lives, and is predicted to become a core technology in the near future. Despite its potentials, however, existing designs and solutions for VR applications remain at the infant stage, introducing limited usability and efficiency for real-world users. Besides, the increasing prevalence of VR presents new security and privacy threats due to the vast amount of information stored in or accessible through VR devices. To bridge this gap, we exploit and combine techniques from computer science and human biology, as well …

Military Metaverse: Key Technologies, Potential Applications And Future Directions, Zhao Zhang, Yujie Guo, Xiaoning Zhao, Baoliang Sun

Journal of System Simulation

Abstract: Since its emergence, the concept of metaverse has been applied to many fields. At present, the transformation of military intelligence, digitalization and information technology is advancing in an allround way. The military enabled by metaverse technology will accelerate the process of military reform in the new era. To explore the potential application of metaverse in the military field, this paper first introduces several key technologies of metaverse and their functions in the military field, and then discusses some potential directions for the application of metaverse in weapon development and support, training and teaching of officers and soldiers, tactical command …

Modeling And System Realization Of Assembly Robot Based On Digital Twin, Jian Xu, Xin Song, Xiuping Liu, Bo Chen

Journal of System Simulation

Abstract: Aiming at the problems of time-consuming, labor-intensive and poor accuracy in programming of complex operations of industrial assembly robots, an online programming method for robots based on digital twin is proposed. From the four dimensions of geometry, contact dynamics, behavior and rules, the faithful mapping of the robot from real to virtual space is realized. The verification of multi-part assembly trajectory planning program, real-time synchronous operation and state monitoring are realized. An assembly robot modeling and online programming system based on digital twin is designed and built. A six-axis industrial robot cell is taken as an example to verify …

An Advanced Simulation Architecture For Testing Autonomous And Connected Vehicles Enabled By Virtual Reality, Defu Cui

Electrical & Computer Engineering Theses & Dissertations

With the advancement of intelligent transportation systems, autonomous driving and connected driving are seen as potential solutions to alleviate traffic congestion, enhance traffic safety, and improve efficiency. Extensive testing and validation of autonomous vehicles (AVs) and connected vehicles (CVs) including connected autonomous vehicles are crucial to ensure their safety and reliability. However, testing and validating AVs and CVs on public roads faces challenges such as high costs, inadequate support from transportation infrastructure with communication technologies, and safety concerns, among others. Simulations have become essential tools for testing autonomous driving and connected driving. As mixed traffic involves multiple domains including traffic …

Research And Development Of Immersive Aero-Engine Scene Simulation System, Shun Yao, Zhongzhi Hu, Wenyu Cao, Jiali Yang

Journal of System Simulation

The research and development of aero-engines has the characteristics of high precision and interdiscipline. In order to reduce communication costs and to display the engine structure and the state of semi-physical simulator, by applying virtual reality technology,an immersive scene simulation system is built. By studying CAD data lightweight technology and physics-based real-time rendering technology,a rendering optimization method for similar object dynamic batching is proposed, which effectively improves the rendering frame rate. A dynamic parallax adjustment algorithm is proposed to solve the problem of dizziness when having a close look to stereoscopic images. The system achieves the …

Enhancing The Battleverse: The People’S Liberation Army’S Digital Twin Strategy, Joshua Baughman

Military Cyber Affairs

No abstract provided.

The Importance Of Hand Motions For Communication And Interaction In Virtual Reality, Alex Adkins

All Dissertations

Virtual reality (VR) is a growing method of communication and play. Recent advances have enabled hand-tracking technologies for consumer VR headsets, allowing virtual hands to mimic a user's real hand movements in real-time. A growing number of users now utilize hand-tracking when using VR to manipulate objects or to create gestures when interacting with others. As VR grows as a tool and communication platform, it is important to understand how the rising prevalence of hand-tracking technology might affect users' experiences.

The goal of this dissertation is to investigate, through a series of experiments, how using hand motions in VR influences …

A Real-Time Ultrasound Simulation Platform Using Ray Tracing And Its Integration With Virtual Reality, Bo Peng, Qiang Wang, Ruibing Qing, Lixue Yin, Jingfeng Jiang

Journal of System Simulation

Abstract: In order to further improve the efficacy of ultrasound training and reduce the cost. An ultrasound training system that is integrated with a VR environment is developed. The main contribution of this study is to incorporate the Ray-tracing based ultrasound image simulation approach into a virtual reality environment, taking advantage of immersive VR experience for medical ultrasound training. The simulated ultrasound images obtained by the proposed method are then compared to images that are simulated using a generative adversarial network (GAN) technique and Field II ultrasound simulator. The data show that the ultrasound simulator can produce high-quality simulated …

Multi-Sensory Fusion Method For Power Transformer Virtual Assembly, Xuqiang Shao, Haowei Zhang, Xiaohua Feng

Journal of System Simulation

Abstract: Virtual assembly technology is to truly restore the equipments and physical scenarios. In the real world, people can interact with the physical world through visual, auditory, tactile and other sense organ. Aiming at the existing virtual assembly system being limited the single sense human-computer interaction mode, so a multi-sense fusion information interaction method is proposed to improve the sense of immersion and operability. An improved AABB Octree bounding box collision detection algorithm of large size diffidence component to be assembled is proposed, which can greatly reduce the amount of calculation and improve the calculation accuracy. The experimental result verifies …

Design And Realization Of 6-Dof Parachuting Simulation Training System, Xiaoguang Zhou, Peng Zhu, Yuanyuan Zhang, Huan Lu, Yuan Zhou

Journal of System Simulation

Abstract: Aiming at restoring the 6-DOF motion process of each stage during parachuting, a 6-DOF parachuting simulation training system is designed and implemented. The architecture of the simulator is designed, and the realization of the sub-systems such as the motion calculation, 6-DOF motion platform, control loading system, virtual reality scene, somatosensory system and management console is explained. Compared with the same type of parachute simulator, this system has introduced a 6-DOF motion platform which can drive the trainees to simulate the various postures of parachuting. It can also help the trainees master the control methods of parachute and enhance the …

Research On Immersive Virtual Reality Interactive System For Flow Visualization, Shijian Xu, Dan Zhao, Chengyu Su, Fupan Wang, Xiaorong Zhang, Fang Wang, Yadong Wu

Journal of System Simulation

Abstract: Aiming at the problem of low interaction efficiency due to the lack of depth information in the current desktop flow visualization application based on the two-dimensional display interaction environment, virtual reality technology is introduced to design and implement an immersive flow visualization system with diversified interaction methods. Based on the gaze and gesture interaction technology, the system designs and implements gesture-based navigation methods and three-dimensional interactive widgets based on user requirements, as well as immersive interactive panels based on user gaze and gestures. We designed and implemented an immersive interactive management method based on visual scenes and a visual …

Enter The Battleverse: China's Metaverse War, Josh Baughman

Military Cyber Affairs

No abstract provided.

Warehouse And Logistics: Smart Picking With Vuzix Smart Glasses, Elise Hemink

Frameless

Vuzix is an industry leader in augmented reality (AR) technology. We provide innovative products to an array of industries, a few being defense, security, enterprise, and consumers. Our AR technology provides a perfect balance of engagement in the digital and real worlds thanks to their innovative optics, AI apps and 5G capability.

Vr Sound Mapping: Make Sound Accessible For Dhh People In Virtual Reality Environments, Ziming Li, Roshan Peiris

Frameless

In-game audio plays an important role in enhancing the sense of reality and immersion in the gaming experience. In many games, sounds are also used to provide notifications and clues which are essential to the gameplay. However, in this case, the DHH (deaf and hard of hearing) players may fail to access the information conveyed by sounds, which degrades their gaming experience (Jain et al. 2021).

A Youthful Metaverse: Towards Designing Safe, Equitable, And Emotionally Fulfilling Social Virtual Reality Spaces For Younger Users, Divine Maloney

All Dissertations

Social virtual reality (VR) represents the modern rendition of the metaverse, this dissertation aims to fill the research gaps while highlighting trends of youth in VR. The scientific contributions of this research include 1) expanding the current HCI understanding of the social dynamics and the interactions of teens in emerging novel online digital spaces; 2) bridging two research areas that have not been widely studied in HCI, social VR and young users in social VR; and 3) generating design implications to inform the design of future social VR platforms to better support and protect teens’ online social experiences, results which …

Incorporating Virtual Reality Training In An Introductory Public Speaking Course, Kevin Kryston, Henry Goble, Allison Eden

Journal of Communication Pedagogy

This study presents the results of two studies using a virtual reality (VR) public-speaking training simulation as an instructional aid in a basic communication course. Results from the first study suggest that VR practice was associated with higher subsequent speech delivery grades in the course compared to no practice. However, VR practice did not reduce public speaking anxiety (PSA). In a follow-up study, VR practice was compared with other forms of lab-based practice including in front of a mirror and a recorded video session. All forms of lab practice (VR, mirror, or video) were associated with higher speech grades than …