Artificial Intelligence and Robotics Commons^™

Interoperability Model And Application Of Military Training System For Combination Of Virtuality And Reality, Jianxing Gong, Hai Hu, Haihui Ren, Ruixiang Wu

Journal of System Simulation

Abstract: With the development of AI technology, VR technology, and combat simulation technology, in order to achieve the practical training effect of "how to fight and how to train soldiers", virtual and real training has become a widely popular military training mode. It has become a trend to integrate digital systems, virtual equipment, semi-physical models, physical models, and other heterogeneous systems to carry out training in the same training environment. Therefore, it is necessary to study the interoperability model and application of training systems for the combination of virtuality and reality. This paper proposed the definition of interoperability between virtuality …

Simulating Chill: Exploring The Cognitive And Therapeutic Potential Of Cold Vr Environments, Jessica Turner, Piper Hutson, James Hutson

Faculty Scholarship

This study investigates the cognitive and therapeutic potential of immersive virtual reality (VR) environments designed to simulate cold conditions. Through the engagement of participants through multisensory stimuli—including vivid visual representations of the Athabasca Glacier, auditory effects of icy winds, and corresponding haptic feedback—the research evaluates neurological and physiological responses associated with attention, emotional regulation, and stress modulation. Participants experienced virtual scenarios featuring icy winds and snow, activating specific neurological pathways involving the occipital lobe, primary visual cortex, superior colliculus, and insula, thus reinforcing sensory integration. Through predictive coding, the anterior insula and hypothalamus were engaged, prompting thermoregulatory simulations and subconscious …

International Expert Consensus On The Current Status And Future Prospects Of Artificial Intelligence In Metabolic And Bariatric Surgery, Mohammad Kermansaravi, Sonja Chiappetta, Shahab Shahabi Shahmiri, Julian Varas, Chetan Parmar, Yung Lee, Jerry T. Dang, Asim Shabbir, Daniel Hashimoto, Amir Hossein Davarpanah Jazi, Ozanan R. Meireles, Edo Aarts, Hazem Almomani, Aayad Alqahtani, Ali Aminian, Estuardo Behrens, Dieter Birk, Felipe J. Cantu, Ricardo V. Cohen, Maurizio De Luca, Nicola Di Lorenzo, Bruno Dillemans, Mohamad Hayssam Elfawal, Daniel Moritz Felsenreich, Michel Gagner, Hector Gabriel Galvan, Carlos Galvani, Khaled Gawdat, Omar M. Ghanem, Et Al

School of Medicine Faculty Publications

Artificial intelligence (AI) is transforming the landscape of medicine, including surgical science and practice. The evolution of AI from rule-based systems to advanced machine learning and deep learning algorithms has opened new avenues for its application in metabolic and bariatric surgery (MBS). AI has the potential to enhance various aspects of MBS, including education and training, decision-making, procedure planning, cost and time efficiency, optimization of surgical techniques, outcome and complication prediction, patient education, and access to care. However, concerns persist regarding the reliability of AI-generated decisions and associated ethical considerations. This study aims to establish a consensus on the role …

Analyzing Visual Attention In Virtual Crime Scene Investigations Using Eye-Tracking And Vr: Insights For Cognitive Modeling, Wen-Chao Yang, Chih-Hung Shih, Jiajun Jiang, Sergio Pallas Enguita, Chung-Hao Chen

Electrical & Computer Engineering Faculty Publications

Understanding human perceptual strategies in high-stakes environments, such as crime scene investigations, is essential for developing cognitive models that reflect expert decision-making. This study presents an immersive experimental framework that utilizes virtual reality (VR) and eye-tracking technologies to capture and analyze visual attention during simulated forensic tasks. A 360° panoramic crime scene, constructed using the Nikon KeyMission 360 camera, was integrated into a VR system with HTC Vive and Tobii Pro eye-tracking components. A total of 46 undergraduate students aged 19 to 24–23, from the National University of Singapore in Singapore and 23 from the Central Police University in Taiwan—participated …

Technoculture And Language Models In Archaeology: Reconstructing And Preserving Cultural Narratives Through Digital Humanities, James Hutson

Faculty Scholarship

Technoculture, which examines the intersection of culture and technology, has increasingly permeated archaeological practice, transforming both scholarly research and public engagement [1-3]. The introduction of digital tools such as virtual reality (VR), geographic information systems (GIS), and large language models (LLMs) has democratized access to archaeological knowledge, enabling communities to engage more actively with their cultural heritage [4-6]. This short article explores the mutual influence of technocultural studies and AI technologies on archaeology, with a focus on the preservation and reconstruction of cultural narratives through digital means.

The first aspect of this intersection lies in how technocultural tools are creating …

Do Realistic Avatars Make Virtual Reality Better? Examining Human-Like Avatars For Vr Social Interactions, Alan D. Fraser, Isabella Branson, Ross C. Hollett, Craig P. Speelman, Shane L. Rogers

Research outputs 2022 to 2026

No abstract provided.

How People Prompt Generative Ai To Create Interactive Vr Scenes, Setareh Aghel Manesh, Tianyi Zhang, Yuki Onishi, Kotaro Hara, Scott Bateman, Jiannan Li, Anthony Tang

Research Collection School Of Computing and Information Systems

Generative AI tools can provide people with the ability to create virtual environments and scenes with natural language prompts. Yet, how people will formulate such prompts is unclear---particularly when they inhabit the environment that they are designing. For instance, it is likely that a person might say, "Put a chair here,'' while pointing at a location. If such linguistic and embodied features are common to people's prompts, we need to tune models to accommodate them. In this work, we present a Wizard of Oz elicitation study with 22 participants, where we studied people's implicit expectations when verbally prompting such programming …

Toward Intuitive 3d Interactions In Virtual Reality: A Deep Learning- Based Dual-Hand Gesture Recognition Approach, Trudi Di Qi, Franceli L. Cibrian, Meghna Raswan, Tyler Kay, Hector M. Camarillo-Abad, Yuxin Wen

Engineering Faculty Articles and Research

Dual-hand gesture recognition is crucial for intuitive 3D interactions in virtual reality (VR), allowing the user to interact with virtual objects naturally through gestures using both handheld controllers. While deep learning and sensor-based technology have proven effective in recognizing single-hand gestures for 3D interactions, research on dual-hand gesture recognition for VR interactions is still underexplored. In this work, we introduce CWT-CNN-TCN, a novel deep learning model that combines a 2D Convolution Neural Network (CNN) with Continuous Wavelet Transformation (CWT) and a Temporal Convolution Network (TCN). This model can simultaneously extract features from the time-frequency domain and capture long-term dependencies using …

Automated Cinematographer For Vr Viewing Experiences, Zihan Wu

Dartmouth College Master’s Theses

As the virtual reality (VR) industry continues to evolve, the question of how to effectively capture VR experiences for an audience remains a challenge. The predominant method of showcasing VR applications through first-person recordings lacks cinematic interest, failing to capture other viewpoints and the essence of the moment. Meanwhile, manually setting up cameras and editing videos requires technical expertise on behalf of the user. In this paper, we propose the use of machine learning (ML) to automatically select the most compelling predefined viewpoint in a VR environment, at any given moment. Our models, trained on actor motion and voice volume, …

Creative Technologies: A Conversation With Roy Magnuson, Roy Magnuson, Maureen Russell

Faculty Publications - Music

[In lieu of an abstract, the introduction is provided.] Today I am speaking with Roy Magnuson, Associate Professor Creative Technologies in the School of Music at Illinois State University (ISU). (see Figure 1) His music has been performed throughout the United States and Europe at venues such as the World Saxophone Congress, WASBE, CBDNA, the RED NOTE New Music Festival, and the Robb Composers’ Symposium. Magnuson is also the creator of the virtual reality composition software solsticeVR and the conducting software RibbonsVR. He is a member of ASCAP, and his music is recorded on Albany Records and NAXOS.

Discourses Across Periods Of Time, Amanda Raquel Moreno

Theses

This literature review explores the revolutionary effect of generative artificial intelligence (AI) and virtual reality (VR) on digital art history, specifically concentrating on their capacity to enable dialogical exchanges with historical figures and deepen the understanding of artworks. This study considers the current state of research, detecting key methodologies, areas of improvement, and possible challenges and ethical concerns. The example historical figure used in this analysis is the iconic Mexican artist Frida Kahlo. Kahlo’s refusal to correspond to a specific artistic style makes her an ideal subject for generative AI and VR-based investigation, offering fresh insights into her work. The …

Military Metaverse: Key Technologies, Potential Applications And Future Directions, Zhao Zhang, Yujie Guo, Xiaoning Zhao, Baoliang Sun

Journal of System Simulation

Abstract: Since its emergence, the concept of metaverse has been applied to many fields. At present, the transformation of military intelligence, digitalization and information technology is advancing in an allround way. The military enabled by metaverse technology will accelerate the process of military reform in the new era. To explore the potential application of metaverse in the military field, this paper first introduces several key technologies of metaverse and their functions in the military field, and then discusses some potential directions for the application of metaverse in weapon development and support, training and teaching of officers and soldiers, tactical command …

Modeling And System Realization Of Assembly Robot Based On Digital Twin, Jian Xu, Xin Song, Xiuping Liu, Bo Chen

Journal of System Simulation

Abstract: Aiming at the problems of time-consuming, labor-intensive and poor accuracy in programming of complex operations of industrial assembly robots, an online programming method for robots based on digital twin is proposed. From the four dimensions of geometry, contact dynamics, behavior and rules, the faithful mapping of the robot from real to virtual space is realized. The verification of multi-part assembly trajectory planning program, real-time synchronous operation and state monitoring are realized. An assembly robot modeling and online programming system based on digital twin is designed and built. A six-axis industrial robot cell is taken as an example to verify …

Research And Development Of Immersive Aero-Engine Scene Simulation System, Shun Yao, Zhongzhi Hu, Wenyu Cao, Jiali Yang

Journal of System Simulation

The research and development of aero-engines has the characteristics of high precision and interdiscipline. In order to reduce communication costs and to display the engine structure and the state of semi-physical simulator, by applying virtual reality technology,an immersive scene simulation system is built. By studying CAD data lightweight technology and physics-based real-time rendering technology,a rendering optimization method for similar object dynamic batching is proposed, which effectively improves the rendering frame rate. A dynamic parallax adjustment algorithm is proposed to solve the problem of dizziness when having a close look to stereoscopic images. The system achieves the …

Enhancing Pedestrian-Autonomous Vehicle Safety In Low Visibility Scenarios: A Comprehensive Simulation Method, Zizheng Yan, Yang Liu, Hong Yang

Modeling, Simulation and Visualization Student Capstone Conference

Self-driving cars raise safety concerns, particularly regarding pedestrian interactions. Current research lacks a systematic understanding of these interactions in diverse scenarios. Autonomous Vehicle (AV) performance can vary due to perception accuracy, algorithm reliability, and environmental dynamics. This study examines AV-pedestrian safety issues, focusing on low visibility conditions, using a co-simulation framework combining virtual reality and an autonomous driving simulator. 40 experiments were conducted, extracting surrogate safety measures (SSMs) from AV and pedestrian trajectories. The results indicate that low visibility can impair AV performance, increasing conflict risks for pedestrians. AV algorithms may require further enhancements and validations for consistent safety performance …

A Proposed Meta-Reality Immersive Development Pipeline: Generative Ai Models And Extended Reality (Xr) Content For The Metaverse, Jay Ratican, James Hutson, Andrew Wright

Faculty Scholarship

The realization of an interoperable and scalable virtual platform, currently known as the “metaverse,” is inevitable, but many technological challenges need to be overcome first. With the metaverse still in a nascent phase, research currently indicates that building a new 3D social environment capable of interoperable avatars and digital transactions will represent most of the initial investment in time and capital. The return on investment, however, is worth the financial risk for firms like Meta, Google, and Apple. While the current virtual space of the metaverse is worth $6.30 billion, that is expected to grow to $84.09 billion by the …

Murder On The Vr Express: Studying The Impact Of Thought Experiments At A Distance In Virtual Reality, Andrew Kissel, Krzysztof J. Rechowicz, John B. Shull

Philosophy Faculty Publications

Hypothetical thought experiments allow researchers to gain insights into widespread moral intuitions and provide opportunities for individuals to explore their moral commitments. Previous thought experiment studies in virtual reality (VR) required participants to come to an on-site laboratory, which possibly restricted the study population, introduced an observer effect, and made internal reflection on the participants’ part more difficult. These shortcomings are particularly crucial today, as results from such studies are increasingly impacting the development of artificial intelligence systems, self-driving cars, and other technologies. This paper explores the viability of deploying thought experiments in commercially available in-home VR headsets. We conducted …

Vr Computing Lab: An Immersive Classroom For Computing Learning, Shawn Pang, Kyong Jin Shim, Yi Meng Lau, Swapna Gottipati

Research Collection School Of Computing and Information Systems

In recent years, virtual reality (VR) is gaining popularity amongst educators and learners. If a picture is worth a thousand words, a VR session is worth a trillion words. VR technology completely immerses users with an experience that transports them into a simulated world. Universities across the United States, United Kingdom, and other countries have already started using VR for higher education in areas such as medicine, business, architecture, vocational training, social work, virtual field trips, virtual campuses, helping students with special needs, and many more. In this paper, we propose a novel VR platform learning framework which maps elements …

Expressiveness Of Real-Time Motion Captured Avatars Influences Perceived Animation Realism And Perceived Quality Of Social Interaction In Virtual Reality, Alan D. Fraser, Isabella Branson, Ross C. Hollett, Craig P. Speelman, Shane L. Rogers

Research outputs 2022 to 2026

Using motion capture to enhance the realism of social interaction in virtual reality (VR) is growing in popularity. However, the impact of different levels of avatar expressiveness on the user experience is not well understood. In the present study we manipulated levels of face and body expressiveness of avatars while investigating participant perceptions of animation realism and interaction quality when disclosing positive and negative experiences in VR. Moderate positive associations were observed between perceptions of animation realism and interaction quality. Post-experiment questions revealed that many of our participants (approximately 40 %) indicated the avatar with the highest face and body …

A Real-Time Ultrasound Simulation Platform Using Ray Tracing And Its Integration With Virtual Reality, Bo Peng, Qiang Wang, Ruibing Qing, Lixue Yin, Jingfeng Jiang

Journal of System Simulation

Abstract: In order to further improve the efficacy of ultrasound training and reduce the cost. An ultrasound training system that is integrated with a VR environment is developed. The main contribution of this study is to incorporate the Ray-tracing based ultrasound image simulation approach into a virtual reality environment, taking advantage of immersive VR experience for medical ultrasound training. The simulated ultrasound images obtained by the proposed method are then compared to images that are simulated using a generative adversarial network (GAN) technique and Field II ultrasound simulator. The data show that the ultrasound simulator can produce high-quality simulated …

Multi-Sensory Fusion Method For Power Transformer Virtual Assembly, Xuqiang Shao, Haowei Zhang, Xiaohua Feng

Journal of System Simulation

Abstract: Virtual assembly technology is to truly restore the equipments and physical scenarios. In the real world, people can interact with the physical world through visual, auditory, tactile and other sense organ. Aiming at the existing virtual assembly system being limited the single sense human-computer interaction mode, so a multi-sense fusion information interaction method is proposed to improve the sense of immersion and operability. An improved AABB Octree bounding box collision detection algorithm of large size diffidence component to be assembled is proposed, which can greatly reduce the amount of calculation and improve the calculation accuracy. The experimental result verifies …

Design And Realization Of 6-Dof Parachuting Simulation Training System, Xiaoguang Zhou, Peng Zhu, Yuanyuan Zhang, Huan Lu, Yuan Zhou

Journal of System Simulation

Abstract: Aiming at restoring the 6-DOF motion process of each stage during parachuting, a 6-DOF parachuting simulation training system is designed and implemented. The architecture of the simulator is designed, and the realization of the sub-systems such as the motion calculation, 6-DOF motion platform, control loading system, virtual reality scene, somatosensory system and management console is explained. Compared with the same type of parachute simulator, this system has introduced a 6-DOF motion platform which can drive the trainees to simulate the various postures of parachuting. It can also help the trainees master the control methods of parachute and enhance the …

Research On Immersive Virtual Reality Interactive System For Flow Visualization, Shijian Xu, Dan Zhao, Chengyu Su, Fupan Wang, Xiaorong Zhang, Fang Wang, Yadong Wu

Journal of System Simulation

Abstract: Aiming at the problem of low interaction efficiency due to the lack of depth information in the current desktop flow visualization application based on the two-dimensional display interaction environment, virtual reality technology is introduced to design and implement an immersive flow visualization system with diversified interaction methods. Based on the gaze and gesture interaction technology, the system designs and implements gesture-based navigation methods and three-dimensional interactive widgets based on user requirements, as well as immersive interactive panels based on user gaze and gestures. We designed and implemented an immersive interactive management method based on visual scenes and a visual …

Warehouse And Logistics: Smart Picking With Vuzix Smart Glasses, Elise Hemink

Frameless

Vuzix is an industry leader in augmented reality (AR) technology. We provide innovative products to an array of industries, a few being defense, security, enterprise, and consumers. Our AR technology provides a perfect balance of engagement in the digital and real worlds thanks to their innovative optics, AI apps and 5G capability.

Whole Ship Simulation Training Platform Based On Virtual Reality, Jiawen Sun, Hongxiang Ren, Fangbing Xiao, Xiaobin Jiang

Journal of System Simulation

Abstract: Under the current navigation education mode, it is common that trainees lack the overall navigation knowledge and the operation practice on onboard equipment and instruments. It is necessary to develop a whole ship simulation training platform that is not limited by time and sites. The six-degree freedom mathematical model of ship motion is established by using the separation modeling theory of ship maneuverability. The flexible object simulation technology, the modeling and rendering of ocean wave technology, and the reverse dynamics technology are adopted to realize the ship scene roaming and equipment interaction. The simulation results show that …

Virtual Training System For Spacecraft Maintenance Based On Desktop Virtual Reality, Shisong Wei, Zhengdong Zhou, Xuling Zhang, Mao Ling, Junshan Jia, Chuanle Liu

Journal of System Simulation

Abstract: In order to shorten the training period of aerospace maintenance personnel and improve their operation proficiency, a spacecraft maintenance simulation system based on the desktop virtual reality technology is designed and developed. It is superior to the head-mounted display, which is bulky, isolated from the real world, easy to make users dizzy and hard to be shared. The assembly management scheme based on XMind and SqlServer database is proposed to realize the visualization of assembly rule editing, which makes assembly and disassembly flexible and convenient.

Interactive Assembly Simulation Platform For Complex Gearbox In Virtual Environmen, Chunyang Jia, Xiangjun Zou, Jinhui Li, Zeqin Zeng, Wang Jie

Journal of System Simulation

Abstract: Aiming at the human-computer interaction, behavior simulation and user design in virtual assembly, based on the complex gearbox of large tractors, the simulation method of dynamic motion and behavior of gear with multi-collision body is studied. The positioning method of multi-part model based on axis constraints and surface constraints in virtual environment is proposed. The virtual environment of human-computer interaction is developed. The consumed time of per frame is about 10 ms, and the complex variable-speed motion behavior and its simulation are realized. The virtual assembly system is equipped with AR interface, which can realize the active intelligent virtual …

Research Progress On Virtual Pruning Fruit Trees, Mengmeng Lü, Shenglian Lu, Xinyu Guo

Journal of System Simulation

Abstract: The tree structure has a strong influence on fruit yield and quality, and pruning is one of the most important management measures of tree shapes with high yield and good quality. Pruning test or technology demonstration in real fruit trees are irreversible defects, high cost, and difficult to fast rendering of pruning. Virtual pruning of fruit trees, which is in this context with rapid progress of virtual reality technology has developed rapidly and becomes the hot research topic. The aim is to create a space of virtual and simulate fruit tree pruning and the influence of pruning, so as …

Study On Hand Gesture Recognition And Portfolio Optimization Model Based On Svm, Zhiwei Cai, Shuyan Wu, Junfeng Song

Journal of System Simulation

Abstract: Hand gesture recognition was researched. The idea of extracting related features was proposed by using SVM algorithm in machine learning domain, and combination optimization method was used, which consists of ANN, HMM and DTW, to do hand gesture recognition. The experimental results show that portfolio optimization model based gesture recognition method has high accuracy and is very effective.

Virtual Reality In Transportation: A Survey, Zhaohui Wu, Guo Yu, Wang Hui, Xiuna Miao

Journal of System Simulation

Abstract: The state-of-the-art of virtual reality application in the field of transportation was focused on. The novel virtual reality applications were summarized from the following six aspects: traffic simulation, accident simulation, virtual driving system, transportation achievements exhibition, behavior and psychology of traffic, virtual design and maintenance, and so on. The existing problems were discussed in the virtual reality application of transportation. The trends of virtual reality were proposed to try to solve the real traffic problems based on the transportation status of China.