Graphics and Human Computer Interfaces Commons^™

Generalizing Classification Of Pilot Workload: Transfer Learning Versus A Jepa-Inspired Transformer Architecture, Naim Barnett, Shivani Nagrecha, Morgan Glover, Clayton Harper, Justin Wilson, James Maher, Eric C. Larson

International Journal of Aviation, Aeronautics, and Aerospace

Within the context of learning, there poses difficulty when objectively measuring human performance. In this work, we investigate the evaluation of human performance via its relation to the individual's mental capacity by classification of cognitive load within the domain of aviation. By utilizing a mixed virtual and physical flight simulation environment in conjunction with biometric sensing, we create and evaluate the predictive capabilities of a Joint-Embedding Predictive Architecture (JEPA) and compare the architecture and results to traditional methods for transfer learning and domain adaptation. We find that our JEPA inspired architecture can achieve more than 70% accuracy of cognitive workload, …

Automation To Autonomy: Temporal Dynamics Of Trust And Visual Attention Allocation Did Not Evolve, Tetsuya Sato, Eric Chancey, Yusuke Yamani

Psychology Faculty Publications

Emerging work environments are expected to implement autonomy that performs various functions without human input. Previous works has shown that trust in automation is negatively correlated with visual attention allocation, indicating that trust is a dynamic construct. Moreover, trust in automation and trust in autonomy appears to evolve in similar ways. However, recent work has demonstrated differences between trust in automation and trust in autonomy within Kaber’s (2018) theoretical framework (Sato et al., 2023b). Yet, it is uncertain whether the development of trust and visual attention allocation differs between automation and autonomy. The present study examined the temporal dynamics of …

Flexible Hybrid Self-Powered Piezo-Triboelectric Nanogenerator Based On Bto-Pvdf/Pdms Nanocomposites For Human Machine Interaction, Wentao Dong, Mengyun Li, Chang Chen, Kun Xie, Jinhua Hong, Lin Yang

Civil & Environmental Engineering Faculty Publications

As flexible and wearable electronics play more and more important role in smart watches, smart glass and virtual reality, and the power supply to the wearable electronics have been revealed more attentions for long-term usage and continuous healthy monitoring. To overcome the challenge, flexible self-powered BTO-PVDF/PDMS piezoelectric-triboelectric electric hybrid generators (BPP-HNG) are developed to human gesture monitoring and human machine interaction (HMI) application without external power supply. BPP-HNG based on BTO-PVDF and PDMS films are prepared by sol-gel and spin-coating method. When the BTO content is 20 wt.%, BPP-HNG exhibits better electrical performance with an output voltage of 20.51 V. …

Insights In Adaptation: Examining Self-Reflection Strategies Of Job Seekers With Visual Impairments In India, Akshay Kolgar Nayak, Yash Prakash, Sampath Jayarathna, Hae-Na Lee, Vikas Ashok

Computer Science Faculty Publications

Significant changes in the digital employment landscape, driven by rapid technological advancements and the COVID-19 pandemic, have introduced new opportunities for blind and visually impaired (BVI) individuals in developing countries like India. However, a significant portion of the BVI population in India remains unemployed despite extensive accessibility advancements and job search interventions. Therefore, we conducted semi-structured interviews with 20 BVI persons who were either pursuing or recently sought employment in the digital industry. Our findings reveal that despite gaining digital literacy and extensive training, BVI individuals struggle to meet industry requirements for fulfilling job openings. While they engage in self-reflection …

Human Perception Of Ai Capabilities At Classifying Perturbed Roadway Signs, Katherine R. Garcia, Jing Chen, Yanru Xiao, Scott Mishler, Cong Wang, Bin Hu

Computer Science Faculty Publications

Artificial Intelligence (AI) is crucial to numerous functions required for driving automation systems, including the computer vision techniques used to detect the roadway environment and make real-time decisions. However, the images used as inputs to the AI system may be maliciously perturbed, or manipulated, causing the AI system to make an incorrect classification. In this study, we examined humans’ perception of the AI’s computer vision capability of classifying various road sign images, including the original images, images with two different types of malicious attacks, and images that are scrambled randomly at the pixel level. Our results showed that participants rated …

An Optimized Generalized Multi-Color Point Implicit Solver For Intel Gpus Using Oneapi Esimd, Joseph Wassell, Mohammad Zubair, Aaron Walden, Gabriel Nastac, Eric Nielsen, Timothée Ewart

Computer Science Faculty Publications

This paper presents an efficient implementation of a linear-solver kernel relevant to FUN3D, a suite of computational fluid dynamics software developed at NASA’s Langley Research Center. The linear solver is optimized for a range of block sizes commonly used in FUN3D. The implementation targets Aurora, the Argonne Leadership Computing Facility’s (ALCF) exascale machine featuring Intel Data Center Max 1550 GPUs. The linear solver’s performance is memory bandwidth-bound due to its low arithmetic intensity. The primary performance challenges stem from variable matrix row lengths and indirect memory access patterns inherent in unstructured-grid applications. Variable block sizes introduce additional complexity through differing …

The Interacting Roles Of Attention Allocation And Trust In Highly Automated Aam Environments, Yusuke Yamani

Psychology Faculty Publications

[First slide]

Mechanisms of attentive visual processing

• Attention control
• Visual search
• Eye movement
• Aging and individual differences

Limits of human performance in applied environment

• Complex displays
• Machine operation
• Surface transportation
• Advanced air mobility
• Nuclear operation

Methods to ameliorate human cognitive performance

• Human-machine interface
• Human autonomy/AI teaming
• Human-systems integration
• Training

Smartphone Haptics Can Uncover Differences In Touch Interactions Between Asd And Neurotypicals, Ivonne Monarca, Franceli L. Cibrian, Isabel López Hurtado, Monica Tentori

Engineering Faculty Articles and Research

Utilizing touch interactions from smartphones for gathering data and identifying digital markers for screening and monitoring neurological disorders, such as Autism Spectrum Disorder (ASD), is an emerging area of research. Smartphones provide multiple benefits for this kind of study, including unobtrusive data collection via built-in sensors, integrated haptic feedback systems, and the capability to create specialized applications. Acknowledging the significant yet understudied presence of tactile processing differences in individuals with ASD, we designed and developed Feel and Touch, a mobile game that leverages the haptic capabilities of smartphones. This game provides vibrotactile feedback in response to touch interactions and collects …

Integrating Authentication Schemes In Augmented And Virtual Reality Classrooms, Naheem Noah

Electronic Theses and Dissertations

Augmented Reality and Virtual Reality (AR/VR) technologies are revolutionizing educational experiences, but their widespread adoption hinges on addressing critical security and usability challenges, particularly in the domain of user authentication. This research presents an investigation into the security landscape of AR/VR and explores a graphical authentication scheme called “Things” that enhances both security and usability in immersive learning environments. Through a systematic evaluation of popular AR/VR devices and applications, potential vulnerabilities and limitations were identified, such as high usage of pin/passwords which are susceptible to shoulder-surfing attacks, lack of multi-factor authentication, and unclear data-sharing practices. A review of existing knowledge-based …

Exploring Healthcare Chatbot Information Presentation: Applying Hierarchical Bayesian Regression And Inductive Thematic Analysis In A Mixed Methods Study, Samuel Nelson Koscelny

All Theses

High blood pressure, also known as hypertension, significantly increases the risk of heart disease and stroke, which are leading causes of death in the United States. While contributing to over 691,000 deaths in 2021 alone in the United States (U.S.), it also imposes immense economic burden on the healthcare system, costing approximately $131 billion annually. One way to address this issue is for increased self-care behaviors and medication adherence, both of which require sufficient health literacy. Despite the importance of health literacy, 90% of U.S. adults struggle with health-related subjects. Overcoming the issues associated with health literacy requires addressing the …

Creative Insights Into Motion: Enhancing Human Activity Understanding With 3d Data Visualization And Annotation, Isaac Browen, Hector M. Camarillo-Abad, Franceli L. Cibrian, Trudi Di Qi

Engineering Faculty Articles and Research

This paper presents a novel 3D system for human motion analysis - Motion Data Visualization and Annotation (MoViAn). Designed to provide a comprehensive visual representation of 3D human motion data, MoViAn incorporates detailed visualization of gaze direction, hand movements, and object interactions, alongside an interactive interface for efficient data annotation. A user study involving eight participants indicates that MoViAn enables users to thoroughly explore and annotate human motion data, with System Usability Scale (SUS) results demonstrating a satisfactory usability level. The contribution of this paper lies in the development of an interactive and usable data analytics tool aimed at deepening …

Next-Generation Crop Monitoring Technologies: Case Studies About Edge Image Processing For Crop Monitoring And Soil Water Property Modeling Via Above-Ground Sensors, Nipuna Chamara

Dissertations and Doctoral Documents, University of Nebraska-Lincoln, 2023–

Artificial Intelligence (AI) has advanced rapidly in the past two decades. Internet of Things (IoT) technology has advanced rapidly during the last decade. Merging these two technologies has immense potential in several industries, including agriculture.

We have identified several research gaps in utilizing IoT technology in agriculture. One problem was the digital divide between rural, unconnected, or limited connected areas and urban areas for utilizing images for decision-making, which has advanced with the growth of AI. Another area for improvement was the farmers' demotivation to use in-situ soil moisture sensors for irrigation decision-making due to inherited installation difficulties. As Nebraska …

Vr Circuit Simulation With Advanced Visualization For Enhancing Comprehension In Electrical Engineering, Elliott Wolbach

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

As technology advances, the field of electrical and computer engineering continuously demands innovative tools and methodologies to facilitate effective learning and comprehension of fundamental concepts. Through a comprehensive literature review, it was discovered that there was a gap in the current research on using VR technology to effectively visualize and comprehend non-observable electrical characteristics of electronic circuits. This thesis explores the integration of Virtual Reality (VR) technology and real-time electronic circuit simulation with enhanced visualization of non-observable concepts such as voltage distribution and current flow within these circuits. The primary objective is to develop an immersive educational platform that makes …

Cloud Computing Integration Into Mixed-Reality: Physical To Abstraction, Yassine Chahid, Patrick Slattery

Publications and Research

This research evaluates the progression of cloud computing and mixed-reality technologies, and to identify how these technologies influence advancements in the latter. Both cloud computing and mixed reality have significantly impacted the IT field and the services available to the public and various institutions. Cloud computing provides a valuable way to process information or allocate computational resources on otherwise limited hardware. Augmented or virtual reality hardware would greatly benefit from this by offloading resource-intensive tasks to other machines. The research methodology involves analyzing essential components of both innovations, divided into multiple categories. These components range from physical, hardware-based elements to …

Immersive Japanese Language Learning Web Application Using Spaced Repetition, Active Recall, And An Artificial Intelligent Conversational Chat Agent Both In Voice And In Text, Marc Butler

MS in Computer Science Project Reports

In the last two decades various human language learning applications, spaced repetition software, online dictionaries, and artificial intelligent chat agents have been developed. However, there is no solution to cohesively combine these technologies into a comprehensive language learning application including skills such as speaking, typing, listening, and reading. Our contribution is to provide an immersive language learning web application to the end user which combines spaced repetition, a study technique used to review information at systematic intervals, and active recall, the process of purposely retrieving information from memory during a review session, with an artificial intelligent conversational chat agent both …

Virtual Reality & Pilot Training: Existing Technologies, Challenges & Opportunities, Tim Marron, Niall Dungan, Brian Mac Namee, Anna Donnla O'Hagan

Journal of Aviation/Aerospace Education & Research

The introduction of virtual reality (VR) to flying training has recently gained much attention, with numerous VR companies, such as Loft Dynamics and VRpilot, looking to enhance the training process. Such a considerable change to how pilots are trained is a subject that warrants careful consideration. Examining the effect that VR has on learning in other areas gives us an idea of how VR can be suitably applied to flying training. Some of the benefits offered by VR include increased safety, decreased costs, and increased environmental sustainability. Nevertheless, some challenges ahead for developers to consider are negative transfer of learning, …

Dung Dkar Cloak: Exploring Soft Interfaces For Sonic Interactions, Judit Eszter Kárpáti, Esteban De La Torre

Textile Society of America: Symposium Proceedings

The importance of crossmodal interaction within the contemporary cultural, technological and scientific panorama has evidently gained significant attention due to its remarkable advantages in creating a meaningful, interwoven, and integrated experience. The use and recontextualization of textiles in such exploratory quest into the human senses has proven to be critical. Computational science, algorithmic logic and digital devices have always been rooted and closely interwoven with textile crafts and practices. Recent technological advancements have further combined technology and textile, generating interactive textile surfaces, constructing endless possibilities for multisensorial experiences. In this presentation we will examine how we can weave a sensitive …

An Analysis Of Precision: Occlusion And Perspective Geometry’S Role In 6d Pose Estimation, Jeffrey Choate, Derek Worth, Scott Nykl, Clark N. Taylor, Brett J. Borghetti, Christine M. Schubert Kabban

Faculty Publications

Achieving precise 6 degrees of freedom (6D) pose estimation of rigid objects from color images is a critical challenge with wide-ranging applications in robotics and close-contact aircraft operations. This study investigates key techniques in the application of YOLOv5 object detection convolutional neural network (CNN) for 6D pose localization of aircraft using only color imagery. Traditional object detection labeling methods suffer from inaccuracies due to perspective geometry and being limited to visible key points. This research demonstrates that with precise labeling, a CNN can predict object features with near-pixel accuracy, effectively learning the distinct appearance of the object due to perspective …

Enhancing Research Productivity: Seamless Integration Of Personal Devices And Hpc Resources With The Cybershuttle Notebook Gateway, Yasith Jayawardana, Dimuthu Wannipurage, Eroma Abeysinghe, Suresh Marru

Computer Science Faculty Publications

Scientists often utilize personal laptops and workstations for initial research stages and turn to high-performance computing (HPC) supercomputers for compute-intensive tasks. However, seamless transitions between these environments are vital for enhancing productivity and accelerating research progress. Our paper presents the Cybershuttle Notebook Gateway, an open-source framework crafted to streamline this transition, optimize resource utilization, and reduce time-to-science for researchers. Leveraging JupyterLab, the framework extends kernel mechanics for seamless provisioning and connection to remote HPC cluster kernels. We delve into its architecture, which separates user authentication, kernel provisioning, and remote file system access. Additionally, we highlight practical capabilities like analyzing network …

‘I Know It When I See It’– Developing Quality Schedules Considering Subjective Or Unspecified Criteria, Douglas L. Moody

Publications and Research

Most timetabling problems have a given objective function to measure the quality of a solution. However, users may have a “I know it when I see it” recognition of a quality schedule, without specifying the complete basis for their judgment. In this situation, the objective function cannot be exclusively used as a solution quality measurement. This work presents an AI based approach to aid in categorizing the solution’s quality when the users have not explicitly defined all factors used in their criteria.

Reducing Food Scarcity: The Benefits Of Urban Farming, S.A. Claudell, Emilio Mejia

Journal of Nonprofit Innovation

Urban farming can enhance the lives of communities and help reduce food scarcity. This paper presents a conceptual prototype of an efficient urban farming community that can be scaled for a single apartment building or an entire community across all global geoeconomics regions, including densely populated cities and rural, developing towns and communities. When deployed in coordination with smart crop choices, local farm support, and efficient transportation then the result isn’t just sustainability, but also increasing fresh produce accessibility, optimizing nutritional value, eliminating the use of ‘forever chemicals’, reducing transportation costs, and fostering global environmental benefits.

Imagine Doris, who is …

Enhanced Privacy-Enabled Face Recognition Using Κ-Identity Optimization, Ryan Karl

Department of Electrical and Computer Engineering: Dissertations, Theses, and Student Research

Facial recognition is becoming more and more prevalent in the daily lives of the common person. Law enforcement utilizes facial recognition to find and track suspects. The newest smartphones have the ability to unlock using the user's face. Some door locks utilize facial recognition to allow correct users to enter restricted spaces. The list of applications that use facial recognition will only increase as hardware becomes more cost-effective and more computationally powerful. As this technology becomes more prevalent in our lives, it is important to understand and protect the data provided to these companies. Any data transmitted should be encrypted …

Bridging Domain Gaps For Cross-Spectrum And Long-Range Face Recognition Using Domain Adaptive Machine Learning, Cedric Armel Nimpa Fondje

Dissertations and Doctoral Documents, University of Nebraska-Lincoln, 2023–

Face recognition technology has witnessed significant advancements in recent decades, enabling its widespread adoption in various applications such as security, surveillance, and biometrics applications. However, one of the primary challenges faced by existing face recognition systems is their limited performance when presented with images from different modalities or domains( such as infrared to visible, long range to close range, nighttime to daytime, profile to f rontal, etc.) Additionally, advancements in camera sensors, analytics beyond the visible spectrum, and the increasing size of cross-modal datasets have led to a particular interest in cross-modal learning for face recognition in the biometrics and …

Demo Abstract: Vgglass - Demonstrating Visual Grounding And Localization Synergy With A Lidar-Enabled Smart-Glass, Darshana Rathnayake, Dulanga Weerakoon, Meeralakshmi Radhakrishnan, Vigneshwaran Subbaraju, Inseok Hwang, Archan Misra

Research Collection School Of Computing and Information Systems

This work demonstrates the VGGlass system, which simultaneously interprets human instructions for a target acquisition task and determines the precise 3D positions of both user and the target object. This is achieved by utilizing LiDARs mounted in the infrastructure and a smart glass device worn by the user. Key to our system is the union of LiDAR-based localization termed LiLOC and a multi-modal visual grounding approach termed RealG(2)In-Lite. To demonstrate the system, we use Intel RealSense L515 cameras and a Microsoft HoloLens 2, as the user devices. VGGlass is able to: a) track the user in real-time in a global …

Balanced Blended Space: Proposing A Universal Theoretical Framework For Combinative Reality, David Smith, Frederick Bianchi

Publications and Research

In today's fragmented societies, a unified framework for communication and collaboration across different realities is crucial. We introduce Balanced Blended Space (BBS) as a framework for describing combinative reality, encompassing virtual, physical, and conceptual realms, all intrinsically connected. Interactions within these environments shape our perceptual space. This paper outlines key axiomatic assumptions, criteria for a universal framework, and fundamental terminology. We identify deep symmetries enabling the BBS framework, including Cognitive and Computational Symmetry, Physical and Virtual Symmetry, Mediation Pathway Symmetry, Space-Time Symmetry, and Sensory Symmetry. We propose tests to determine its viability, emphasizing virtual intelligence as a collaborative partner. We …

Understanding The Role Of Interactivity And Explanation In Adaptive Experiences, Lijie Guo

All Dissertations

Adaptive experiences have been an active area of research in the past few decades, accompanied by advances in technology such as machine learning and artificial intelligence. Whether the currently ongoing research on adaptive experiences has focused on personalization algorithms, explainability, user engagement, or privacy and security, there is growing interest and resources in developing and improving these research focuses. Even though the research on adaptive experiences has been dynamic and rapidly evolving, achieving a high level of user engagement in adaptive experiences remains a challenge. %????? This dissertation aims to uncover ways to engage users in adaptive experiences by incorporating …

All Hands On Deck: Choosing Virtual End Effector Representations To Improve Near Field Object Manipulation Interactions In Extended Reality, Roshan Venkatakrishnan

All Dissertations

Extended reality, or "XR", is the adopted umbrella term that is heavily gaining traction to collectively describe Virtual reality (VR), Augmented reality (AR), and Mixed reality (MR) technologies. Together, these technologies extend the reality that we experience either by creating a fully immersive experience like in VR or by blending in the virtual and "real" worlds like in AR and MR.

The sustained success of XR in the workplace largely hinges on its ability to facilitate efficient user interactions. Similar to interacting with objects in the real world, users in XR typically interact with virtual integrants like objects, menus, windows, …

Balanced Blended Space: Foundational Human–Ai Dialogues In A Symmetry-Based Mediation Framework, David Smith

Publications and Research

This working paper documents the early development of the Balanced Blended Space (BBS) framework through a series of iterative interactions between a cognitive agent (human researcher) and a computational agent (AI system) conducted in 2023. The work is motivated by the need for a universal theoretical model capable of describing the integration of physical, virtual, and conceptual spaces, particularly in response to increasing fragmentation across contemporary communication systems.

BBS is proposed as a symmetry-based mediation framework in which relationships between domains—such as physical and virtual space, cognition and computation, and multiple sensory modalities—are treated as structurally equivalent and mappable. Central …

Culture In Computing: The Importance Of Developing Gender-Inclusive Software, Creighton France

Computer Science and Computer Engineering Undergraduate Honors Theses

The field of computing as we know it today exists because of the contributions of numerous female mathematicians, computer scientists, and programmers. While working with hardware was viewed as “a man’s job” during the mid-20th century, computing and programming was viewed as a noble and high-paying field for women to occupy. However, as time has progressed, the U.S. has seen a decrease in the number of women pursuing computer science. The idea that computing is a masculine discipline is common in the U.S. today for reasons such as male-centered marketing of electronics and gadgets, an inaccurate representation of what it …

Procedural City Generation With Combined Architectures For Real-Time Visualization, Griffin Poyck

All Theses

The work and research of this paper sought to build upon traditional city generation and simulation in creating a tool that both realistically simulates cities and their prominent features and also creates aesthetic and artistically rich cities using assets that combine several contemporary or near contemporary architectural styles. The major city features simulated are the surrounding terrain, road networks, individual buildings, and building placement. The tools used to both create and integrate these features were created in Houdini with Unreal Engine 5 as the intended final destination. This research was influenced by the city, town, and road networking of Ghost …