Other Electrical and Computer Engineering Commons^™

Harmonicthreads: A Formative Evaluation Of A Fabric-Based Digital Musical Instrument Toward Inclusive Music-Making, Ellie Nguyen, Franceli L. Cibrian

Engineering Faculty Articles and Research

Background:

Inclusive music-making requires instruments that support varied bodies, abilities, musical backgrounds, and forms of participation. Digital musical instruments provide diverse approaches to sound creation, and fabric-based interfaces offer an alternative interaction modality that may support participation for some users and contexts. Their tactile and deformable properties enable forms of interaction that differ from conventional rigid or screen-based controllers and may offer inclusive possibilities in particular settings.

Objective:

This paper presents HarmonicThreads as a formative interaction-design case of a fabric-based digital musical instrument. The prototype explores how tactile cues, fabric deformation, projected visual feedback, and assisted accompaniment can support low-barrier …

Coherent Two-Photon Backscattering And Induced Angular Quantum Correlations In Multiple-Scattered Two-Photon States Of The Light, Nooshin M. Estakhri, Theodore B. Norris

Engineering Faculty Articles and Research

We present the emergence of coherent two-photon backscattering, a manifestation of weak localization, in multiple scattering of maximally entangled pure and fully mixed two-photon states and examine the effect of entanglement and classical correlations. Quantum correlations in backscattering are investigated for finite three-dimensional disordered structures in the weak localization regime, as well as systems of a small number of scatterers with specified spatial arrangements. No assumptions are made about the statistical behavior of the scattering matrix elements. Furthermore, we study the interplay between quantum correlations induced by multiple scattering and the correlations that may be present in the illumination fields, …

Familybloom: Examining Ecologies Of Collaboration In Family-Centered Health Tracking, Lucas M. Silva, Aehong Min, Evropi Stefanidi, Franceli L. Cibrian, Jesus A. Beltran, Cassie Zeiler, Sabrina E. B. Schuck, Kimberley D. Lakes, Gillian R. Hayes, Daniel A. Epstein

Engineering Faculty Articles and Research

Family health informatics tools can help support well-being with shared data tracking. Prior work typically focused on shared data review, but often in specific moments, like bedtime, or centered on caregiving of children or elderly members. To investigate how tracking can support mutual health collaboration between family members pervasively across daily contexts, we designed and deployed FamilyBloom, a glanceable smartwatch and home display system for mood and goal tracking. Twelve families with both neurotypical and ADHD members used FamilyBloom for three months on average. Our findings reveal how family-centered tracking created collaboration opportunities and tensions across multiple ecological systems: individual …

Multi-Modal Tensor Fusion For Alzheimer’S Disease Recognition, Mason Li, Tiffany Le, Jiajing Huang, Yuxin Wen

Engineering Faculty Articles and Research

Accurate and early diagnosis of Alzheimer’s disease (AD) is critical for effective intervention, disease monitoring, and patient care. Traditional diagnostic approaches rely on a single modality, such as clinical assessments, neuroimaging, or genetic markers, which may fail to capture the complex, multifaceted nature of AD. Multimodal learning has therefore been explored to integrate complementary information across data sources. However, conventional fusion strategies, including early feature concatenation and late decision-level fusion, often model modalities independently and fail to capture high-order cross-modal interactions. To address these limitations, we propose a multimodal tensor fusion network (MTFN) that integrates heterogeneous data sources, including visual …

Parentcoach: Designing An Mhealth Parenting App To Enhance Parental Involvement In Adhd Support, Franceli L. Cibrian, Nancy Herrera, Jesus A. Beltran, Lucas M. Silva, Mikaela Pulse, Kayla Anderson, Cassie Zeiler, Luc Rieffel, Daniel I. Lee, Sabrina E. B. Schuck, Kimberley D. Lakes

Engineering Faculty Articles and Research

Introduction: Parents play a vital role in supporting self-regulation and managing behaviors in children with Attention-Deficit/Hyperactivity Disorder (ADHD). However, many face barriers to accessing consistent, evidence-based support. Mobile health (mHealth) technologies offer a promising way to deliver flexible, low-burden guidance for parents on best practices and strategies to support their children's self-regulation. However, designing them is non-trivial.

Objective: This paper introduces ParentCoach, a mobile application designed to support parents of children with ADHD through brief daily lessons, reflection prompts, and skill-building activities.

Methods: ParentCoach was developed in two phases: (1) secondary analysis of qualitative data from over 30 families …

Design Considerations For Conversational Agents To Assess The Social-Emotional Well-Being Of Young Children In Low-Income South African Communities, Lucretia A. Williams, Elizabeth A. Ankrah, Catherine E. Draper, Caylee J. Cook, Franceli L. Cibrian, Jesus A. Beltran, Kimberley D. Lakes, Gillian R. Hayes

Engineering Faculty Articles and Research

A variety of digital technologies have been used to support early childhood development (ECD) programs in low-income South African communities. Even though technology has provided opportunities to increase access to health interventions, the lack of trust and socio-economic constraints under which these tools would need to work pose complex challenges. We examine home visitors’ work processes, experiences, and preferences of a conversational agent to support their work of administering social-emotional well-being assessments to young children ages 0-5. Analysis of the results of focus groups with 51 home visitors indicates the need for designing conversational agents that support ECD in the …

Advancements In Refreshable Braille Display Technology: A Comprehensive Survey, Maryam Etezad, Rajeev Joshi, Franceli L. Cibrian

Engineering Faculty Articles and Research

This paper provides a comprehensive mapping literature review of the advancements in refreshable Braille display (RBD) technology, which employs dynamically movable pins or dots to render Braille characters for individuals with blindness and visual impairment. This literature review, which includes 96 papers, aims to summarize the current evidence on six distinct types of RBDs: piezoelectric, electromagnetic, electroactive polymer (EAP), pneumatic, shape memory alloy (SMA), and microfluidic technologies. For each type, we discuss the underlying mechanisms, alongside the current trends and research opportunities. This comparative analysis aims to inform the selection of appropriate RBD technology based on specific user needs and …

Tellings Of The Pacific Ocean: A Landscape-Based Approach For Multispecies Design And Hci, Maliheh Ghajargar

Engineering Faculty Articles and Research

Environmental disturbances induced by climate change have caused significant changes in our ecosystems and are threatening the health of our environments. As a response to this issue, a growing body of work has emerged in HCI and design, which seeks to foreground more-than-human stories in support of making more sustainable and just futures. This research contributes to this broad agenda by probing graphic novels as a multispecies storytelling method for design and HCI. Combining ideas from Anna Tsing’s adventures of landscape and from HCI and design’s use of sequential art (e.g., storyboards), we use landscape as the main protagonist of …

Optical Character Recognition For Early Handwriting Legibility Assessment, Franceli L. Cibrian, Kayla Anderson, Yingying 'Yuki' Chen, Lauren Min, Lizbeth Escobedo

Engineering Faculty Articles and Research

Monitoring children’s handwriting, such as avoiding writing assignments, displaying uneven letter formation, or showing slow writing speed, can help identify developmental and academic issues early. Poor handwriting affects up to 34% of children, leading to academic and self-esteem challenges. Handwriting assessments, typically conducted by teachers, are often delayed due to workload and could be subjective and inconsistent. This paper explores the potential of Optical Character Recognition (OCR) technology to augment and ease handwriting assessments. Based on an evaluation of 10 OCR algorithms using 33 handwriting samples assessed by two experts, the research indicates that Pen to Print and Google are …

Real-World Implementation Of A Noninvasive, Ai-Augmented, Anemia-Screening Smartphone App And Personalization For Hemoglobin Level Self-Monitoring, Robert G. Mannino, Julie Sullivan, Jennifer K. Frediani, Paul George, Jeremy Whitson, James Tumlin, L. Andrew Lyon, Erika A. Tyburski, Wilbur A. Lam

Engineering Faculty Articles and Research

Anemia, characterized by low blood hemoglobin (Hgb) levels, afflicts >2 billion individuals worldwide. Here, we report real-world data generated by a smartphone app that noninvasively screens for anemia using only “fingernail selfies.” App data for anemia screening were obtained from >1.4 million uses across the United States enabling geographic mapping of Hgb levels. Of those, 9,061 users also self-reported complete blood count Hgb levels for comparison, resulting in accuracy and performance that match gold standard laboratory testing and a sensitivity and specificity of 89% and 93%, respectively, when using an anemia cutoff of 12.5 g/dL. Geotagged data enabled construction of …

Conversational Voice User Interfaces Supporting Individuals With Down Syndrome: A Literature Review, Franceli L. Cibrian, Concepción Valdez, Lauren Min, Vivian Genaro Motti

Engineering Faculty Articles and Research

Conversational Voice User Interfaces (CVUIs) are widely used in commercial applications such as personal assistants. CVUIs are beneficial for most users as they enable interaction through speech and natural language. However, recent studies indicate that underrepresented user groups, such as individuals with speech impairments and specifically those with Down syndrome, face challenges in using voice commands to control CVUIs. The anatomical and physiological differences affecting the voice, speech, fluency, and prosody of users with Down syndrome hinder their experience with CVUIs. This article presents the results of 43 papers related to the use of voice user interfaces supporting individuals with …

Danzens: A Toolkit For Sensing, Labeling And Visualizing Dance Movements, Yanelly Mego, Concepción Valdez, Hector M. Camarillo-Abad, Franceli L. Cibrian

Engineering Faculty Articles and Research

Wearable technology offers new opportunities for analyzing complex movements like dance, where precision, coordination, and feedback are key. In this demo, we present DanZens, a novel toolkit for real-time motion analysis in dance, leveraging wearable sensors to provide accessible and actionable feedback. Combining DanceTag to capture and annotate movements with DanceVis to visualize performance differences, DanZens uses Sony Mocopi sensors to analyze motion and generate intuitive heat maps through Dynamic Time Warping (DTW). This system enables precise, cost-effective comparisons between two people doing dance-related movements, offering personalized feedback and eliminating the need for expensive biomechanical labs. Designed to advance pervasive …

Harmonicthreads – An Interface That Supports Accessibility In Musical Interaction, Ellie Nguyen, Miyuki Weldon, Franceli L. Cibrian

Engineering Faculty Articles and Research

Traditional musical instruments often can create boundaries due to their cost, training, mobility, and cognitive requirements, making musical expression inaccessible. To address this challenge, we developed HarmonicThreads, a novel pervasive computing interface consisting of a responsive, flexible fabric. HarmonicThreads provides a tactile and auditory experience, allowing users to easily create and control sounds. Using embedded sensors and real-time processing, HarmonicThreads interprets the user's natural movements and interactions to create adaptable musical outputs. This enables context-aware musical interaction, demonstrating the potential of pervasive interfaces in reducing barriers and making musical expression more accessible.

Limitations In Speech Recognition For Young Adults With Down Syndrome, Franceli L. Cibrian, Yingying 'Yuki' Chen, Kayla Anderson, Cecilia Marie Abrahamsson, Vivian Genaro Motti

Engineering Faculty Articles and Research

Speech recognition has the potential to make technology more accessible to users. However, the accuracy of speech recognition remains limited for users with disabilities, including those with Down Syndrome, and the types and frequencies of recognition errors are poorly understood. This paper characterizes these problems, focusing on errors occurring when recognizing Down Syndrome speech. We analyze the transcripts from six speech recognition algorithms (Google, IBM, Otter.ai, Microsoft, AssemblyAI, OpenAI) using the audio content of 15 individuals with Down Syndrome (331 dialogues; 3428 words). Our analysis shows: (1) significant difference in speech recognition accuracy for people with Down Syndrome compared to …

Feasibility And Acceptability Of The Mazi Umntanakho Digital Tool In South African Settings: A Qualitative Evaluation, Catherine E. Draper, Caylee J. Cook, Elizabeth A. Ankrah, Jesus A. Beltran, Franceli L. Cibrian, Kimberley D. Lakes, Hanna Mofid, Lucretia Williams, Gillian R. Hayes

Engineering Faculty Articles and Research

To address the need for interventions targeting social emotional development and mental health of young children in South Africa, the Mazi Umntanakho (‘know your child’) digital tool was co-designed, and piloted with caregivers and 3–5-year-old children involved in home visiting programmes promoting early childhood development. The aim of this study was to qualitatively evaluate the feasibility and acceptability of this tool in four urban and four rural low-income communities, from the perspective of home visitors and caregivers. Focus groups were conducted with home visitors (n = 117) and caregivers (n = 72). Issues relating to the feasibility of …

Movian: Advancing Human Motion Analysis With 3d Visualization And Annotation, Trudi Di Qi, Isaac Browen, David Zhang, Hector M. Camarillo-Abad, Franceli L. Cibrian

Engineering Faculty Articles and Research

Human motion analysis, including data visualization and annotation, is crucial for understanding human behavior and intentions during various activities, aiding in the development of innovative tools that support independent living. Current wearable sensing technology provides rich 3D spatial movement data but generates multimodal complex datasets that require specialized skills for effective analysis. Despite the need, limited research exists on tools for effective visualization and easy annotation of such complex motion data. MoViAn (Motion Data Visualization and Annotation) is an innovative 3D data analysis system offering enriched visual representations of 3D human motion data (e.g., gaze, hand movements), along with an …

Analyzing Handwriting Legibility In Children Using Smart Vs. Traditional Pen, Franceli L. Cibrian, Lauren Min, Yingying 'Yuki' Chen, Kayla Anderson, Oscar Gutierrez, Lizbeth Escobedo

Engineering Faculty Articles and Research

Handwriting, traditionally acquired through paper, pen, or pencil, is crucial for children’s development, learning, and communication. The legibility of letters holds crucial implications for children’s composition and even self-esteem. In order to ensure legibility, timely input from educators and parents is essential, although it primarily depends on their experience. With current technological advancements, smartpens, augmented with sensing capabilities, could offer a novel approach providing feedback. However, it is unclear if the smartpen’s weight and form factor could affect children’s handwriting legibility is unclearsmartpen’s weight and form factor could affect children’s handwriting legibility. This study involves 16 children aged 9 to …

Towards A Multimodal Approach For Assessing Adhd Hyperactivity Behaviors, Franceli L. Cibrian, Lauren Min, Vitica Arnold

Engineering Faculty Articles and Research

Attention Deficit Hyperactivity Disorder (ADHD) is the most prevalent childhood psychiatric condition that needs an assessment of inattention, hyperactivity, and impulsiveness symptoms. Particularly in young children, hyperactivity-impulsivity stands out as a primary concern. However, those behaviors may or may not be evidenced when a child is in a small room, one-on-one with a single adult. Therefore, Ambient Intelligence technology that supports data collection in a natural setting, paired with expert human decision-making can potentially improve the quality of assessments. In this paper, we conduct a literature review and analysis to align ADHD assessment criteria with potential sensor technologies to collect …

Etherealbreathing: A Holographic Biofeedback Game To Support Relaxation In Autistic Children, Arturo Morales Téllez, Isabel López Hurtado, Franceli L. Cibrian, Monica Tentori

Engineering Faculty Articles and Research

Biofeedback training for box breathing is becoming increasingly accessible due to advancements in consumer-grade breathing sensors. However, there is limited research on their design and applications for specialized populations. This study evaluates a novel biofeedback holographic game, EtherealBreathing, designed to support autistic children. In EtherealBreathing, children practice box breathing to collect virtual elements to maintain the Earth's balance, using a wearable sensor to measure chest expansion for breath detection. A deployment study with 20 autistic children revealed that EtherealBreathing effectively promotes box breathing, leading to better health-related outcomes, such as lowering participants’ heart and respiratory rates than traditional practices. Biofeedback …

Smartphone Haptics Can Uncover Differences In Touch Interactions Between Asd And Neurotypicals, Ivonne Monarca, Franceli L. Cibrian, Isabel López Hurtado, Monica Tentori

Engineering Faculty Articles and Research

Utilizing touch interactions from smartphones for gathering data and identifying digital markers for screening and monitoring neurological disorders, such as Autism Spectrum Disorder (ASD), is an emerging area of research. Smartphones provide multiple benefits for this kind of study, including unobtrusive data collection via built-in sensors, integrated haptic feedback systems, and the capability to create specialized applications. Acknowledging the significant yet understudied presence of tactile processing differences in individuals with ASD, we designed and developed Feel and Touch, a mobile game that leverages the haptic capabilities of smartphones. This game provides vibrotactile feedback in response to touch interactions and collects …

Digital Health Intervention For Children With Adhd To Improve Mental Health Intervention, Patient Experiences, And Outcomes: A Study Protocol, Nancy Herrera, Franceli L. Cibrian, Lucas M. Silva, Jesus Armando Beltran, Sabrina E. B. Schuck, Gillian R. Hayes, Kimberley D. Lakes

Engineering Faculty Articles and Research

Background

Attention Deficit Hyperactivity Disorder (ADHD) is the most prevalent childhood psychiatric condition with profound public health, personal, and family consequences. ADHD requires comprehensive treatment; however, lack of communication and integration across multiple points of care is a substantial barrier to progress. Given the chronic and pervasive challenges associated with ADHD, innovative approaches are crucial. We developed the digital health intervention (DHI)—CoolTaCo [Cool Technology Assisting Co-regulation] to address these critical barriers. CoolTaCo uses Patient-Centered Digital Healthcare Technologies (PC-DHT) to promote co-regulation (child/parent), capture patient data, support efficient healthcare delivery, enhance patient engagement, and facilitate shared decision-making, thereby improving access to …

Joint Modeling Of Degradation Signals And Time-To-Event Data For The Prediction Of Remaining Useful Life, Sebastian Brumm, Erik Linstead, Junde Chen, Narayanaswamy Balakrishnan, Yuxin Wen

Engineering Faculty Articles and Research

Accurate prediction of remaining useful life (RUL) for in-service systems plays an important role in ensuring efficient operation of industrial equipment and in preventing unexpected equipment failures. In this paper, we present a prognostic framework for real-time RUL prediction based on joint modeling of both degradation signals and time-to-event data. The proposed model employs a change point-based general path model to capture signal non-linearity and Neural network (NN) based Cox model to link the time-to-event data with the estimated degradation trend. An empirical two-step scheme for hyperparameter estimation is proposed to enhance prognostic accuracy. Furthermore, an efficient Bayesian model updating …

Long-Distance Photon-Mediated And Short-Distance Entangling Gates In Three-Qubit Quantum Dot Spin Systems, Nooshin M. Estakhri, Ada Warren, Sophia E. Economou, Edwin Barnes

Engineering Faculty Articles and Research

Superconducting resonator couplers will likely become an essential component in modular semiconductor quantum dot (QD) spin qubit processors, as they help alleviate crosstalk and wiring issues as the number of qubits increases. Here, we focus on a three-qubit system composed of two modules: a two-electron triple QD resonator coupled to a single-electron double QD. Using a combination of analytical techniques and numerical results, we derive an effective Hamiltonian that describes the three-qubit logical subspace and show that it accurately captures the dynamics of the system. We examine the performance of short-range and long-range entangling gates, revealing the effect of a …

M3t-Lm: A Multi-Modal Multi-Task Learning Model For Jointly Predicting Patient Length Of Stay And Mortality, Junde Chen, Qing Li, Feng Liu, Yuxin Wen

Engineering Faculty Articles and Research

Ensuring accurate predictions of inpatient length of stay (LoS) and mortality rates is essential for enhancing hospital service efficiency, particularly in light of the constraints posed by limited healthcare resources. Integrative analysis of heterogeneous clinic record data from different sources can hold great promise for improving the prognosis and diagnosis level of LoS and mortality. Currently, most existing studies solely focus on single data modality or tend to single-task learning, i.e., training LoS and mortality tasks separately. This limits the utilization of available multi-modal data and prevents the sharing of feature representations that could capture correlations between different tasks, ultimately …

Predictive Residual Neural Networks For Optical Trapping Of Small Particles, Nasim Mohammadi Estrakhri, Ponthea Zahraii, Saman Kashanchi, Nooshin M. Estakhri

Engineering Faculty Articles and Research

Optical tweezers provide a non-contact method to trap, move, and manipulate micro- and nano-sized objects. Using properly designed dielectric and plasmonic nanostructure configurations, optical tweezers have been tailored to create stable and precise trapping for nanoscale objects. Recent advances in numerical optimization techniques allow further enhancement in nanoscale optical traps through inverse optimization of such configurations. One of the main challenges in such optimization approaches is the time-consuming nature of full-wave simulation of nanostructures and postprocessing steps to extract optical forces. To address this challenge, we introduce a surrogate solver based on residual neural networks that can accurately predict the …

Multimodal Mixing Convolutional Neural Network And Transformer For Alzheimer’S Disease Recognition, Junde Chen, Yun Wang, Adnan Zeb, M. D. Suzauddola, Yuxin Wen

Engineering Faculty Articles and Research

Early recognition of Alzheimer’s disease (AD) and its precursor state, mild cognitive impairment (MCI), is pivotal in interrupting the progression of the disease and providing suitable treatment. Recent development in deep learning techniques has drawn great research attention for improving the efficacy of AD recognition. However, numerous current methods solely utilize data from a single auxiliary domain, limiting their ability to harness valuable intrinsic insights from multiple domains. To cope with the challenge, this paper is devoted to establishing an innovative multimodal medical data fusion model, termed as MMDF, to perform Alzheimer’s disease recognition. Multimodal data including clinical records and …

Fpca-Setcn: A Novel Deep Learning Framework For Remaining Useful Life Prediction, Junde Chen, Yuxin Wen, Xuxue Sun, Adnan Zeb, Mohammad Saleh Meiabadi, Sasan Sattarpanah Karganroudi

Engineering Faculty Articles and Research

The accurate prediction of remaining useful life (RUL) can serve as a reliable foundation for equipment maintenance, thereby effectively reducing the incidence of failure and maintenance costs. In this study, a novel deep learning (DL) framework that incorporates functional principal component analysis (FPCA) and enhanced temporal convolutional network (TCN) is proposed for RUL prediction. Precisely, FPCA is employed to capture the changing patterns in multistream degradation trajectories. Subsequently, the reconstructed signals from FPCA are fed into a convolutional block for extracting deep-level features. An enhanced squeeze-and-excitation (ESE) block is then incorporated into the network for adaptive feature recalibration, enhancing the …

Creative Insights Into Motion: Enhancing Human Activity Understanding With 3d Data Visualization And Annotation, Isaac Browen, Hector M. Camarillo-Abad, Franceli L. Cibrian, Trudi Di Qi

Engineering Faculty Articles and Research

This paper presents a novel 3D system for human motion analysis - Motion Data Visualization and Annotation (MoViAn). Designed to provide a comprehensive visual representation of 3D human motion data, MoViAn incorporates detailed visualization of gaze direction, hand movements, and object interactions, alongside an interactive interface for efficient data annotation. A user study involving eight participants indicates that MoViAn enables users to thoroughly explore and annotate human motion data, with System Usability Scale (SUS) results demonstrating a satisfactory usability level. The contribution of this paper lies in the development of an interactive and usable data analytics tool aimed at deepening …

Toward Intuitive 3d Interactions In Virtual Reality: A Deep Learning- Based Dual-Hand Gesture Recognition Approach, Trudi Di Qi, Franceli L. Cibrian, Meghna Raswan, Tyler Kay, Hector M. Camarillo-Abad, Yuxin Wen

Engineering Faculty Articles and Research

Dual-hand gesture recognition is crucial for intuitive 3D interactions in virtual reality (VR), allowing the user to interact with virtual objects naturally through gestures using both handheld controllers. While deep learning and sensor-based technology have proven effective in recognizing single-hand gestures for 3D interactions, research on dual-hand gesture recognition for VR interactions is still underexplored. In this work, we introduce CWT-CNN-TCN, a novel deep learning model that combines a 2D Convolution Neural Network (CNN) with Continuous Wavelet Transformation (CWT) and a Temporal Convolution Network (TCN). This model can simultaneously extract features from the time-frequency domain and capture long-term dependencies using …

Co-Designing Situated Displays For Family Co-Regulation With Adhd Children, Lucas M. Silva, Franceli L. Cibrian, Clarisse Bonang, Arpita Bhattacharya, Aehong Min, Elissa M. Monteiro, Jesus A. Beltran, Sabrina E. B. Schuck, Kimberley D. Lakes, Gillian R. Hayes, Daniel A. Epstein

Engineering Faculty Articles and Research

Family informatics often uses shared data dashboards to promote awareness of each other’s health-related behaviors. However, these interfaces often stop short of providing families with needed guidance around how to improve family functioning and health behaviors. We consider the needs of family co-regulation with ADHD children to understand how in-home displays can support family well-being. We conducted three co-design sessions with each of eight families with ADHD children who had used a smartwatch for self-tracking. Results indicate that situated displays could nudge families to jointly use their data for learning and skill-building. Accommodating individual needs and preferences when family members …