Engineering Commons^™

Long-Term Human Video Activity Quantification In Collaborative Learning Environments, Venkatesh Jatla

Electrical and Computer Engineering ETDs

Research on video activity detection has mainly focused on identifying well-defined human activities in short video segments, often requiring large-parameter systems and extensive training datasets. This dissertation introduces a low-parameter, modular system with rapid inference capabilities, capable of being trained on limited datasets without transfer learning from large-parameter systems. The system accurately detects specific activities and associates them with students in real-life classroom videos. Additionally, an interactive web-based application is developed to visualize human activity maps over long classroom videos.

Long-term video activity detection in classrooms presents challenges, such as multiple simultaneous activities, rapid transitions, long-term occlusions, duration exceeding 15 …

Spanish And English Phoneme Recognition By Training On Simulated Classroom Audio Recordings Of Collaborative Learning Environments, Mario J. Esparza Perez

Electrical and Computer Engineering ETDs

Audio recordings of collaborative learning environments contain a constant presence of cross-talk and background noise. Dynamic speech recognition between Spanish and English is required in these environments. To eliminate the standard requirement of large-scale ground truth, the thesis develops a simulated dataset by transforming audio transcriptions into phonemes and using 3D speaker geometry and data augmentation to generate an acoustic simulation of Spanish and English speech. The thesis develops a low-complexity neural network for recognizing Spanish and English phonemes (available at github.com/muelitas/keywordRec). When trained on 41 English phonemes, 0.099 PER is achieved on Speech Commands. When trained on 36 Spanish …