Open Access. Powered by Scholars. Published by Universities.®
Physical Sciences and Mathematics Commons™
Open Access. Powered by Scholars. Published by Universities.®
Articles 1 - 5 of 5
Full-Text Articles in Physical Sciences and Mathematics
Annapurna: Building A Real-World Smartwatch-Based Automated Food Journal, Sougata Sen, Vigneshwaran Subbaraju, Archan Misra, Rajesh Krishna Balan, Youngki Lee
Annapurna: Building A Real-World Smartwatch-Based Automated Food Journal, Sougata Sen, Vigneshwaran Subbaraju, Archan Misra, Rajesh Krishna Balan, Youngki Lee
Research Collection School Of Computing and Information Systems
We describe the design and implementation of a smartwatch-based, completely unobtrusive, food journaling system, where the smartwatch helps to intelligently capture useful images of food that an individual consumes throughout the day. The overall system, called Annapurna, is based on three key components: (a) a smartwatch-based gesture recognizer to identify eating gestures, (b) a smartwatch-based image capturer that obtains a small set of relevant and useful images with a low energy overhead, and (c) a server-based image filtering engine that removes irrelevant uploaded images, and then catalogs them through a portal. Our primary challenge is to make the system robust …
Smartwatch-Based Early Gesture Detection & Trajectory Tracking For Interactive Gesture-Driven Applications, Tran Huy Vu, Archan Misra, Quentin Roy, Kenny Tsu Wei Choo, Youngki Lee
Smartwatch-Based Early Gesture Detection & Trajectory Tracking For Interactive Gesture-Driven Applications, Tran Huy Vu, Archan Misra, Quentin Roy, Kenny Tsu Wei Choo, Youngki Lee
Research Collection School Of Computing and Information Systems
The paper explores the possibility of using wrist-worn devices (specifically, a smartwatch) to accurately track the hand movement and gestures for a new class of immersive, interactive gesture-driven applications. These interactive applications need two special features: (a) the ability to identify gestures from a continuous stream of sensor data early–i.e., even before the gesture is complete, and (b) the ability to precisely track the hand’s trajectory, even though the underlying inertial sensor data is noisy. We develop a new approach that tackles these requirements by first building a HMM-based gesture recognition framework that does not need an explicit segmentation step, …
Magi: Enabling Multi-Device Gestural Applications, Tran Huy Vu, Choo Tsu Wei, Kenny, Youngki Lee, Richard Christopher Davis, Archan Misra
Magi: Enabling Multi-Device Gestural Applications, Tran Huy Vu, Choo Tsu Wei, Kenny, Youngki Lee, Richard Christopher Davis, Archan Misra
Research Collection School Of Computing and Information Systems
We describe our vision of a multiple mobile or wearable device environment and share our initial exploration of our vision in multi-wrist gesture recognition. We explore how multi-device input and output might look, giving four scenarios of everyday multi-device use that show the technical challenges that need to be addressed. We describe our system which allows for recognition to be distributed between multiple devices, fusing recognition streams on a resource-rich device (e.g., mobile phone). An Interactor layer recognises common gestures from the fusion engine, and provides abstract input streams (e.g., scrolling and zooming) to user interface components called Midgets. These …
The Case For Smartwatch-Based Diet Monitoring, Sougata Sen, Vigneshwaran Subbaraju, Archan Misra, Rajesh Krishna Balan, Youngki Lee
The Case For Smartwatch-Based Diet Monitoring, Sougata Sen, Vigneshwaran Subbaraju, Archan Misra, Rajesh Krishna Balan, Youngki Lee
Research Collection School Of Computing and Information Systems
We explore the use of gesture recognition on a wrist-worn smartwatch as an enabler of an automated eating activity (and diet monitoring) system. We show, using small-scale user studies, how it is possible to use the accelerometer and gyroscope data from a smartwatch to accurately separate eating episodes from similar non-eating activities, and to additionally identify the mode of eating (i.e., using a spoon, bare hands or chopsticks). Additionally, we investigate the likelihood of automatically triggering the smartwatch's camera to capture clear images of the food being consumed, for possible offline analysis to identify what (and how much) the user …
Using Infrastructure-Provided Context Filters For Efficient Fine-Grained Activity Sensing, Vigneshwaran Subbaraju, Sougata Sen, Archan Misra, Satyadip Chakraborty, Rajesh Krishna Balan
Using Infrastructure-Provided Context Filters For Efficient Fine-Grained Activity Sensing, Vigneshwaran Subbaraju, Sougata Sen, Archan Misra, Satyadip Chakraborty, Rajesh Krishna Balan
Research Collection School Of Computing and Information Systems
While mobile and wearable sensing can capture unique insights into fine-grained activities (such as gestures and limb-based actions) at an individual level, their energy overheads are still prohibitive enough to prevent them from being executed continuously. In this paper, we explore practical alternatives to addressing this challenge-by exploring how cheap infrastructure sensors or information sources (e.g., BLE beacons) can be harnessed with such mobile/wearable sensors to provide an effective solution that reduces energy consumption without sacrificing accuracy. The key idea is that many fine-grained activities that we desire to capture are specific to certain location, movement or background context: infrastructure …