Rhythmic Fitness Beat 'Em All using Real-Time Monocular Human Pose Estimation
Yanis Zaoui, Valentin Bégel, Mohamed Adjel
原文摘要
Short and frequent physical activity exercices mitigate the health risks of sedentary behavior. Yet, the lack of motivation remains a critical barrier to adoption. Exergames gamify physical activity, but most rely on dedicated hardware, and outside dance and rhythm games they rarely use the musical beat. We present a work-in-progress beat 'em up exergame played with the whole body in front of a single smartphone, which runs the entire pipeline: an off-the-shelf real-time 3D human mesh recovery model on the phone's neural processing unit, gesture recognition, and the game, shown on any TV through a USB-C to HDMI cable. Our contributions are (1) a 3D recognition of five shadow-boxing moves (guard, punch, slip, squat, jump), each defined by one metric measure in a body frame aligned with gravity and one threshold with hysteresis, personalised during a tutorial; (2) a rhythm stage whose discrete cues are placed on the song's kick drum and judged on the movement's capture time, with a delay learned for each player; and (3) a phone-only architecture, running at 30 frames per second, that turns a phone and an HDMI cable into an exergame console.
cs.HC