EyeQ: Exploring Corneal Reflection for Mobile Context Sensing
Jaewook Lee, Judith Amores, Donghoon Shin, Bo-Eun Choi, Jason Kim, Davin Win Kyi, Seok-Young Kim, Jon E. Froehlich 等 9 位作者
原文摘要
What if a single face-facing camera could sense your surroundings---through your eyes? We introduce EyeQ, which explores corneal reflection as a modality for context-aware mobile interaction. Traditional vision-based context sensing relies on world-facing cameras that demand explicit aiming or dual-camera setups; EyeQ instead shows that an off-the-shelf VLM can read scene context from corneal reflections captured with a smartphone camera. To recover usable detail from mobile corneal crops, EyeQ pairs on-device iris-guided capture with server-side AI enhancement, then leverages VLMs to extract scene types, names, and objects from warped, unfamiliar eye-image sequences. In an N=10 study, we characterize how capture conditions shape reflection fidelity, evaluate what VLMs detect, misclassify, and hallucinate, and demonstrate interaction opportunities, including detecting a steering wheel to distinguish drivers from passengers. This work reframes the eye in HCI---from a target for gaze estimation to a passive mirror of one's surroundings---and examines the privacy tensions it raises.
cs.HC