Gesture Control
Scope
Code Project · Computer Vision
Client
College Project
Year
[2023]
Industry
Software Development
Apple Vision Pro intrigued me for a while and to be honest, It was enough to get me into this rather difficult (for me) computer vision project. I liked the gesture based controls it had & I liked it even more when it was demonstrated at WWDC.
At the end I did get bored very quickly after trying to get it to move my cursor faster by increasing the framerate at which the model processes images of my hand from my laptop’s webcam. Best I could manage was 60hz tracking with 99% GPU usage.
Safe to say I’ll not be trying that again..
Context
This project was inspired by Apple Vision Pro's gesture inputs & other github projects that explored gesture control.
Approach
Computer vision libraries in python were used to train a model that placed & tracked markers on the user's right hand. Gestures were trained for left click, right click & drag.
Outcome
Image throughput from the webcam to the script was heavily GPU bound. Limitations resulted in a 30hz tracking rate & unstable GPU temps.
Credits
Developed by Saraj Raja