Nice to see people getting interested in eye gaze. There are two things that you might like to look at that can help the UX.
1 - Calibration. Looking at static dots is BORING. The best idea I've seen is Tobii's gaming calibration where you look at dots to make them wobble and pop. This makes the whole process feel like a game, even when you've done it a hundred times before. I would love to see more ideas in this space to give a much more natural-feeling calibration process - even better if you can improve the calibration over time with a feedback loop, when users interact with an element.
2 - Gaze feedback. You are absolutely right that seeing a small, inaccurate and jumpy dot does more harm than good. Again, Tobii have led the way with their 'ghost overlay' for streamers.
For an example, see the following video. After calibration the ghost overlay is used to give approximate feedback. This is enough that some naive users are able to make small adjustments to a constant calibration error, or at least give feedback that the gaze is wrong, not that the UI is not responding.
The WebGaze software used in this page was introduced in by Papoutsaki and co-workers by a team from Brown university and Georgia Tech in 2016:
- Papoutsaki, A. et al. (2016). "WebGazer: Scalable Webcam Eye Tracking Using User Interactions."
Presented at the International Joint Conference on Artificial Intelligence (IJCAI).
https://www.ijcai.org/Proceedings/16/Papers/540.pdf
While we are at it, you may also find the following research publications relevant to this discussion:
Impressive work - I have experimented with Tobii trackers - they are pretty accurate to work with. But accomplishing this with a single camera is definitely something. Would love to follow your work further. Keep going :)
Ha! The timing is impeccable. This is a great demo. I've been experimenting with using gaze and eye tracking for cursor prediction as a side project. I like the idea of pressing 'space' for each dot. I just had the 9-dot movement going from one point to another. I'm using Mediapipe's face landmarks model (I wasn't aware of WebGazer). I'll continue to work on it, but it's great to see a similar thought process.
I may be the result of some evolutionary bottleneck, but wherever there is a camera lens, I assume eye tracking (and sentiment prediction) is at least possible, and at most globally always on.
Very cool demo! Once this becomes good enough (now it's very wobbly and requires huge UI) I'd love to be able to read articles and navigate just using the eyes. Feels very natural.
This is very cool!! Have you considered making a WebGL game that uses eye tracking for things like aiming? Could be very cool and one of the very few accessible games
Show HN: Eyesite – Experimental website combining computer vision and web design
(blog.andykhau.com)126 points by akchro 12 June 2025 | 23 comments
Comments
1 - Calibration. Looking at static dots is BORING. The best idea I've seen is Tobii's gaming calibration where you look at dots to make them wobble and pop. This makes the whole process feel like a game, even when you've done it a hundred times before. I would love to see more ideas in this space to give a much more natural-feeling calibration process - even better if you can improve the calibration over time with a feedback loop, when users interact with an element.
2 - Gaze feedback. You are absolutely right that seeing a small, inaccurate and jumpy dot does more harm than good. Again, Tobii have led the way with their 'ghost overlay' for streamers.
For an example, see the following video. After calibration the ghost overlay is used to give approximate feedback. This is enough that some naive users are able to make small adjustments to a constant calibration error, or at least give feedback that the gaze is wrong, not that the UI is not responding.
https://youtu.be/mgQY4dL-09E?feature=shared&t=36
- Papoutsaki, A. et al. (2016). "WebGazer: Scalable Webcam Eye Tracking Using User Interactions." Presented at the International Joint Conference on Artificial Intelligence (IJCAI). https://www.ijcai.org/Proceedings/16/Papers/540.pdf
While we are at it, you may also find the following research publications relevant to this discussion:
- "Improving User Perceived Page Load Times Using Gaze" (USENIX NSDI 2017) https://www.usenix.org/conference/nsdi17/technical-sessions/...
- "No clicks, no problem: using cursor movements to understand and improve search" (Huang, White, Dumais from Microsoft Research, ACM SIG CHI '11) https://dl.acm.org/doi/abs/10.1145/1978942.1979125
- Virtual gazing in video surveillance (SMVC '10) https://dl.acm.org/doi/abs/10.1145/1878083.1878089
Why?