The Evolution of User Interfaces • Ken Pfeuffer • GOTO 2025

2026-06-24 · Source: GOTO Conferences · Field: Technology & Digital — Emerging Technologies & Innovation, Artificial Intelligence & Machine Learning · Depth: Advanced, extended

Summary

The evolution of user interfaces, particularly in Human-Computer Interaction (HCI), is shifting towards combined eye and hand interaction, exemplified by the Apple Vision Pro's "gaze and pinch" paradigm. This approach, where users look at elements and perform pinch gestures for selection, represents a major departure from traditional hand-centric input devices like mice and touchscreens. The speaker's decade-long research, initiated during a "desert" period for eye-tracking in 2013, explored various eye-hand interface concepts in 2D and 3D/VR, including multi-touch with gaze and indirect gestures. Apple's 2023 announcement and 2024 integration of gaze and pinch validated this research, signaling a future where eye-tracking is central to XR, with ongoing exploration into advanced concepts like "look and drop" and two-handed interaction.

Key takeaway

For AI Scientists and Product Designers developing XR experiences, the "gaze and pinch" paradigm, validated by Apple Vision Pro, fundamentally changes how users will interact with spatial computing. You should prioritize integrating eye-tracking with simple hand gestures to create intuitive, less fatiguing interfaces. This approach moves beyond legacy controller-based or direct-touch models, proving critical for broad XR adoption and future interface design.

Key insights

Combined eye and hand interaction, exemplified by "gaze and pinch," is the next major UI paradigm shift for XR.

Principles

• UI evolution favors natural, spatial interaction.
• Legacy thinking hinders novel interface adoption.
• Simplicity (e.g., pinch) is key for broad usability.

Method

Iterative prototyping, user studies, and empirical data collection refined eye-hand interaction concepts across 2D and 3D/VR contexts.

In practice

• Explore gaze-based input for XR navigation.
• Integrate simple pinch gestures for selection.
• Consider "look and drop" for reduced fatigue.

Topics

• Human-Computer Interaction
• User Interface Evolution
• Extended Reality
• Apple Vision Pro
• Gaze and Pinch
• Eye Tracking
• Hand Tracking

Best for: AI Product Manager, Product Manager, Research Scientist, Product Designer, AI Scientist

Related on AIssential

• Mira Murati steps back into the spotlight, carefully — Thinking Machines Lab is developing "interaction models" for real-time, continuous human-AI communication beyond turn-based prompts.
• 5 new features for Android XR — Android XR's new features deepen immersion and natural interaction for Samsung Galaxy XR users.
• ACM Human-Computer Interaction Conference (CHI) 2026 — Apple's CHI 2026 participation highlights human-centered AI and accessibility in UI design and wearable technology.
• Reimagining a 50-year-old interface (the mouse pointer) with AI — An AI-enabled pointer can interpret complex user intent by combining pointing with voice, text, and visual understanding.
• Computational framework to predict and shape human–machine interactions in closed-loop, co-adaptive neural interfaces — Control and game theory model co-adaptive neural interfaces, predicting how decoder parameters shape user learning and system performance.
• Meet the former Apple designer building a new AI interface at Hark — Hark aims to redefine human-AI interaction by deeply integrating AI models, hardware, and interfaces for a personalized, intelligent system.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by GOTO Conferences.