How Will Apple’s Vision Pro Keyboard Actually Work?

With WWDC now come and gone, there’s been a lot to unpack. From news of the new M2 Ultra, a new lineup of MacBook Air laptops, and of course this new VR/AR headset, we’ve been busy trying to fully explore each story thoroughly. However, even with plenty of news and hands-on reviews hitting the internet, there are still so many questions to be answered.

Most of the questions have to do with Apple Vision Pro and the bold new future it envisions, yet only lightly describes.

Luckily, at a recent developer session, Apple lifted the veil a bit more on how users will eventually be interacting with Vision Pro, most notably about how eyes and hands will be the primary way of using the headset with the support of both physical and virtual keyboards and other controls.

Let’s take a look at this new virtual workflow and see how it might be a future part of your own one day.

Intuitive Design for Spatial Input

If you’d like to follow along at home, Apple has released the full video from their recent “Design for Spatial Input” session in which they go over how everything with the Vision Pro is going to work from an interaction perspective. With Apple’s usual intuitive flair, it’s no surprise that so much about the input flow is easy to follow and understand.

Designed really for just your eyes and hands, the majority of what you’ll be able to control with this new AR/VR headset will be as easy as a look and small hand movement. However, it still might be helpful to review the best practices for each input method as a way to create spatial experiences which are comfortable, intuitive, and satisfying.

Here are the different sections if you’d like to check those out as well:

• 0:00 - Introduction
• 2:22 - Eyes
• 12:21 - Hands
• 18:36 - Conclusion

Do you think you'd like to virtually type on this?Credit: Apple

Perhaps the most interesting piece of virtual technology to be introduced since the Tamagotchi, Apple Vision Pro’s virtual keyboard looks to be as crazy, but probably intuitive, as promised. Designed by Apple designer Eugene Krivoruchko, it’s going to potentially be an important part of a user’s virtual workflow, depending on which tasks they chose to do in the experience.

While you obviously won’t get the same tactile feedback as you would by jamming your fingers on a keyboard in real life, this virtual keyboard will utilize an array of visual and audio feedback to let you know what’s clicking and how your inputs are reading out. It’s odd to think about, but the differences between hovering over a key versus tapping one are indeed quite different.

The virtual experience will also support voice input, which can be coupled with any virtual keyboards or other controls too as a way to further engage the interface in a natural and meaningful way. With six microphones built into the Vision Pro, Apple is fast at work trying out new and innovative ways in which you’ll be able to use classic controls (like keyboards and mouse-like hand movements) with new ones we haven’t even thought of yet.

Are video professionals even interested in virtual workflows at this point?Credit: Apple

How Will This Work for Video Editing and Production?

The big question for film professionals and video editors is going to have to do with how these new virtual tools and workflows might eventually help out with video. Will video editors choose to work on their projects by using Apple’s Vision Pro interface? There are certainly plenty of new possibilities to explore like endless multi-screen workflows and the ability to grab, drag, drop, or cut clips by hand—something we haven’t seen in video editing since the analog film days.

Or will video editors who do adopt virtual still want to have a keyboard to be able to use their preferred Final Cut Pro (or Premiere Pro if Apple allows it) shortcuts for editing as they usually would on an IRL computer or workstation?

This is where things get truly exciting and about as open-ended as anything we’ve ever seen in film or tech. While we’ll get more answers eventually, the questions are going to keep coming…

How do you feel about Apple’s Vision Pro virtual keyboard and other virtual workflow possibilities?

Let us know your thoughts in the comments below.