Vision Pro

Exploring Spatial Computing

Feb. 16, 2024 - Willem L. Middelkoop

Currently, I am overlooking a lake at Mount Hood while writing this. I hear birds in the distance and see the lake calm, with subtle waves and some mist in the distance. Yet, it is fake, as I am sitting on our top floor, a barely furnished room full of items belonging to a family house with two young kids. I am using Apple's Vision Pro to explore what Spatial Computing can be. I am in awe; let me explain in this blog post.

... not! Just our attic full of "family stuff"

Apple Vision Pro

Imagine a set of great earphones for listening to music, offering you private access to high-quality audio playback. The Apple Vision Pro is like that, but for your eyes instead of your ears. You put it on like a set of goggles, and inside, you see a digital world blending with your reality. You can choose to let the real outer world in or filter it out, much like how active noise cancelling works for audio.

Using some very advanced chips, sensors, and cameras, the Apple Vision Pro headset is capable of projecting virtual objects into your real world, positioning them in a fixed place where they stay as long as you want them there. You can do things like watching a movie on a gigantic cinema display or immersing yourself in your family's photos. Or, you can use the Vision Pro for work, as I do, using a window where I type my text, right at the lakeside of Mount Hood.

Other websites and blogs have posted some very in-depth discussions on Apple Vision Pro, discussing the hardware, its advantages, and disadvantages. I recommend you check those out as I do not want to repeat what others have written or said. Some reviews worth your time:

Spatial Computing

More than this particular first generation of hardware, I am interested in what Apple is showing the world with its VisionOS software. It's not just a fancy wearable projector; it's interactive! Cameras on the inside track your iris position, i.e., where you're exactly looking. This information is used to enable interaction with the digital world by just looking at things. A subtle tap with your fingers is registered by another set of cameras, all seamlessly integrating an experience that enables you to "look and tap" like you would otherwise do with "point and click" or "touch and swipe".

This fundamental interaction model is very well executed; after a few minutes, it feels totally natural, and I have since been wondering why my iPhone or iPad does not respond in the same manner. It's classic Apple magic as all the heavy lifting is done without giving me much (if any) friction. It just works.

Bluetooth keyboard and trackpad connected to Vision Pro

To get some work done, I have connected a standard bluetooth keyboard and trackpad to the Vision Pro, leveraging my ability to touch type; it all feels very natural. The Vision Pro does not require a computer; it is the computer! It features quite capable Apple Silicon chips, and plenty of onboard storage (mine has 1 TB).

You're looking at a fully capable computer system here - and a cup of coffee

If you have been reading my blog, you know I have a thing for tablets as they are portable yet very capable. The Vision Pro might be another step forward: it is very portable, yet it offers an entire virtual world for your eyes! It's like being able to carry a massive multi-monitor setup with me; it's bonkers!

Not that crazy: display, keyboard, trackpad (note the hovering piece of UI above the hardware keyboard)

You can easily arrange a multi-monitor work setup like this

It is hard to capture in a 2D photo, but for your eyes, there is real depth in the setup, like those two screens are really there

You can look around them, move closer to them, or rearrange them by dragging them through your room

The projections appear so natural that my mind is really convinced it can touch things.

Mixing the digital and real world is very accessible; it allows you to stay aware of things around you. I like it as it makes you feel a little less "enclosed inside the computer". It works for certain workflows, like sending some emails, looking things up, or making a call. The experience gets a little different if you decide to immerse yourself in any of the Vision Pro's virtual environments.

Some folks refer to it as "deep work", the type of work you need some serious focus for. I find Vision Pro especially powerful for getting myself into the state of flow that is needed for the heavy lifting. I can immerse myself with context (images, logs, code, mockups) and filter out any visual clutter from the real world.

These windows represent different views on a particular piece of work - they are big, the tall window in the center is approximately 3 meters high!

The 'smallest' window from above is in fact very large; for comparison of scale, I removed the virtual moon surface so you can see the window next to my little daughter's shoes.

Imagine a set of windows so big that you can literally stand between them. That is how I like my most powerful Vision Pro setups. You almost become one with your context, seriously. You can create an environment that enables you to really connect with what you're doing. I love walking around the windows, looking at some code or server output, and sort of getting a feel of it being a "big and working machine". In a way, it feels like standing in a big machine room. It is really unlike any conventional desktop experience.

Again, it is very hard to capture in a 2D photo, but being able to walk around your digital context is simply incredible!

These windows are really big, conveying a sense of greatness like a statue of some kind

If you're in a virtual environment, the Vision Pro will warn you if you risk walking into something - which can be a real hazard if you're sharing the house with some kids, ha!

Conclusion

I will continue to explore, learn, and experiment with Vision Pro, but already I'm blown away by the spatial greatness of actually seeing the 3rd dimension digitally. It feels very natural in the same sense as the touchscreen on the original iPhone made me giggle whenever I swiped the "slide to unlock" slider. There is a lot to unlock here; come and join me on this next frontier!