Thursday, November 26, 2020

Explaining the Need for Varifocal Lenses in VR

 Vision comes with a few components that give you a sense of depth. Parallax is the effect where (over time) moving a perspective shows near objects moving faster than far objects giving the effect of depth. Stereoscopic vision is similar but you have two perspectives in different places which place near objects further apart than far objects and your mind creates a sense of depth from that. Finally there is focal distance. Focal distance can give you an effect of scale and distance from a single perspective. Much like 'tilt-shift' makes things look small, messing with the focus can emphasize the relative and even the absolute distance and size of objects.

The Index & Rift have a set focal distance of about 2 meters while the vive has a focal distance of 0.75m. That means that if you look at an object at that exact distance then it will be perfectly in focus in addition to having correct stereo and parallax effects. When you look at an object at a different distance then the dissonance in the focal distance from the other effects causes discomfort and 'blurriness'.

Focus on your finger a few cm from your eye with a background a few meters away. Move your finger further and further until you notice the background come into focus. You'll notice that there's a bit of a logarithmic scale in terms of how much the focal effect presents itself based on distance. The difference in focus between 2meters and infinity is a lot less significant than the difference between 5cm and 25cm.

Effectively, current headsets are well tuned to keep objects in the same virtual room as you in focus until you get to close. Objects in the far distance aren't easy to make out because of low resolution but if they weren't, you would see that they are also slightly blurry because they're out of focus. Near objects like your hands that you bring to your face will go out of focus a lot more easily and they're also large relative to the display so resolution isn't an issue. They will be stereoscopically accurate but completely out of focus. You should be able to notice that if you close one eye and try and focus on an still object (like a book with text) near the camera, you should be able to focus on it clearly and even read it easily but the scale will appear to be incorrect and very large.

Varifocal lenses can either move the lenses like a camera to change the focal distance based on your eye movements and where in the virtual world you are looking or they can toggle lenses on and off to match the focal distance you should be seeing as closely as possible. The new issue that arises is that now all objects in the scene are going to be in focus all the time regardless of where you are focused. While better, this is still quite inaccurate. The solution can be digital by artificially blurring all objects as accurately as possible but there are also ways to have multiple displays with different focal distances or even change the focal distance differently in different parts of the screen.

Eye tracking is a prerequisite for such tech so foveated rendering will appear first in consumer headsets then we'll probably have to wait another generation to get this tech. I've used the Half-dome oculus prototype varifocal headset and while it's nice to be able to focus correctly, the digital blurring and eye tracking leave a lot to be desired and honestly, I think improving refresh rate and FOV along with adding effective foveated rendering will provide far larger improvements than varifocal lenses.