Skip to content

Blogs

Comparing VR headsets’ tracking performance

Consumer VR headsets have been available since 2015, when a large number of companies announced their own versions of the products. By 2016, more than 230 companies were working on their own VR HMD projects. Most of these projects have since ended and companies have gone bankrupt, but some keep releasing new products, both stand-alone and tethered versions, on an almost annual basis. Oculus (Facebook) released new Quest and Rift devices in 2019, while HTC came up with a new VIVE and Valve announced their Index glasses. Then there is Varjo, a company that keeps amazing their fans with human-eye resolution products, as well as numerous new Chinese companies that are also catching up fast in product design and performance. In CES 2020, there were dozens of new players and products coming to both the VR and AR markets. [1] [2]

While we have all these awesome new products available, the actual sales figures also tell an interesting story. To this day, the biggest market share of VR headsets belongs to Sony. Sony announced their Project Morpheus (PSVR) in 2014 and released it in October 2016. By March 2019, PSVR had sold more than 4.2 million units. According to SuperData’s Q4 2018 XR market report, Sony’s PSVR became the market leader with sales of 700,000 units — the largest number of headsets sold across any category. By comparison, the standalone Oculus Go has sold 555,000 units, with the tethered Oculus Rift and HTC Vive selling only 160,000 and 130,000 units, respectively. [3][4][5]

How did Sony do this? It’s obvious that the PS4 ecosystem and the ease of accessing VR content play a big role in this. Most of us tech geeks would still like to know how the PSVR performs in terms of tracking and image quality, since these are the main factors that define the HMD quality. There are a lot of good news articles and blogs in the web that break down the fundamentals of AR/VR HMD immersion and the technological challenges in meeting the standards of today and the future. For further reading, we recommend checking out UploadVR, NextReality, RoadtoVR, Daniel Wegner’s article and Steven Lavalle’s book Virtual reality.

OptoFidelity has been designing and building testing and calibration systems for VR/AR products and components for years. These systems cover solutions for movement tracking accuracy and latency as well as for optics and image quality. Recently, Oliver Kreylos did a great review of PlayStation’s VR headset FOV and resolution. We wanted to contribute to this and run some tests to see whether PSVR’s tracking performance stands up against some other popular headsets in the market. [6]

Test setup

Our measurement setup consists of OptoFidelity BUDDY-3 equipment and several VR headsets including Sony PSVR, HTC Vive, Oculus Quest and Valve Index. OptoFidelity BUDDY-3 tests the head tracking performance of VR/AR headsets using a robot, a smart camera and signal processing algorithms that allow determining the orientation of the virtual content in a non-intrusive manner by just imaging the display. The absolute tracking method is covered in detail in a blog post on the OptoFidelity website.

When the robot moves the headset, the robot’s actual motion is captured by an encoder counter device which is synchronized to the same clock as the camera. This allows temporal comparison of the content and robot orientation signals and calculating the motion-to-photon latency. 

BUDDY SONY test setup 2

Figure 1: OptoFidelity Buddy-3 test setup

Drift testing

Drifting means how well the headset can anchor itself to the environment or given position. Typically this tells how successfully SLAM (simultaneous localization and mapping) has been done.

The robot was moved to random positions at full speed for one minute. This was repeated 15 times with a 10-second pause between each run. The images on the figures show the drift on each axis at the end of the 10-second pauses.    

drifting                         

Figure 2: Sony PSVR        

drifting-1                   

Figure 3: HTC Vive

drifting_summary

Figure 4: Oculus Quest                                                                     

drifting_summary-1

Figure 5: Valve Index

Jitter and stationary jitter

Jitter and stationary jitter are typically caused by inaccurate sensor data or in most cases problems within the motion prediction algorithms. The user can perceive it as an unstable or vibrating image while moving or staying still. Jitter and tracking error can be visualized by plotting the difference of the virtual world pose to the real-world robot pose.

jitter

Figure 6: Sony PSVR                                                                                 

jitter-1

Figure 7: HTC Vive

jitter-2

Figure 8: Oculus Quest                                                                             

jitter-3

Figure 9: Valve Index

Stationary jitter

The robot was kept still for one minute and then moved.

stationary_jitter

Figure 10: Sony PSVR                                                                               

stationary

Figure 11: HTC Vive

stationary-1

Figure 12: Oculus Quest                                                                           

stationary-2

Figure 13: Valve Index

Motion-to-photon (M2P) latency

M2P latency is the main factor in causing motion sickness for the user. Today latencies below 5ms are considered acceptable in VR and near 0ms latencies the same for AR. Latency was measured by moving the robot along each axis separately and comparing the pose to the content.

mtp

Figure 14: Sony PSVR                                                                             

mtp-1

Figure 15: HTC Vive

mtp-2

Figure 16: Oculus Quest                                                                           

mtp-3

Figure 17: Valve Index

Tracking

General tracking performance measurement was done by moving all the robot axes simultaneously and calculating the difference between the encoder data and rendered content image in the headset.

tracking

Figure 18: Sony PSVR                                                                               

tracking-1

Figure 19: HTC Vive

tracking-2

Figure 20: Oculus Quest                                                                           

tracking-3

Figure 21: Valve Index

Summary

The general tracking and latency performance have developed a lot since the first VR equipment was introduced to the market. Even though the tracking accuracy and latency differences between HMD models seem very small, they might have a huge effect on the immersion and possibly cause motion sickness to the user. PSVR seems to stand its ground in terms of tracking performance against its newer competitors. As mentioned earlier, it is or has been mostly the content and ecosystem that drives Sony’s PSVR sales. However, sales figures do not tell the actual utilization rates of these purchased devices, especially in the VR market.

In the coming years, a growing number of devices will rely on an external computing unit for signal processing and image rendering. Be it cloud or the mobile phone in your pocket, the data transfer will be a big factor in the tracking performance.

We at OptoFidelity are committed to help the AR/VR industry develop and build the best devices for the market! If you have any questions or feedback, please do not hesitate to contact us, and our experts in our US, EU and APAC offices will support you.

References:

1: https://www.wired.com/2016/04/magic-leap-vr/

2: https://mp.weixin.qq.com/s/6ysEyBG1ik8c34SB6jMXpw

3: https://www.statista.com/statistics/671403/global-virtual-reality-device-shipments-by-vendor/

4: https://www.viar360.com/virtual-reality-market-size-2018/

5: https://uploadvr.com/editorial-vr-devs-market-size/

6: http://doc-ok.org/?p=1882

Get in touch to learn more about our test systems.

We're happy to answer any inquiries.