From prototypes to future tech: how PS VR2 was built

When PlayStation VR2 released earlier this year, it offered players a chance to experience virtual game worlds bristling with detail and immersive features. PS VR2 was the culmination of several years of development, which included multiple prototypes and testing approaches.

To learn more, we asked PS VR2’s Product Manager Yasuo Takahashi about the development process of the innovative headset and PlayStation VR2 Sense Controller, and also gained insight into the various prototypes that were created as part of this process.

Yasuo Takahashi, Sony Interactive Entertainment, Global Product Strategy & Management Dept. Section #1 Principal Product Manager

Developing the PS VR2 headset

PlayStation Blog: When did development for the PS VR2 headset start?

Yasuo Takahashi: Research on future VR technology was being conducted even prior to the launch of the original PlayStation VR as part of our R&D efforts. After PS VR’s launch in 2016, discussion around what the next generation of VR would look like began in earnest. We went back and reviewed those R&D findings and we started prototyping various technologies at the beginning of 2017. Early that same year, we began detailed conversations on what features should be implemented in the new product, and which specific technologies we should explore further.

From prototypes to future tech: how PS VR2 was built

For additional context, take a look at the PS VR2 Headset Teardown video which digs deeper into the features of the headset and its inner workings.

PSB: Which feature or design element took the longest to develop?

YT: From the concept planning phase, we were already thinking about introducing new features that would represent a generational leap compared to PS VR. These included eye tracking, headset feedback, and a single-cord setup. We also needed to include these features into the headset while maintaining a light and compact design. To achieve this, it was important to design a system that handled all the functions efficiently. Our teams worked very closely from the beginning to produce detailed technical estimates, and discussed how they could achieve an optimized design. It was a very important part of the early stages of the project, and although the research itself did not take that long, we were constantly optimizing even after the prototype was built.

Additionally, we dedicated a lot of time into making sure that the headset was comfortable to wear and hold.

PSB: Looking at these initial headset prototypes, I’m guessing that the weight significantly reduced during the development process?

YT: Yes, the initial prototype was focused on evaluating functionality rather than weight, as we were considering how to incorporate new features into the original PS VR design. We were also exploring the best system for tracking, and with our first outside-in/see-through prototype, we were simultaneously testing the see-through view camera as well.

After we performed functional evaluations, we then integrated the components into the actual system. The “tracking system study prototype” was a prototype created just prior to the integration, and although it was very large and heavy, it served an important role for evaluation.

Outside In/ See through prototype:
“The first prototype of would become PS VR2 featured IR LEDs for outside-in tracking, and a see-through view camera on the front and center part of the headset.”

Outside In prototype.
“A second prototype was created in parallel with the Outside-in/See-through Prototype. This one featured multiple IR LEDs for outside-in tracking placed on the front of the headset to test out tracking capabilities. A second prototype was created in parallel with the Outside-in/See-through Prototype.”

Inside Out prototype.
“This prototype was equipped with four cameras for inside-out tracking, and was created to evaluate the headset’s new tracking capabilities.”

Tracking camera positioning prototype
“Prototype used for considering the position and angles of the four cameras. Using a dial, the position and angle of each camera could be finely adjusted.”

Tracking system prototype
“Prototype for testing the four cameras on the front of the headset by connecting them to an evaluation board. This was performed prior to the production of the SoC (system-on-chip), which is responsible for integrating all components.”

In parallel, we also assessed the comfort and wearability of the headset. Since we already knew its expected weight, we evaluated numerous configurations of how the headset can be worn based on that estimate. We added many features, but in the end, we put a lot of thought into the materials and shape to make the headset feel lightweight. Each component itself was made lighter, the design was made as simple as possible, and the thickness of the plastic was made as thin as possible while maintaining strength. The mechanical engineers worked hard to achieve just the right balance between weight reduction and durability.

PSB: PS VR2’s eye tracking feature is really impressive– how was this developed?

YT: We wanted to push the boundaries of play with PS VR2, so we were exploring eye tracking as a new UI feature. Eye tracking technology had been around for some time and we knew it was a great fit for VR, so we wanted to make sure we leveraged it appropriately. We also looked at foveated rendering to achieve a high-fidelity visual experience, which is a technique that optimizes the resolution of where the player is looking at, while reducing rendering performance of the surrounding peripheral vision.

PSB: Did you encounter any difficulties when testing eye tracking?

YT: Some people wear glasses and everyone has different eye colors, so the feature needed to accommodate all kinds of players. To ensure the feature was tracking correctly, we spent a long-time testing and optimizing the feature with a variety of users.

PSB: Headset feedback is unique to PS VR2. How was this first added as a new feature and how did you go about developing it?

YT: Mechanical design engineers removed the rumble motor from a DualShock 4 wireless controller and attached it to PS VR to try it out. They found that it actually increased the feeling of immersion and sense of reality, though there were challenges when trying to actually integrate the feature into the headset. That’s why we created the next prototype, “Eye Tracking Evaluation Prototype Part 2” with a built-in motor, and tested how the motor affected eye tracking and head tracking.

Eye tracking evaluation prototype 2
“Testers were asked to look at different screens to test the eye tracking. The sides of the device can be opened, so that the developers can visually check the distance between the lens and the eyes. This was done because without visual confirmation, it was difficult to know if the device was being worn at the correct lens distance.”

We worked closely with PlayStation Studios to evaluate what kind of headset feedback would be the most effective based on what we learned from the prototypes. We compiled our findings into a design guide so that other developers could learn from it, and many developers used it as a reference.

PSB: What PlayStation Studios feedback did the team end up incorporating into the final PS VR2 headset?

YT: The placement of the tracking camera was adjusted based on their feedback, while confirming the range of hand movement expected in actual gameplay. When PlayStation Studios members gathered in San Francisco for an event, we brought prototypes from Japan and made adjustments based on the feedback we received there.

Skeleton prototype
“This is a prototype that allows the internal structure of the headset to be visible from the outside.”

Developing the PS VR2 Sense Controller

Check out the PS VR2 Sense Controller Tear Down Video for an in-depth look at the internal structure of the controller.

PSB: When did the development of the PS VR2 Sense Controller start?

YT: As with the headset, we kicked off discussions in 2016 and began prototyping in 2017. The inside-out method is a tracking technology that is paired with the headset, so we considered this from the initial stages. We also discussed other features we wanted to add, how many buttons the controller should have, and to what extent we could incorporate features from the DualSense wireless controller for PlayStation 5.

Members included those from electrical design, mechanical design, R&D, as well as mechanical design. The mechanical design team was integral to developing and exploring various shapes for the controller.

PSB: What were the features or design aspects that took the longest to develop?

YT: The PS VR2 Sense controller incorporates key features from the DualSense wireless controller, such as haptic feedback and adaptive triggers. We also added a sensor for finger touch detection, and we spent a lot of time adjusting the controller to make it easier and lighter to hold. We also conducted extensive user testing. PlayStation Studios has a dedicated user testing team, and we worked closely with the team in Europe to ensure that we had as many people test the controller as possible.

PSB: How did you land on the final design and shape for the Sense controller?

YT: In the earliest prototype we used a different tracking method and utilized the sphere from the PlayStation Move controller. It also included features such as adaptive triggers, haptic feedback, finger-touch detection, and L1/R1 shoulder buttons. We equipped these features in the prototype to verify which features were the most important.

Prototype 1
“This is one of the earliest prototypes. This was a prototype to test the effectiveness of adaptive triggers, haptic feedback, and finger touch detection in VR controllers. Tracking was enabled by the PlayStation Move motion controller sphere.”

The next prototype used inside-out tracking, and this was how we determined where the IR LEDs should be placed and started experimenting with several different shapes for the controller.

Prototype 2
“This prototype was built for tracking performance study,evaluating the performance when the IR LEDs (for tracking) were placed separately on the top and bottom of the controller.”

Prototype 3
“A prototype for the tracking performance study. This prototype was used to evaluate the performance when IR LEDs were placed on a ring on the back of the hand.”

This next prototype is close to its current final shape, but the main difference is the size of the grip. The shape is the result of adding various features, such as adaptive triggers, haptic feedback, and of course, the battery. However, it was too large to hold for extended game play sessions, so our mechanical and electrical engineers worked hard to optimize the grip to make it thinner and more comfortable to hold.

Prototype 4
“Another prototype for the tracking performance study. This was to evaluate the performance when IR LEDs were placed around the controller with both design and features in mind.”

PSB: What kind of evaluation process did you go through when adding completely new features such as finger-touch detection?

YT: We created a prototype that incorporated features such as adaptive triggers, haptic feedback, finger-touch detection, and other elements that achieved a stronger feeling of immersion in VR experiences. We considered various forms of finger-touch detection, including ones that required button presses.

Each time we created a prototype, we asked for feedback from PlayStation Studios and used those insights for further testing. We asked how they wanted to utilize the controller in their games, and determined which features we wanted to include and how to include them. The optimal number of buttons, for example, is something that game developers can provide the best feedback for, so we closely listened to their thoughts during development.

PSB: The final shape of the controller feels natural in the player’s hand. Why and how did you decide on this final design?

YT: Our goal was to balance the weight and center of gravity to make the controller easier to grip and hold for longer gameplay sessions. We conducted extensive prototyping and user testing in order to achieve a comfortable fit. It was not enough to simply piece the components together; we had to make drastic changes to the internal structure. The next prototype is close to the final product, but the fit is not yet fully optimized.

Prototype 5
“This prototype is equipped with all final features and incorporates the unique “orb” shape. It was also used to evaluate the tracking feature, which was close to final at this stage. The grip design and comfortability were not yet fully optimized.”

PSB: It seems like the team put a lot of effort into IR tracking during development. When creating the prototypes, did you have to make any drastic changes from your initial planning?

YT: At the start of the project, we weren’t sure whether inside-out or outside-in tracking was better, so we explored both possibilities in parallel. We split into two teams and produced prototypes for both. After that, we decided to focus on the inside-out method and continued to optimize the tracking. The prototype produced to test the placement of the headset tracking camera enabled us to fine-tune the position and angles of the four cameras with a dial. It was also used to verify the range that the cameras could detect.

PSB: Were you involved in software development at the initial research phase? What feedback did you incorporate?

YT: We couldn’t fully envision how the controller would act and perform in an actual game, so the PlayStation Studios team provided feedback on the proposed product specifications early on in the technical review process. For example, if the player is standing and grabs an object at the waist, the head must remain facing forward, so we made sure the camera could detect an object below the headset. On the other hand, if you are pulling out a bow and arrow from your back, your arm will not be in the range of the camera. In instances like this when the controller was out of range from the camera, we had to make adjustments based on feedback, so that the tracking felt more natural.

Optimizing ergonomics while incorporating both adaptive triggers and haptic feedback was challenging, but we followed through due to strong demand from the software development team. The adaptive triggers must be located near the fingertips, while the haptic feedback needs to be incorporated where players grip the controller. The adaptive trigger modules are reasonably large, so if you account for the modules as well as the battery, various components end up crowding the grip area. This is not ideal ergonomically and sacrifices comfort. Originally, we intended on utilizing the same adaptive trigger components as the DualSense wireless controller, but we ended up creating a customized, dedicated module for the PS VR2 Sense controller.

All the prototypes for PS VR2 headset and the PS VR2 Sense controllers.

PSB: Finally, what are you proudest of with regards to the final PS VR2 headset?

YT: The original PS VR was used by many game developers to expand the world of VR games, and we believe that the PS VR2 pushes the boundaries even further. PS VR2 is capable of rendering stunning 4K HDR visuals (2000×2040 per eye), and eye tracking also delivers high-quality graphics with efficient rendering techniques. Combined with the controller, headset and comfortable design, players can fully immerse themselves in the extraordinary VR worlds created by game developers. The diverse portfolio of VR games will continue to expand in the future, and we hope that players will enjoy them to the fullest.