Choosing an interaction system for VR

There are a lot of components that go into developing a VR experience in Unity. A VR project uses most of the same elements as a 3D project, which are already work intensive, with the added challenge of integrating VR controllers. Fortunately, Unity is built to support 3D development and provides great infrastructure to support 3D modeling and physics. In terms of how work is divided up on the team, our artist Rachel is handling the modeling to make an immersive environment. David and I will take care of any physics that doesn’t come with Unity as well as any management systems needed to control the progression of the experience; for example, if you need a key to unlock a door, the software needs some way to keep track of that.

The other major piece is allowing the player to interact with the objects in the experience, which requires that we have a system for governing how controllers interface with objects. Our experience would be pretty sad with no interaction. In order to develop this functionality, we will use an interaction system, which is a toolkit containing the basic building blocks for making interactable objects in Unity and VR. Interaction systems usually don’t directly connect with the hardware. For the existing systems, there are a couple layers of code libraries between the hardware and the interaction system.

VR development is a little bit of a Wild West environment. There isn’t a widely accepted standard for interaction systems, nor are the existing systems polished. There are a few major interaction systems out there: the Virtual Reality Toolkit (VRTK), SteamVR (SVR), and NewtonVR (NVR). VRTK and NVR are both small projects run by an individual (The Stonefox) and a start-up (Tomorrow Today Labs), respectively. Steam VR is supported by Steam and the Valve corporation.

We originally intended to use VRTK, but recently the author announced the discontinuation of the project, which left us in need of a new interaction system. Given that we have funding for another three years, it doesn’t make sense to use an interaction system that will probably become outdated as the technology evolves. In choosing between the SVR Interaction System and Newton VR, we had to weigh the benefits and limitations of each system:

Newton VR:

– Provides more out of the box functionality. Has presets for basic physics and a bunch of simple devices (levers, switches, buttons).

– Lots of prebuilt options that can be changed in the Unity Editor.

– Small company supporting Newton VR: More risk of company going belly up and the package losing support.

– Little to no documentation.

SVR Interaction System:

– Code base seems to be geared for providing developers with tools to build their own systems.

– Established company supporting the product.

– Decent documentation.

– No ready-made simple devices (levers, switches, buttons).

Both:

– The interaction system is built around an organizing trinity of Player, Hand, and Interactable class.

– Support basic physics interactions and picking up objects.

– Both are built on top of the SteamVR API.

The main advantage to NewtonVR seems to be that there are more presets, so it would be easier to use in a project that ramps up quickly or save some time on building basic devices. The Steam Interaction System generally seems to be better crafted, documented, and supported, but is not as extensive. There were additional considerations to take into account:

– We don’t entirely know what the core interactivity of the project will be yet. Since we’re still developing the narrative to use, it’s hard to say what the primary means the user will have of interacting with the environment.

– It looks implausible to (easily) integrate both systems, so there’s no way to “choose both”. Each interaction system uses its own mapping of controller buttons onto the hand class and its functions depend on the hand class that goes with the interaction system.

– Since the code for both projects is open source, we may be able to use functionality from one interaction system to as a guide for how to build the same functionality in the other.

All in all, I think the SVR Interaction System is the better choice. It should give us all the tools we need to build any interactivity we’ll need and we can be more confident it will continue to be supported (and backwards compatible) for the lifespan of our project. The time we need to figure out what our narrative will be should give us time to construct any functionality we need, and we can use NewtonVR’s designs as inspiration for how to put together simple devices.