Building Multiparty VR Applications with WebRTC

The virtual reality industry has recently experienced a shift. What was once viewed as a predominately solo activity-centric experience, has become an avenue for developing rich social connections and collaboration. Developers have realized that virtual reality does not need to take place in isolation and are starting to look for ways to increase social interaction in their applications.

But creating “social” virtual reality experiences is not simple. While there are server-based products on the market today that allow for the integration of voice communication, multi-party collaborative live video and data has posed a significant hurdle for AR/VR application developers.

One of the most exciting possibilities for enabling multi-party virtual reality experiences is Web Real-Time Communications (WebRTC). In this post we will explore how WebRTC can be used to make shared, cross-platform VR experiences possible and describe some benefits its integration brings to the AR/VR space. The featured image showcases LiveSwitch customer Cavrnus™ and their use of LiveSwitch Video Stream for its Metaverse platform builder.

Benefit #1: WebRTC Can Transmit Real-Time Video Between VR Headsets and Web Browsers

Web Real-Time Communications (WebRTC) is a standard that seeks to embed real-time voice and video natively between web browsers. With WebRTC, end-users are able to communicate in the web browser directly with each other via video without installing special plug-ins, extensions, or using a closed ecosystem communication vendor.

If you want users to transmit real-time video directly from an AR/VR enabled device to a browser without installing plugins or extensions, then WebRTC is currently your only option. In an excellent session at ClueCon in the past, Dan Jenkins, a WebRTC expert, demonstrated that it is relatively easy to add WebRTC video conference streams into a virtual reality environment.

The LiveSwitch advantage: LiveSwitch's API supports peer-to-peer connections but moves far beyond basic WebRTC capabilities by also scaling for multi-party conversations, broadcasting, SIP-based telephony integration and more. Flexible architectures like the one used in LiveSwitch Cloud reduce costs by choosing your application’s optimal topology. 

Benefit #2: WebRTC for Low-Latency Audio and Video

Benefit #3: WebRTC is Multi-Platform (Sort-Of)

According to Tsahi Levent-Levi, a WebRTC industry leader, WebRTC is supported (to varying degrees) in a number of different browsers and platforms including Chrome, Firefox, Safari, Edge, Android and iOS. This cross browser/platform compatibility makes it easier for virtual and augmented reality companies to create applications that function across web, mobile and desktop applications. However, this list is far from complete and does present some gaps in device usage.

The LiveSwitch advantage: LiveSwitch was designed to be one of the most versatile, real-time communications SDK and server stacks available on the market today. It has seamless interoperability and functionality across web, mobile and desktop environments. It supports a massive range of languages, platforms and browsers including Windows, Linux, macOS, iOS, Android, Java, Xamarin, Firefox, Explorer, Chrome, Safari, Opera and UWP making it one of the most flexible real-time voice and video products around. This cross-platform support makes it easier for developers to create applications for one platform and transfer that code to other platforms.

Benefit #4: Ultra Low Latency Datachannels Too

Sending and receiving data between two end-users can be a complicated process. However much of this headache can be alleviated by using WebRTC’s RTCDataChannel API to transfer data directly between peers.

The RTCDataChannel API supports a flexible set of data types which means that virtual reality developers can use them to deliver any type of data that they need to make multi-party VR experiences a reality. All of the properties necessary to build 3D voice and video such as coordinates, color, depth texture, 3D sound and mesh data can all be sent from one peer to another using one or more data channels.

The LiveSwitch advantage: While WebRTC data channels make it easy to transfer data from peer-to-peer, LiveSwitch takes it a step further.

This means that you can send data to N people without taking on Nx bandwidth that is normally incurred in a simple peer-to-peer mesh network. In simpler terms, if 5 people were participating in a virtual hangout, and one participant needed to send everyone 3 Mbps of data, with LiveSwitch that person would only need to upload 3 Mbps of data in comparison to the 15 Mbps required for a peer-to-peer mesh virtual reality session. This number is unchanged regardless of the number of participants in a session. This ultimately reduces overall bandwidth utilization and improves headset battery power consumption which makes multi-party virtual reality that much more accessible for everyone.

Need a live video API engineered for robust, scalable growth to drive the next stage of your VR/XR/AR application's revenue growth? Contact us today to see how LiveSwitch Inc's APIs can help you grow.