[ShangHai, China, March 10, 2021] XR is an umbrella term for VR, AR, and MR technologies, and refers to the integration of digital and physical worlds, that has resulted from groundbreaking advances in computing, connectivity, and display. XR has not only revolutionized the display field, but also reimagined human-computer interactions, and has facilitated the opening of a new market that's estimated to already be worth tens of billions of dollars.

Virtual reality (VR) uses modern means such as computers to virtualize and reconstruct the real world. This technology enables users to freely interact with objects in three-dimensional space, and in real time. VR emphasizes the leading role that humans play in virtual systems. In the past, people could only observe the computing results from outside of a computer system, and obtain one-dimensional digital information produced by the keyboard, mouse, and other devices. But now it's possible to interact with multi-dimensional information using multiple sensors.

VR has enabled viewers to enjoy new perspectives over the air, for instance, during the broadcast of the 70th anniversary National Day parade in China, with the placement of VR equipment on both sides of Chang'an Street. VR is also being widely applied in the field of education. By wearing VR glasses, students in remote rural areas are able to study the geographical features of Australia in a fresh and more immersive manner, and even learn about the transmission of blood in the human body by literally "seeing" it in action. These next-level experiences have helped pique students' interest, and VR has since been applied in engineering training, as well as other fields.

Augmented reality (AR) technology integrates information from real and virtual scenarios. AR glasses integrate display, interactive, sensing and multimedia technologies, displaying virtual information through a first-person lens, for ultimate multi-sensory immersion. AR offers greater commercial potential than VR. After several years of steady user growth, AR has not only made significant strides in entertainment and leisure applications, but also made inroads into a wide range of 2B application fields, including security, heavy industry, tourism, and healthcare. In heavy industry for instance, AR smart glasses have been able to streamline industrial operations and improve work efficiency through remote assistance.

Mixed reality (MR) technology combines the real world with the virtual world to create a new visual environment, in which physical and digital objects coexist and interact with each other in real time. MR is a mix of both VR and AR, introducing real scenario information into the virtual environment, as well as setting up an information loop between the virtual world, real world, and users to further enhance the authenticity of the user experience. MR has managed to transcend the limitations of completely virtual VR technology, exploring methods of human-machine interaction, and expanding the scope of commercial applications.

Challenges for Existing XR Devices

Low resolution: Let's take VR as an example. The ideal resolution for human eyes is about 60 PPD (pixels per degree), and in general, a chipset capable of 4Kp60 decoding can support single-eye resolution of 21.3 PPD, which is equivalent to 480p TV display effects, far short of optimal comfort and immersion. If the chipset can be upgraded to 8Kp120 hard decoding capability, twice the resolution under current mainstream VR will be attainable, that is, single-eye resolution of 42.7 PPD, which can result in a significantly enhanced visual experience.

Dizziness: Dizziness can detract from a highly immersive experience. 25% of consumers attribute dizziness as the primary reason for not choosing to purchase an XR device. The delay caused by insufficient CPU processing performance is a main cause of dizziness. This delay causes the information received by the visual system to be out-of-sync with the information perceived by the body. As a result, the user's visual system conflicts with their other sensory channels, creating a sense of disorder.

Powerful NPU Computing Capabilities, Facilitating an Excellent User Experience

NPU-based intelligent scene identification is a core function in AR. Scene identification involves image capture, analysis, and upload to the display. The shorter the time required for this, the faster speeds are, and the better the user experience is. The analysis phase is the most time-consuming part of this process. In order to recognize and analyze hundreds of faces and license plates in mere milliseconds, chipsets must have extremely powerful NPU computing capabilities. Cloud-based AR/VR places rigorous requirements on transmission bandwidth and latency. According to the Huawei X-Labs white paper, hundreds of Mbit/s to several Gbit/s bandwidth and millisecond-level latency are required to ensure a good experience. WiFi 6 is thus better equipped to meet these requirements than traditional 4G and WiFi.

Comprehensive XR (VR/AR/MR) Solution

HiSilicon offers a comprehensive XR (VR/AR/MR) solution, which encompasses the application processor (AP), display, visual, and connectivity components.

High-performance computing:
● Independent in-house architecture NPU, providing up to 9 TOPS of AI computing power
● 8-core high-performance Cortex-A73 processor and high-speed computing to avoid latency

UHD display:
● 4K ultra-HD low-latency encoding, supporting XR remote immersive video conferences and remote assistance applications
● AI PQ enhancement
● XR audio and audio effect processing

Low-power smart sensing:
● Low-power AI visual solution, human/object recognition, and AI visual enhancement
● Intelligent voice solution
● SLAM/6-DoF application

High-speed, low-latency connections:
● Low-latency image transfer, supporting cloud VR applications and short-distance wireless image transfer applications
● High-speed and low-latency connection solution (WiFi 6)

Module WiFi 6/BT Camera

XR products (VR/AR/MR) serve as the gateway to a future lifestyle, bridging virtual and real worlds, with enhanced connections, mapping, and mutually-supporting capabilities. Improvements to XR products will address immersive effects, interactions, end-to-end latency, and wearing comfort. HiSilicon chipsets support 4K/6K/8K ultra-HD decoding displays, 120° FOV, and AI real-time image quality enhancement, to provide an immersive viewing experience. The chipsets incorporate high-performing computing modules, and integrate algorithms such as SLAM and real-time rendering to provide consumers with smooth and natural interactions. In addition, high-speed transmission technologies, such as WiFi 6, minimize transmission latency, while more cohesive chipset integration lowers power consumption and reduces the product size and weight, making it more comfortable to wear for longer periods of time.