HiSilicon Launches New XR Platform with Rokid

HiSilicon has joined hands with Rokid and other device vendors in releasing the extended reality (XR) platform, which is based on an XR chipset that supports 8K decoding and packs a high-performing GPU and NPU. In order to adapt to the high resolution, wide field of view, and low latency in AR/VR scenarios, HiSilicon partnered with Rokid, a leading AR company in China, to optimize the platform's dedicated algorithms and solutions. The collaboration has resulted in bolstered AI computing capabilities, in particular those related to voice, vision, and environmental perception, and borne fruit with a high-resolution, low-latency design for multi-channel video encoding and decoding. The two tech companies have provided the XR sector with core capabilities for AI, interactions, and multi-person collaborations, encompassing chipset specifications, system software, and cloud services.

As an emerging industry already worth more than $10 billion, and an annual market growth rate of more than 70%, the AR/VR market has expanded by leaps and bounds in recent years. AR/VR technologies are widely applied in education, health care, entertainment, and real estate, among other fields, but current AR/VR devices still fall short in offering the optimal experience that today's users have come to expect.

Low Resolution

VR serves as a representative example of how user experience requires further improvement. The ideal resolution for human eyes is about 60 pixels per degree (PPD), and in general, chipsets that support 4Kp60 decoding are able to achieve single-eye resolution of 21.3 PPD, which is equivalent to the display effects on a 480p TV, still far from comfortable and immersive viewing. The 8K hardware decoding capability, by contrast, supports 42.7 PDD for a single eye, providing users with noticeable clearer content.

Dizziness

Dizziness is caused by visual fatigue, and more specifically, inconsistencies between the human visual receptor, balance receptor, and other receptors. Poor image quality related to the screen-door effect, smearing, or flickering can tax the user's eyes, and result in a dizzying sensation.

Simply put: The user sees movement without sensing it, or senses movement without seeing it. In XR scenarios, movement is often inconsistent with the image, due to the presence of latency. Latency is generally caused by two different factors, either the failure of the CPU to process data in a timely manner, or a subpar network. The HiSilicon XR platform supports a high-performing octa-core processor. Its superior CPU processing capability dramatically boosts device computing speed. The platform also optimizes the protocols for network performance to reduce latency caused by the network port when processing data, thereby reducing dizziness to a remarkable extent.

An important application for AR products is facial recognition and license plate recognition. Cameras in AR glasses, for instance, collect and analyze video images in real time, recognizing faces or license plates in each frame, before uploading the recognition results to the server for comparison, and displaying the comparison results on the screen in real time. The shorter the time for image collection, analysis, upload and display, the better the overall user experience. Analysis is the most time-consuming phase during this entire process. Hundreds of faces and license plates need to be recognized and analyzed in just milliseconds, requiring that the chipset possesses formidable NPU computing capabilities. The XR platform possesses an independent HiSilicon-owned NPU architecture, which is supported by up to 9 TOPS of NPU computing power.

Rokid Vision, jointly developed by Rokid and HiSilicon, represents the first-ever AR product to be based on the XR platform. This binocular AR product comes equipped with the pioneering optical waveguide imaging technology, and features brightness, permeability, and field of view (FoV) that outperform those of existing products on the market. Due to these standout attributes, Rokid Vision is widely expected to become a trailblazing product.

HiSilicon will continue to collaborate with Rokid and its other partners, to provide comprehensive XR product solutions, including chipsets, core algorithms, software systems, and software as a service (SaaS), through its innovative XR platform. This broad array of technology will promote the development of the AR/VR industry, and help establish a new paradigm.