[Shenzhen, China, Dec 22, 2021] Smart vision is a critical technology behind new types of infrastructure, and tomorrow's smart solutions. As picture quality (PQ) technology and AI technology continue to mature and mutually reinforce each other, HiSilicon (Shanghai) has doubled down its commitment to state-of-the-art smart vision technologies and solutions, by cultivating innovation in device-side PQ and AI technology.

Vision is not just fundamental to how we perceive our world; it's also fundamental to how smart devices operate, from consumer electronics and intelligent vehicles, to traffic management and intelligent manufacturing. Visual technology has been part and parcel to the development of a whole host of industries. The evolution from mere "sight" to clear "perception" has occurred alongside the evolution from analog to digital to networked intelligence. HiSilicon has been a key player in these trends, working in-depth on core technologies like image signal processing (ISP), video encoding and decoding, and AI computing. Thanks to its clearer and smarter visual SoC solutions, HiSilicon has spurred groundbreaking developments in the HD to UHD sections, working hand-in-hand with industry partners. This has improved all-hours image quality, and helped extract key insights from data, which has enormous implications across industry.

"Perception" as the New Paradigm

The major challenges for visual technology are capturing images on a 24/7 basis, and collecting valuable data, as seen in the development of smart cities and smart vehicles. Though we have entered an era of 4K/8K UHD, image clarity remains a drive for industrial upgrade. A common issue is capturing images in backlit and low light environments, which results in poor video resolution and blurry objects, thereby lowering the AI recognition rate.

HiSilicon's deep dedication to the ISP technology brings about constant innovation, which helps delivering superb image clarity regardless of back light or low light conditions.

In low light conditions, HiSilicon manages to deliver industry-leading low light image quality, by utilizing 8-stage 3DNR and multi-spectral fusion, among other cutting-edge technologies. 8-stage 3DNR technology provides for noise reduction and details enhancement during spatial filtering, as well as the joining of two different time modes during temporal filtering. It includes an image enhancement (IE) module for details enhancement, and a single sharpening module, which enhances noise reduction while also preserving more image details. Multi-spectral fusion technology delivers vivid, colorful images in low light conditions, with overexposed and underexposed regions fine-tuned for optimal accuracy. This was achieved by using an infrared light source to compensate, which avoids light pollution while fusing both visible light and infrared light in the ISP.

In dimly-lit underground parking lots, the technologies mentioned above work together to deliver vibrant color, vivid details, and exceptional noise reduction, accurately capturing information and minimizing blind spots.

In back light scenarios which require high dynamic range, HiSilicon adjusts the exposure strategy in real time, generating enhanced imaging in challenging areas such as tunnel entrances and exits, via identification and classification of target image areas. In static scenarios, HiSilicon's SoCs significantly outperform other solutions in terms of object recognition, overall details, definition, and saturation. In dynamic environments such as tunnel entrances and exits, HiSilicon technology features accurate exposure and timely adjustments, which help bolster driving safety.

How Intelligence Has Unleashed New Applications for Smart Vision

Ongoing improvements to data collection, computing power and algorithms in recent years, especially regarding the interaction between video data and AI, have spurred groundbreaking smart device solutions. Significant enhancements to image quality made possible by full networking and full HD, have worked in concert with mass data collection capabilities. Intelligent conversion from video to structured data has become the focus of industry development. HiSilicon began deploying AI visual processing SoCs in 2016, and launched cross-era flagship processors in 2018, pioneering the development of AI and vision-centric sectors, by helping industry partners quickly obtain valuable object information across a wide range of scenarios.

Due to the unique characteristics of different devices, numerous factors such as computing power, cost, and power consumption need to be considered. Deep neural network computing is based on a large number of matrix multiplications, and the most basic unit is the multiply-accumulate (MAC) unit. HiSilicon uses the in-house Da Vinci Architecture to accelerate matrix multiply-add operations, and offers a next-generation, cost-effective Da Vinci AI engine that is tailored for smart devices. Thanks to full-scale hardware and software optimization, both performance power area (PPA) and MAC usage far exceed industry norms. Robust computing power is not sufficient on its own. To satisfy a wide range of different application scenarios and development requirements, developers urgently need better and more handy development kits. HiSilicon AI SoCs support all mainstream development frameworks, including PyTorch, TensorFlow, Caffe, and MindSpore, as well as Open Neural Network Exchange (ONNX), flexibly adapting to different development frameworks.

In addition to neural network-based applications, HiSilicon has analyzed the features of smart vision applications in-depth, and integrated computer vision operators on hardware, such as image resizing, color space conversion (CSC), basic filtering, object detection, license plate recognition, and 3D depth detection. This has proved more efficient in terms of computing, and more convenient for developers. Applications range widely, from drones, robots, and automated guided vehicles (AGVs), to assisted driving, and smart cities.

As vision enters a newly intelligent era, smart vision will soon become indispensable to a range of everyday services. For example, smart parking solutions can bring parking lot management into the unmanned era. Cameras and AI will be used to automatically identify license plates, reducing labor and card maintenance costs, while also improving driving efficiency.

From card swiping to contactless clock-in, or manned gates to quick commuting, smart vision has enormous implications for office efficiency. During the Covid-19 pandemic, physical clock-in/out and contact fingerprint-based clock-in/out were gradually replaced by smart clock-in/out solutions. The multi-channel smart vision capability was applied to many smart clock-in/out machines and passage gates. Thanks to highly-integrated, multi-dimensional perception, commuting efficiency has improved by leaps and bounds, with reduced contact between personnel.

Greater Cohesion Between Vision and AI Is Inevitable

Traditional image processing technologies, such as those based on filters, tend to encounter bottlenecks in low light and wide dynamic range scenarios, meaning that it is increasingly difficult to make significant improvements. AI-based image enhancement has become a new research focus, and significant progress has already been made. According to HiSilicon's research, utilizing neural network (NN) technology for noise detection and removal, and utilizing the multi-spectral fusion technology, can better distinguish moving objects from static backgrounds, while also optimizing the noise form and SNR. The intelligent recognition rate has soared to 95% and above at 0.1 lux. In addition to low light noise reduction, AI is also applicable to a range of other scenarios, such as dynamic range compression (DRC). In recent years, AI-based optimization of photos has been broadly applied on high-end mobile phone cameras to widespread acclaim. Visual sectors require real-time AI optimization for high-resolution and high-frame-rate video, which places higher requirements on algorithms and computing power. In particular, implementing the AI ISP on devices with limited computing power, and using it to generate images and videos that are clearer than those achieved via traditional ISPs, are the new frontiers that HiSilicon and industry partners are exploring.

Vision and AI are key technologies that will bolster tomorrow's infrastructure and next-level smart device capabilities. With massive improvements to data collection, algorithms, and computing power, convergence is inevitable, and new solutions will be broadly applied across industries. HiSilicon's prowess in perception, computing, connectivity, expression, and semiconductor engineering makes it perfectly positioned to blaze new trail. In the visual domain alone, HiSilicon has extensive experience and industry-leading core capabilities related to on-device visual perception and AI computing. HiSilicon will continue to invest and innovate, with the goal of delivering cutting-edge technologies and solutions, addressing smart city, smart home, and smart transportation field, with a robust semiconductor foundation to bring smart device performance through the roof.