As early as in science fiction movies decades ago, holographic communication is the representative of future technology, a technology that combines virtuality and reality. This year, the “parity” of the future technology has been realized: the two parties have achieved holographic communication with “real people” through Microsoft 3D cameras, OPPO 5G smartphones, Nreal’s AR glasses and the 5G network environment provided by Ericsson. Among them, the total price of 3D cameras, 5G smartphones and AR glasses is over 10,000 yuan.
The realization process of such “grounding” has made holographic communication the focus of the industry again: can holographic communication, which is not available in 4G network, become a popular application of 5G network?
The appearance of holographic communication in 5G is an inevitable trend
Holographic communication requires real-time capture, encoding, transmission and rendering of each other’s 3D images, which are then displayed in their own specific space.
Wang Haobo, deputy general manager of Ericsson China Technology Department, said that there are three ways to realize holographic communication. One is to extract human facial expressions very simply and reproduce them on the other end in the form of animation. The effect is not very good; the second is to extract human facial expressions very simply. It can often be seen at some major scientific and technological events. The overall transmission of 3D images of people to the scene is achieved through very sophisticated and complex instruments, a special transmission network and a very professional projection environment. The effect is very good. But it is still very far away from the application of ordinary people. “We use the third type, which is to use commercial-grade equipment and commercial networks to realize holographic communication that the common people can use.” Wang Haobo said.
Holographic communication needs to present a holographic effect. First, the size must be large, close to the size of a real person; second, it must have a sense of space, which is also the difference between holographic images and video images. Hu Xianlong, vice president of Nreal, told the reporter of China Electronics News: “The imaging method of AR glasses enables us to see a holographic image that is close to life size without having to use a large display screen. The video communication images we see on mobile phones and computers, the experience is completely different.”
Tang Hai, vice president of the OPPO Research Institute and chief scientist of 5G, told the “China Electronics News” reporter that not only is the 5G network more powerful, but the computing power of the terminal has also reached a very high level. For example, the latest terminal chip of 5nm, its processing power has exceeded the processing power of computer CPU a few years ago. Stronger network computing power, transmission capacity and terminal computing power, coupled with the presentation ability of Nreal glasses, on the whole, it is inevitable that holographic communication will appear at this time.
5G can solve the pain point of holographic communication
The amount of uncompressed information in holographic communication is very large. If the camera sensor outputs a 1080P image, plus color and 3D depth images, the raw data of each frame will be around 70MB. If the resolution and frame rate are to be further increased, more bandwidth is required. Some analysts pointed out that for a 70-inch display, holographic communication requires about 1Tbit/s of network bandwidth.
When holographic communication is actually implemented, the whole process starts with capturing the real-time “expressions” of the face and body provided by the sensor, and then encoding it. Do format conversion and filtering before encoding to reduce the requirement for network transmission rate. The encoded and compressed hologram is transmitted to the XR device through the 5G low-latency network. On the XR device, the compressed image information needs to be decoded and processed first, and then rendered to the user through rendering.
5G can solve some of the pain points of holographic communication in this process very well.
Compared with 4G, the connection capability of 5G has been greatly improved. The average speed of 100M can reach 2Gbps in peak value, and the network delay can also be controlled at the millisecond level without obvious lag. Therefore, from the perspective of capability, it is feasible to use 5G network to carry holographic communication.
Holographic communication is a process of information collection, compression, transmission, restoration, and rendering. If the process of re-restoration and re-rendering is implemented on the terminal, the terminal needs to have high computing power. 5G can alleviate the need for terminal computing power. “5G has the ability to intelligently allocate computing power in real time, and 5G also has edge computing capabilities, which can provide computing power support very close to users.” Wang Haobo said.
In addition, 5G can also provide experience guarantee for the application of holographic communication. 5G has the function of network slicing and can reserve resources for holographic communication.
Holographic communication should solve the shortcomings of capture technology
“As a killer application in the future, holographic communication requires the full cooperation of the entire industry chain.” Tang Hai said, “The development level of the entire industry chain has reached a certain stage, and it is possible to use it in mobile phones or AR glasses. In a small-sized device, it provides sufficient transmission, computing power, and presentation for holographic communication, including very stable spatial perception, viewing angle, and very good portrait modeling. Only then can a better user experience and truly usable holography be brought. Communication, I am sure that day will come soon.”
Talking about the support of the whole industry chain, Hu Xianlong believes that the obvious shortcoming of holographic communication is information collection. “In the first stage of holographic communication development, the collection of holographic information is very important. We hope to have a more efficient and low-cost collection end for holographic models.” Hu Xianlong said.
At present, Intel, Microsoft, and iPhone already have relevant acquisition functions, and domestic enterprises still need to work together in this regard. Tang Hai said that the current information acquisition of holographic communication is based on high-precision 3D depth cameras, which require very high computing power processing capabilities.
Some experts also refer to the acquisition function as capture technology. Holographic capture is the process of creating a measurable 3D display of an object, person or environment, including four steps: acquisition, depth estimation, data fusion, and post-processing. Acquisition is the use of visual sensors to capture various useful information. 3D capture is currently most common using ToF (time-of-flight sensors), which measure distance by calculating the time it takes for a pulse of light to reach its destination and return.
In depth estimation, ToF can directly provide depth information, while stereo cameras and multi-cameras estimate depth through information captured from different angles. Data fusion is the fusion of depth information from different perspectives into a single 3D point data stream. Post-processing is to reduce the amount of data in the 3D point data stream by removing redundant points, noise, and outliers. 3D can generate point clouds or meshes in visual media formats. Real-time data captured by depth cameras requires more computing power to render than avatars.
“4G is the video era, and everyone can shoot short videos. This was unimaginable in the past, because to make a video in the PC era requires professional cameras and editing software to achieve various conditions.” Hu Xianlong said, “In the future, if a few-year-old child can obtain a holographic model or video with simple tools, and achieve a better display effect than now, the explosion of holographic communication will come, and I think this will be realized soon.”
GIPHY App Key not set. Please check settings