From the current product form on the market, users’ dialogue in the virtual space has been realized, and the solutions of some game and hardware manufacturers can also support the transmission of the sense of hitting in the game to the human body. Baidu, NetEase and other companies have also released hope Metaverse products such as soil and Yaotai are available for users to participate and interact in virtual space. However, on the whole, there is still a big gap between the metaverse experience solutions currently available on the market and the metaverse immersion experience outlined by the movie.
Can users hug and shake hands in the metaverse as described in the movie? How long will it take for a fully immersive experience?
“Human” Metaverse
“The core of the so-called metaverse is people.” Yuan Yu, a senior analyst at CCID Consulting, said in an interview with a reporter from China Electronics News.
The core problem to be solved by the Metaverse is to incorporate people into the virtual world, make the human body identify with the perception of the virtual world, and immerse the human body in the experience of the virtual world, so that people can interact and communicate as in real life. “Human” is the core of building the Metaverse, and user “interaction” is an important feature that distinguishes the Metaverse from previous stand-alone games. In order for the human body to have the same experience in the virtual world as the real world, facial expressions, body interaction and tactile perception are necessary to realize the immersive experience of the human body. Correspondingly, facial recognition, motion capture, tactile recognition and tactile feedback technologies are the key technologies to help the human body achieve a more immersive experience beyond the audio-visual senses.
“Using the facial recognition device, users can see each other’s expressions just like in real life.” Wang Congqing, president of HTC China, said in an interview with a reporter from China Electronics News. The face tracker launched by HTC Vive can be mounted on VR glasses and capture human expressions through cameras and convert them into virtual characters.
VIVE face tracker
In terms of motion capture, various equipment manufacturers have roughly launched two technical routes. It is more common to install a radar module and an infrared perception module in the handle, and with the help of an optical vision solution, the position of the hand is transmitted to the host or head-mounted display through the handle. Whether it is inside-out or outside-in, the capture of the position of the hand is completed through an optical solution. Another solution is wearable devices, such as the HTC Vive tracker, which uses the technical path of lidar and IMU inertial sensors to install the motion tracker on the user and use lidar to complete position capture. To avoid laser blocking caused by motion occlusion, IMU (Inertial Measurement Unit) will provide signal complement.
In order to enhance the user’s tactile experience in the virtual world, major manufacturers are also launching their own technical solutions. Last November, Meta launched a haptic glove that uses ridged inflatable plastic sheets to give users the tactile effect of grasping objects and touching surfaces of different materials. In June last year, bHaptics, a haptic feedback company, launched bHaptics TactSuit, a tactile vest, which allows users to wear the product to play supported games and experience the feeling of hitting the characters in the game. In December, bHaptics announced its first consumer-grade haptic glove, the Tactglove, which allows users to experience the sensation of a sphere rolling in the palm of their hand.
On the whole, products that support users to realize stereo sensing in the Metaverse are emerging one after another, but from a specific point of view, it will be found that such products are faced with insufficient market application, slow technological iteration, difficult technological breakthroughs, and lack of motivation to solve technical problems. and many other issues. “The iteration speed of the product is not as fast as expected”, which is probably the most accurate expression of the three-dimensional sensing products facing the metaverse.
Three Obstacles to Stereo Sensing: Content, Price, and Comfort
The vision of stereo sensing is good, but there are few products on the market and limited products for market promotion. Is this a problem of technical bottleneck or a problem of market demand?
According to statistics, in 2021, the global shipments of VR all-in-one machines will be 8 million to 10 million units, of which Oculus Quest products will account for more than 80%. The best-selling Oculus Quest2 comes with two Touch controllers. The domestic VR hardware equipment brand Pico Neo3 is also equipped with a handle. The functions of such handles include several aspects: Gesture Recognition – Detects the position of the user’s hand and reflects it in the virtual space; Haptic Control – The user completes the manipulation of the experience through the handle; Haptic feedback – conveys the user’s presence in the virtual space tactile experience in . The handle is the most common way that the current market provides users with a sense of touch in the virtual world. For gamers, adding tactile experience is obviously more three-dimensional than the sensory experience brought by pure audio-visual experience. For example, the PS5 controller launched by Sony is designed with a tactile feedback function. When players use the controller to play the game “Astros Playroom”, they can experience the characters walking in the grass, hitting objects, and ejecting under force, which are very rich and varied at different levels. vibration. This is very different from the mere “shock” tactile experience that traditional handles can provide. The Joy-Con controller for Nintendo Switch is also widely praised for its ability to provide rich, delicate and layered tactile feedback.
XR stop motion animation “Sick Rose”
However, these hardware products that have been well received by the market to enhance the user’s tactile experience in the virtual world, without exception, have one condition: complete content support. The Oculus Quest 2, which sold well last year, can support more than 4,000 games, many of which are popular music rhythm games such as “Beat Saber”. Sony and Nintendo, because of their strong content production capabilities, provide a very large space and content support for the haptic feedback function of hardware products.
“Content comes first.” In an interview with a reporter from China Electronics News, Li Jie, vice president of Xiaopai Technology, emphasized the importance of content to the development of Metaverse 3D sensing, “If there is no good content, users will not Hardware products are bought.”
Yuan Yu also believes that users buy experience rather than technology. And high-quality content is an important part of the user experience. If the content provides enough attraction for users, and the use of hardware haptic functions will improve the user experience to more heights, users will pay for it. The supporting content is not amazing enough, and the effect of the combination of content and hardware is not amazing enough. This is an important reason why the sales of tactile experience, motion capture, and facial recognition products on the market are not as good as expected.
The high price is also an important reason for the low willingness of users to buy accessories. At present, the common motion tracking devices on the market need 3 to 5 pieces to simulate human movements. One is hung around the waist and the other two are bound to the legs, so that the basic simulation of human movements can be basically realized. With two trackers in both hands, a perfect simulation of human movements can be achieved. However, the price of just 3 tracking devices is close to the price of a VR headset. This is still unacceptable for the vast majority of consumers.
VR concert
Judging from the motion capture technical solutions available on the market, visual motion capture, mechanical motion capture, acoustic motion capture, electromagnetic motion capture, and inertial navigation motion capture are some of the more common technical solutions. Among them, the IMU sensors used for inertial motion capture, infrared sensors used for optical motion capture and other accessories are not expensive per piece, and the primary reason for the high price of motion capture equipment is still the high price brought by the overall solution of the equipment. cost. “The price of each sensor is not high, but the use of end equipment is not enough for one sensor. To integrate all the sensors and make the entire hardware available, the process of technical integration requires technical reserves, which will make The cost of the whole system is not cheap enough.” Wang Congqing said.
The experience of three-dimensional sensing not only brings possible cost pressure to consumers, but also brings pressure to manufacturers to balance product comfort and improve product accuracy. Sun Kai, general manager of products and solutions of Goertek, said that the use of micro-actuators is a cutting-edge solution for conveying force feedback from the virtual world to the user’s hand, including the use of pneumatic actuators, electroactive actuators, and microfluidic actuators. and many more. According to public information, the tactile gloves released by Meta use the solution of pneumatic actuators. Meta claims there are about 15 ridged, inflatable plastic sheets on each finger, and internal sensors capture how the wearer’s fingers bend, and also give the user the sensation of a fingertip touching an object. However, the use of this kind of product not only requires the user to wear bulky gloves, which increases the product load for the user, but also has wire connections, which limit the user’s range of activities. The ultrasonic sensor proposed by ultrasonic sensor manufacturer Emerge gives users the possibility to perceive and feel the shape of objects in the virtual world without wearing additional equipment, but this product also inevitably has the problem that users need to be in a specific location to use the product. .
“90% of the technology used in “Ready Player One” is available now, but 90% of the current technology is immature.” Li Jie said, “Many product technical ideas and prototypes were launched a long time ago. For example, “Ready Player One” “The treadmills in 2016 launched prototype products. But without exception, these products have many problems in terms of comfort, safety, fit and so on.”
From a technical point of view, it is already possible to achieve haptic interaction between one user and another user in a virtual space. However, in order to realize the application of this application to a broad market group and make it a widely used and accessible technology, it still needs to be matured by the whole industry.
Yuan Yu said that the application and promotion of the Metaverse needs to be advanced gradually, and it is currently in its infancy. The so-called primary stage means that the application and exploration of the metaverse in the whole industry is still in the trial stage, and the understanding of the metaverse in the whole society is still in the conceptual stage. Judging from the domestic industrial attempts in the name of the Metaverse, the current industry’s mining of the value of the Metaverse industry is still in the testing stage, and the exploration of the metaverse’s presentation form is still relatively rudimentary. Baidu Xiyang and NetEase Yaotai have built their own metaverse platforms and hosted many events, but users still feel like visiting the exhibition hall online, and the sense of interaction with other users is still not strong enough. Compared with users who can recognize each other’s facial expressions and actions, and feel each other’s more immersive three-dimensional sensor interaction experience brought by handshake, high-five and other actions, currently, Metaverse-related industries including the Metaverse platform are participating. The focus of the authors is still to find the industrial landing direction of Metaverse and implement application scenarios. As for optimizing stereo sensing, it is a higher-level goal to be considered on this basis.
In terms of technical improvement, Li Jie believes that the accuracy of sensor adaptation is not high enough and the corresponding parameter standards are lacking. As an industry that is still in its infancy, technical standards are at the top of each company. The establishment of industry standards requires the discussion and cooperation of many enterprises, which is obviously not a problem that can be solved at this stage. From the perspective of market promotion, the Metaverse supporting hardware represented by VR head-mounted display equipment is still in the niche market stage. The VR head-mounted display market is small, and the market size of its accessories is naturally difficult to achieve huge growth.
VIVE’s vision of an office scene using VR equipment
Solving the market demand of VR equipment and “cultivating scenes” is the first problem to be solved in the process of Metaverse’s industrialization development, and it is also the first problem to be responded to in the development of Metaverse-oriented stereo sensing. What is the demand for improving the stereo sensing effect?
Judging from the Metaverse-related policies introduced by various places this year, the planning of the Metaverse in various places is to promote the integration of the Metaverse and local existing resources, and promote the development of local existing industries. For example, the “Taihu Bay Science and Technology Innovation Belt Leading District Metaverse Ecological Industry Development Plan” issued by Wuxi Binhu District mentioned that the integration and innovation of Metaverse and integrated circuits, blockchain, artificial intelligence, cloud computing and other technologies should be accelerated. The “Metaverse Industry Development Action Plan” issued by Shanghai Hongkou District proposes to focus on building the North Bund Metaverse Development and Application Demonstration Zone, cultivate and introduce a number of high-quality Metaverse scene application companies, and build a number of Metaverse industrial economic spaces.
Cooperating with current existing resources and promoting the development of existing industries is the common answer given by all localities. When it comes to subdivision scenarios, where is the growth space for stereo sensing?
Zhao Siquan, an analyst at IDC China, said in an interview with a reporter from China Electronics News that the gaming and office fields are relatively easy to highlight the advantages of facial and motion tracking technology. In her view, in these two fields, products equipped with facial tracking and motion recognition technology will easily highlight their technical superiority, so that users will finally choose the application.
Yuan Yu also believes that facial recognition technology will be the first among various stereo sensing technologies to achieve market expansion and technical optimization. In his view, image recognition has broad application space in security systems and medical fields. In contrast, the market demand for action recognition is narrower, and will only be used in specific fields such as design, games, movies, and cultural tourism. There is demand, and its demand group is more niche.
For hardware manufacturers, improving product experience and optimizing feedback solutions are the direction of their long-term efforts. In an interview with a reporter from China Electronics News, Sun Kai said: “High precision, high reliability and low latency are the direction of continuous development of products, expanding new applications in the direction of mechanical vibration and tactile sensations, and organically integrating virtual and reality. Together, it can provide end users with a more realistic and rich tactile experience.” Under the condition that the product form remains unchanged, optimizing existing products and improving tactile accuracy is itself a way to optimize users’ three-dimensional perception of the virtual world.
Optimizing the stereo sensing performance that products can provide is also the direction of many manufacturers. On the one hand, various manufacturers are exploring the possibility of realizing a more three-dimensional and immersive virtual world experience. On the other hand, there are also manufacturers trying to find another way. According to Wang Congqing, various types of sensors may have shortcomings, or the sense of wearing is strong, which affects the immersive experience; or the sensitivity and feedback force are different from the user’s own feelings. Therefore, the brain-computer interface may become an enhanced user. A higher-level sensing form for immersion in the virtual world, but Wang Congqing also said that this is a technical imagination farther than perfecting the existing stereo sensing products in the current market, and its technical perfection will take at least ten years. There must be a better product form before.
Shaking hands with friends in the virtual world, an action that can easily be achieved in the real world, has a long way to go before most people can perceive three-dimensionally in the metaverse. Li Jie believes that from now on, it will take at least 5-10 years for the stereo sensing technology to mature. Yuan Yu said that if you are optimistic, it will take about 10 years to realize the large-scale tactile interaction of users in the virtual world.
GIPHY App Key not set. Please check settings