How many steps does it take to get a bite of cake without moving your hands?
First, control the robotic arm holding the fork with your mind, and aim at the cake and fork it down.
Then control the robotic arm holding the knife to cut off a piece.
Finally, move the robotic arm while rotating, lift the cake to your mouth, and you’re done.
The study from Johns Hopkins University used a brain-computer interface to allow a paralyzed volunteer for 30 years to operate two robotic arms simultaneously.
Using a knife and fork is not difficult for ordinary people, but for a brain-computer interface system, the two robotic arms have a total of 34 degrees of freedom to be manipulated.
Including shoulder, elbow, wrist joint movement, palm movement and steering, and finger control.
And this volunteer is the world’s first bilateral implanter, which means that electrode arrays are implanted in the left and right hemispheres of the brain.
Decoding such complex neural signals is still too difficult for current technology.
This time, the research team proposed a new method to reduce the number of degrees of freedom that need to be manipulated from 34 to 12, and most of them are handed over to the algorithm, and humans only need to control no more than 4 at the same time.
Finally, in the cake eating experiment, the participants could fine-tune the robotic arm to control the size of the cake cut according to their own preferences, realizing the fine operation of hand coordination.
A recently released experimental report shows more details of the research.
Think of humans and machines as a holistic computing
The scientific name of the new method is Shared Control Strategy.
To put it simply, the humans participating in the experiment and the two robotic arms are regarded as a human-machine combination within the shared control system, which is uniformly scheduled by the algorithm.
In this way, the system only needs to consider the 3D position and 3D orientation of the end effector (that is, the hand) of the two robotic arms in the final operation execution link.
Calculated, it is 3+3 multiplied by 2 and two hands, a total of 12 degrees of freedom.
Under this system, other degrees of freedom such as joint motion and finger pinch are handed over to the inverse kinematics algorithm (Inverse Kinematics) to solve.
Further, these 12 degrees of freedom are dynamically partitioned during task execution.
For each specific task, the system will divide the steps to be executed, and guide human participants to control up to 3 of them at the same time through voice, and most of the work is handed over to the algorithm to complete.
Participating in the experiment was a 49-year-old volunteer who had been paralyzed by a spinal cord injury for more than 30 years and had limited mobility in the shoulder, elbow and wrist joints.
In this experiment, he needed to actually try to move his arm, not imagine himself moving it.
The resulting neural signals are easier to decode, which is why the video shows the occasional movement of his motor-limited wrist.
Finally, the average decoding accuracy of the left and right hands under the shared control strategy is 63.5% and 67.6%, respectively.
Out of a total of 20 test missions, he successfully performed 17.
The research team believes that the use of shared control strategies to reduce the degree of freedom of manipulation is critical to the development of brain-computer interfaces.
Its greatest significance lies in reducing the energy that people need to invest, reducing the difficulty of decoding the brain-computer interface, and allowing cheaper solutions with fewer implanted electrodes or lower precision to benefit.
One More Thing
In the research report, there is another interesting detail.
The original 20 test tasks did not include eating cake, but the cake was so fragrant, so the volunteers took the initiative to try it.
In the initial stage of the new method, it is not easy to perform such a complex task, and the video demonstration you have seen is 5 times faster.
The reality is that it takes a total of 90 seconds before and after a bite of the cake.
Some netizens said that this is suitable for eating hot pot, and the hot food is just cold when it is delivered to the mouth.
GIPHY App Key not set. Please check settings