Researchers have overcome a major challenge in biomimetic robotics by developing a sensor that, assisted by AI, can slide over braille text, accurately reading it at twice human speed. The tech could be incorporated into robot hands and prosthetics, providing fingertip sensitivity comparable to humans.
Human fingertips are incredibly sensitive. They can communicate details of an object as small as about half the width of a human hair, discern subtle differences in surface textures, and apply the right amount of force to grip an egg or a 20-lb (9 kg) bag of dog food without slipping.
As cutting-edge electronic skins begin to incorporate more and more biomimetic functionalities, the need for human-like dynamic interactions like sliding becomes more essential. However, reproducing the human fingertip’s sensitivity in a robotic equivalent has proven difficult despite advances in soft robotics.
Researchers at the University of Cambridge in the UK have brought it a step closer to reality by adopting an approach that uses vision-based tactile sensors combined with AI to detect features at high resolutions and speeds.
“The softness of human fingertips is one of the reasons we’re able to grip things with the right amount of pressure,” said Parth Potdar, the study’s lead author. “For robotics, softness is a useful characteristic, but you also need lots of sensor information, and it’s tricky to have both at once, especially when dealing with flexible or deformable surfaces.”
The researchers set themselves a challenging task: to develop a robotic ‘fingertip’ sensor that can read braille by sliding along it like a human’s finger would. It’s an ideal test. The sensor needs to be highly sensitive because the dots in each representative letter are placed so closely together.
“There are existing robotic braille readers, but they only read one letter at a time, which is not how humans read,” said study co-author David Hardman. “Existing robotic braille readers work in a static way: they touch one letter pattern, read it, pull up from the surface, move over, lower onto the next letter pattern, and so on. We want something that’s more realistic and far more efficient.”
So, the researchers created a robotic sensor with a camera in its ‘fingertip’. Aware that the sensor’s sliding action results in motion blurring, the researchers used a machine-learning algorithm trained on a set of real static images that had been synthetically blurred to ‘de-blur’ the images. Once the motion blur had been removed, a computer vision model detected and classified each letter.
“This is a hard problem for roboticists as there’s a lot of image processing that needs to be done to remove motion blur, which is time- and energy-consuming,” Potdar said.
Incorporating the trained machine learning algorithm meant the robotic sensor could read braille at 315 words per minute with 87.5% accuracy, twice the speed of a human reader and about as accurate. The researchers say that’s significantly faster than previous research, and the approach can be scaled with more data and more complex model architectures to achieve better performance at even higher speeds.
“Considering that we used fake blur to train the algorithm, it was surprising how accurate it was at reading braille,” said Hardman. “We found a nice trade-off between speed and accuracy, which is also the case with human readers.”
Although the sensor was not designed to be an assistive technology, the researchers say that its ability to read braille quickly and accurately bodes well for developing robot hands or prosthetics with sensitivity comparable to human fingertips. They hope to scale up their technology to the size of a humanoid hand or skin.
“Braille reading speed is a great way to measure the dynamic performance of tactile sensing systems, so our findings could be applicable beyond braille, for applications like detecting surface textures or slippage in robotic manipulation,” said Potdar.
The study was published in the journal IEEE Robotics and Automation Letters, and the below video, produced by Cambridge University, explains how the researchers developed their braille-reading sensor.
Source: University of Cambridge