Computer vision system recognizes 3D objects via heat diffusion
When we see a hand, regardless of whether it's open, in a fist, or pointing a finger, we still recognize it as a hand. If a computer has only been taught to recognize an open hand, however, it will probably have no idea what a fisted hand is. Getting computer vision systems to interpret images more like people do - to realize that a fist is a hand, for instance - has been one of the aims of artificial intelligence researchers for some time now. Things in that field may be about to take a step forward, however, as scientists from Indiana's Purdue University have just announced two new methods of three-dimensional object recognition, both based around heat diffusion.
The two methods, heat mapping and heat distribution, could be used together or separately. They do not measure heat, however, but instead scan an object's surface, then apply algorithms to estimate how heat would be diffused across it. Because heat diffusion patterns are predictable, the underlying shape of the object can be determined, regardless of its orientation.
The algorithms are based on equations first developed by Albert Einstein and 18th century physicist Jean Baptiste Joseph Fourier.
In the heat-mapping method, the computer divides the surface of an object up into a meshwork of triangles, then applies algorithms to determine how the heat would flow from triangle to triangle. Even if the image contains "noise," such as laser scanning imperfections, the object's shape can still be determined. The computer is also able to determine the object's "heat mean signature," which allows it to divide the object up into segments, identifying the center of each one, and assigning more or less importance to different segments.
"Being able to assign a weight to segments is critical because certain points are more important than others in terms of understanding a shape," said Professor of Mechanical Engineering, Karthik Ramani. "The tip of the nose is more important than other points on the nose, for example, to properly perceive the shape of the nose or face, and the tips of the fingers are more important than many other points for perceiving a hand."
In the heat distribution method, the object is once again broken down into a pattern of triangles. Through the application of algorithms, the computer is able to build a model of the whole object in the form of a histogram, which is a two-dimensional mapping of a three-dimensional shape. While a 3D map would provide more information, a 2D map is much easier to process.
"It's very efficient and very compact because you're just using a two-dimensional histogram," Ramani said. "Heat propagation in a mesh happens very fast because the mathematics of matrix computations can be done very quickly and well."
The Purdue team have so far used the two methods to quickly identify complex shapes including hands, the human form, and a model of the mythical half-human, half-horse centaur.