If you were an animator who was instructed to “Make a street that looks like it’s in Paris,” chances are you might not know what to do. Sure, you could occasionally put the Eiffel Tower in the background, but you couldn’t do that for every shot. If you were using a new data-mining system developed by Carnegie Mellon University and INRIA/École Normale Superiéure in Paris, however, it would show you what you should include. The software automatically looks through photos taken in various cities, and identifies the recurring visual features unique to each place.
So far, the system has analyzed over 250 million visual elements found in 40,000 Google Street View images of Paris, London, New York, Barcelona and eight other cities. As it did so, it looked for elements that occurred frequently, yet that were also specific to each city. Something like the Eiffel Tower therefore wouldn’t make the cut, since although it certainly is unique, there’s only one of it in all of Paris. Brick walls also wouldn’t be included, as there are lots of them, but they’re found all over the world.
Things that were picked up included Paris’ cast-iron balcony railings, lamp posts and distinctively-styled street signs. Other examples included New York City’s fire escapes and San Francisco’s bay windows.
Because the images are all geo-tagged, the system is able to identify what sort of places within each city specific elements are found. The iron balcony railings, for instance, are located mostly on Parisian side streets.
The system can also be instructed to be less selective, including all of the elements that occur anywhere within Europe, for instance. In this way, it could be used to explore the architectural differences between continents, or in the creation of street scenes that are intended to just look generally “European.”
The researchers now hope to expand the capabilities of the system, enabling it to identify natural features unique to areas across the entire planet.
Check out the video below for more information on the technology.
Source: Carnegie Mellon University
