Computers

Google makes objects in videos searchable

Google makes objects in videos searchable
Google’s latest AI advancement expands its existing image recognition AI into the realm of video
Google’s latest AI advancement expands its existing image recognition AI into the realm of video
View 3 Images
Google’s latest AI advancement expands its existing image recognition AI into the realm of video
1/3
Google’s latest AI advancement expands its existing image recognition AI into the realm of video
An example of the way the system isolates a single shot in a video and creates weighted tags for each identified element
2/3
An example of the way the system isolates a single shot in a video and creates weighted tags for each identified element
These labels encompass the entirety of the video with percentage weightings of content for more accurate search returns
3/3
These labels encompass the entirety of the video with percentage weightings of content for more accurate search returns
View gallery - 3 images

Have you ever been struck with a mind blank and forgotten the title of a crazy video you wanted to show someone? You know that video… the one with the guy running on the beach with the dog. Google's latest advancement expands its existing image recognition AI into the realm of video, promising a future where we can easily search through video content that is automatically assigned tags by an AI system.

Google Cloud Video Intelligence uses deep-learning models, based on its existing Tensorflow frameworks, to analyze videos and tag them depending on the specific content they contain. The system can search the videos adding numerous noun and verb tags, as well as evaluating and weighing the significance of the tagged elements.

An example of the way the system isolates a single shot in a video and creates weighted tags for each identified element
An example of the way the system isolates a single shot in a video and creates weighted tags for each identified element

The system analyzes the videos shot by shot, weighing each element with a percentage value. This value then determines how prominent it will feature on search results with the ultimate outcome being a searchable database of videos that can be navigated the same way we search web pages or images.

In a demonstration of the technology, Google showed how the system can point users to specific moments in a video containing the objects or events they are searching for. Even more impressively, their demonstration showed the system identifying the specific breed of a dog featured in the video being analyzed.

These labels encompass the entirety of the video with percentage weightings of content for more accurate search returns
These labels encompass the entirety of the video with percentage weightings of content for more accurate search returns

The tool is now available for companies in a private beta mode. Google's chief scientist in machine learning Fei Fei Li writes, "This API is for large media organizations and consumer technology companies, who want to build their media catalogs or find easy ways to manage crowd-sourced content."

It may be some time before the system is more broadly integrated into our general day-to-day search tools, but it promises an extraordinary new level of detail in the content one will be able to hone in on, especially in a future that looks to be dominated with more and more online video content.

Source: Google Cloud Platform Blog

View gallery - 3 images
2 comments
2 comments
Nairda
"This API is for large media organizations and consumer technology companies,"..code for: Too computationally intensive for the average user computer at this point in time.
sk8dad
There seems to be great potential in law enforcement when applied to surveillance footage with both positive and negative ethical implications. On the one hand, a search for all individuals between 2:30pm and 2:50pm who arrived carrying a backpack then left without it during the Boston Marathon would be useful to say the least. On the other haand, there is the search for all males of Middle-Eastern features in order round them up and deport them.