Researchers at Carnegie Mellon University have developed "LiveLight," a machine learning algorithm that can automatically scan through a video, understand what's happening and cut out the repetitive and boring parts. And it can do this without human supervision, saving you plenty of uneventful viewing time. This technology could be especially useful for reviewing security camera feeds or as a help in creating compelling video highlights.
We're moving toward a world in which recording video is getting easier and easier – whether from your phone, your glasses, or even your wristwatch. Recording a long video is one thing, but who has the time to watch it all back? And if you start skipping some parts to save time, how can you make sure you don't miss any of the really important stuff? Luckily, computer scientists at Carnegie Mellon are coming to the rescue.
Prof. Eric P. Xing and PhD student Bin Zhao have developed LiveLight, a machine learning algorithm that's able to decode and evaluate what's happening in a video, look for the most interesting bits and ignore the repetitive or uneventful parts. The software then automatically creates a small video trailer that lets viewers get the gist of the entire video.
Over a single pass of the video, and almost in real time, the software analyzes the content of the video using an artificial intelligence algorithm, catalogs it, and then looks at the summary it has built up to that point to decide whether it's looking at something new or a similar pattern to what it has seen previously.
Using a standard laptop, the process might take one or two hours to process a one-hour video; but the researchers say you could get that time down to minutes if you have the required processing power.
LiveLight produces summaries autonomously, but a human editor is free to modify them as he or she pleases, by choosing to include some of what the algorithm classified as a "boring" part to provide context for one of the highlights, for example. The software also gives the human editor a ranked list of the salient features of the video to make the task of editing more time-efficient.
Xing and Zhao say that the LiveLight technology could ultimately be used to review and tag raw videos taken from, say, GoPro or Google Glass and upload thumbnail trailers to social media instead of the whole thing, saving money in Internet data charges and time for both editors and viewers.
Another interesting application would be in detecting unusual or suspicious behaviors caught by surveillance cameras. Startup PanOptus is now working on exactly this proposition, offering automated highlights through the LiveLight technology.
In the video below, you can see an example of the software at work.
Source: Carnegie Mellon University