Although camera drones already automatically perform functions such as circling or following a subject, they don't know what sort of shots actually "look good" artistically. That could be about to change, though, as scientists are working on a system that turns drones into self-directed aerial cinematographers.
Being developed by a team at Pittsburgh's Carnegie Mellon University, the system has been trained via a technique known as deep reinforcement learning. This involved having human volunteers view computer-generated photo-realistic scenes, in which moving subjects had been "filmed" from the front, back, left and right sides, and from a variety of distances. The subjects were also placed within different parts of the overall shot.
Based on scores that the participants gave to each perspective, the system learned which ones – and which combinations of them – were most preferable. Among other things, it was found that the commonly-used follow-behind shot (in which the drone trails a subject such as a car) soon gets boring, and that while frequent changes in angle helped to keep viewers' interest, it was distracting if those angles changed too often.
Additionally, the system has been optimized to avoid occlusions that block the view of actors' faces, it can anticipate their trajectories as they move through the frame, and it can map their surroundings using LiDAR. It doesn't require prior knowledge of the scripted action that will be taking place in a scene, nor does it require GPS tags of obstacles or landmarks within the shot.
"We're putting the power of a director inside the drone," says robotics PhD student Rogerio Bonatti. "The drone positions itself to record the most important aspects in a scene. It autonomously understands the context of the scene – where obstacles are, where actors are – and it actively reasons about which viewpoints are going to make a more visually interesting scene. It also reasons about remaining safe and not crashing."
Down the road, it is hoped that the technology could also be utilized in applications such as surveillance, allowing users to concentrate on viewing video instead of remotely piloting the drone. The research is being presented next month in Macau, China, at the 2019 International Conference on Intelligent Robots and Systems.
Source: Carnegie Mellon University via EurekAlert
