Adobe has teased a couple of stunning new image and video manipulation tools that will further erode your trust in what you see. ProjectCloak can magically delete objects out of moving video, and SceneStitch uses AI to take the content-aware fill to a new, sometimes surreal level.
The content-aware fill tool is still one of the most intelligent, useful and downright impressive bits of tech in Adobe's incredible Photoshop software. First seen in the 2010 release of CS5, it allows you to select objects in a photo and remove them, allowing the software to take a guess at what should replace them. A lot of the time, it works so well that removing an annoying signpost or ex-girlfriend from an otherwise great photo is a one-step operation.
And now it seems the tech is about to jump to a whole new level. Two new sneak peeks from last week's Adobe MAX 2017 conference show where these tools are headed in the near future, and frankly we want them right now.
#ProjectCloak: Select an item in a moving video and make it disappear
ProjectCloak takes the idea of a content-aware fill into the world of video, and it's mind-boggling. Select and track an object or person in a piece of moving footage, then "cloak" it, and the software makes it disappear. Check out the video below:
ProjectCloak seems to work even better than the content-aware fill tool – that's because it's got more information to work with. In moving footage, it can get a peek at what's behind an object, which lets it build a faithful reconstruction of the background instead of having to take an educated guess. It's an absolutely stunning tool that just about any videographer will want immediately.
#SceneStitch: Bringing AI and millions of stock images into play
One of the current content-aware fill tool's main weaknesses is that when you choose something to remove from your image, all it's got is the rest of the image to try to fill the background in with. So you can end up with obvious doubling of background elements, particularly if you're trying to remove something big.
Enter SceneStitch, which brings artificial intelligence and the giant Adobe Stock Library to bear on the problem. SceneStitch goes hunting through millions of other images to find possible elements to replace the unwanted area with.
It comes back with a bunch of different options you can pick through. Some of them work more or less just like the content aware fill tool, but smarter. Others come up with completely different suggestions that can totally change the context of the shot - as the video below demonstrates:
Both these tools can be expected to ship with upcoming versions of the Adobe Creative Suite.
It's interesting; for nearly 100 years, television and video has more or less been a trustworthy record of the truth. Video manipulation has been available for decades, but as a costly visual effects exercise. Tools like ProjectCloak will give the average videographer an unprecedented ability to alter the visual record, pushing us even further into a "fake news" world where fewer people than ever trust what they see. These are indeed interesting times.
Source: Adobe MAX 2017
