AI & Humanoids

First "visually dubbed" feature film hits US theaters this May

First "visually dubbed" feature film hits US theaters this May
Watch the Skies follows Denise's search for her father, who she believes was abducted by aliens
Watch the Skies follows Denise's search for her father, who she believes was abducted by aliens
View 2 Images
Watch the Skies follows Denise's search for her father, who she believes was abducted by aliens
1/2
Watch the Skies follows Denise's search for her father, who she believes was abducted by aliens
Flawless' TrueSync works entirely in post-production, and doesn't require reshoots or special equipment
2/2
Flawless' TrueSync works entirely in post-production, and doesn't require reshoots or special equipment

Swedish sci-fi adventure flick Watch the Skies is using a technique called 'visual dubbing' to alter the cast's lip movements and map them to dubbed English dialogue for international audiences – without reshooting a single scene.

That's thanks to Los Angeles-based movie-making AI firm Flawless, and its TrueSync technology that analyzes the entirety of an actor’s performance, utilizes deep learning to create a volumetric 3D representation of their face, and then adapts their mouth movements in the footage in post-production.

TrueSync has actually been around for a few years, and was notably used to remove F-bombs from the PG-13 edit of the 2022 British survival horror movie Fall (which happened to have been directed by Flawless CEO Scott Mann).

Fall (2022 Movie) Official Trailer - Grace Caroline Currey, Virginia Gardner

Check out the trailer above: you'd swear too if you found yourself at the top of a giant radio tower without a way down. TrueSync helped replace 35 cusses with less offensive phrases like 'fricking,' without the need to reshoot a single scene. Here's what that looks and sounds like.

Flawless use of TrueSync on the movie "Fall"

This time around, Flawless' tech is being used to visually dub an entire movie. Watch the Skies, which debuted in its home country back in 2022 as UFO Sweden, follows a teenager searching for her missing (and possibly abducted by aliens) father with a UFO watchers' club. The original actors dubbed their own performances into English at a recording studio, so their Swedish accents are intact.

WATCH THE SKIES - World's first AI visual dubbed feature film

"We believe that the audience will experience it as an English-language film without noticing the technical transformation," said Albin Pettersson from the film's production collective Crazy Pictures. "The result is much better than we ever dared to hope for."

Flawless' TrueSync works entirely in post-production, and doesn't require reshoots or special equipment
Flawless' TrueSync works entirely in post-production, and doesn't require reshoots or special equipment

The tech also uses a rights management system to ensure AI-powered editing is carried out only with the artists' consent. TrueSync is also said to help save up to 50% in costs of redoing scenes for dubbing, compared to reshooting them.

If you're in the US, you'll get a chance to see for yourself just how realistic it is over the course of an entire movie. Watch the Skies plays at silver screens across the country starting May 9.

Source: Crazy Pictures

4 comments
4 comments
Trylon
When this is common, I'll miss the good, old-fashioned Godzilla dubs, with their mismatched lip movements and odd pauses. Even the ones where the distributor had the dubbing actors perform with fake Japanese accents. And I don't care what the dubbing haters say, the original audio is not usually superior. The inflections and emphasis of the original languages don't always carry over. I find actors in Japanese movies to be overly melodramatic, although the dubbing in many cases is only marginally less so.
YourAmazonOrder
"I didn't say that!" "It looks, to me, like you said it - here's the video of you saying it." "But I didn't say that! I said the exact opposite of that!" "Then how do you explain the video?"
Such technology, used In the Court of Public Opinion, where one is guilty for simply having been accused, would end not just careers, but lives and a civilization. This is how totalitarianism gets its final foothold. The weakness of any civilization is Public Opinion, married to ideology.
Deres
As a non native english speaker, I must also say that many american films are difficult to understand because of the non perfect live sound. There is a lot of ambiant sounds at the same level as actors lines plus some reverberation. Actors do no have personal microphone to permit good balance. Worst in many modern movies, sometimes people are talking at the same time ...
Deres
I think this will also be a good tool to simplify dubbing. Dubbing can be difficult or altering the original meaning because in different languages words or sentences can have totally different number of syllabs.
It will also permit to escape some reshooting just to change some dialogues. Or when there is no reshoot, the horrible add-on of a narrator or some dialogue taken from the back or at long distance to hide the script modifications.