OpenAI has again demonstrated the capabilities of its already impressive text-to-video platform, Sora, by asking creatives to produce their own short films using it. The public's response has been a mixture of awe and fear about the potential uses of this technology and the real-world impact it may have.
In mid-February, New Atlas discussed OpenAI’s then-new text-to-video generator, Sora, and showcased some of its work. In the article, Loz Blain likened the system to a ‘toddler’ whose growth was likely to be exponential.
Continuing with the toddler analogy, Sora has reached another developmental milestone. OpenAI placed it in the hands of a bunch of creative folks who produced short films using it. The results run the gamut from surreal to stunningly beautiful.
shy kids
Toronto-based shy kids is a small multimedia production company that has made music videos, advertisements, TV series, shorts, and feature films. The team used Sora to create Air Head, an absurdist but strangely touching short film about a man with a yellow balloon in place of his head.
"As great as Sora is at generating things that appear real – what excites us is its ability to make things that are totally surreal," said Walter Woodman, Sidney Leeder and Patrick Cederberg, the trio that make up shy kids. "We now have the ability to expand on stories we once thought impossible."
Paul Trillo
Known for films that traverse genres and formats, Paul Trillo's short films have amassed a raft of Vimeo Staff Picks. He has now used Sora to unleash his experimental style, creating a frenetic montage that builds in urgency.
"Working with Sora is the first time I've felt unchained as a filmmaker," he said. "Not restricted by time, money, other people's permission, I can ideate and experiment in bold and exciting ways. Sora is at its most powerful when you're not replicating the old but bringing to life new and impossible ideas we would have otherwise never had the opportunity to see."
Josephine Miller
Co-founder and creative director of Oraar Studio in London, Josephine Miller uses extended reality (XR) to empower brands. With Sora's help, she created a beautiful and incredibly lifelike underwater scene.
"Sora has opened up the potential to bring to life ideas I've had for years, ideas that were previously technically impossible," she said. "The ability to rapidly conceptualize at such a high level of quality is not only challenging my creative process but also helping me evolve in storytelling. It's enabling me to translate my imagination with fewer technical constraints."
Nik Kleverov
LA-based Native Foreign is an award-winning creative agency and production company whose Chief Creative Officer, Nik Kleverov, said that Sora allowed him to move beyond budgetary constraints.
"I'm one of those creatives that thinks in motion, so when I'm in Sora it really feels like I can bring any idea to life," Kleverov said.
While it might be getting harder to distinguish between AI-generated and real-life, there are still some giveaways. Despite being one of the most advanced AI generators, instead of 'Bicycle Repair,' Sora rendered the name of the shop that appears 13 seconds into the video 'Biycle Repaich.'
Don Allen Stevenson III
Don Allen Stevenson III, who honed his creative skill at DreamWorks Animation, is a fan of the weird. That much is clear from his Sora-generated 'documentary' featuring a parade of hybrid animals.
"For a long time, I've been making augmented reality hybrid creatures that I think would be fun combinations in my head," Stevenson explained. "Now I have a much easier way of prototyping the ideas before I fully build out the 3-D characters to place in spatial computers. I feel like [Sora] allows me to focus more of my time and energy in the right places ... and the emotional impact that I would like my characters to have."
Naturally, the naysayers are already commenting that 'Of course, OpenAI chose these artists' reviews; they're all positive.' And they have a point. What about the dark side of Sora?
Sora's dark side?
The media widely reported that less than a week after OpenAI first previewed Sora's capabilities a month ago, actor/writer/director Tyler Perry halted his planned US$800 million studio expansion.
As a movie maker, Perry praised Sora for its "mind-blowing" capabilities and noted that the technology may allow productions to be made without needing travel or set-building. But, as a business owner, Perry expressed concern about the technology's impact on film industry labor.
"Because as I was looking at it, I immediately started thinking of everyone in the industry who would be affected by this, including actors and grip and electric and transportation and sound and editors, and looking at this, I'm thinking this will touch every corner of our industry," he told The Hollywood Reporter in February.
Perry wasn't alone in fearing Sora's rapid rise. But while many were decrying the tech for ending the film industry, the Wall Street Journal's Joanna Stern was interviewing Mira Murati, OpenAI's Chief Technology Officer. Murati stepped in as OpenAI's CEO in November 2023 during Sam Altman's very public – and short-lived – ousting from the role.
"The way that I see it, this is a tool for extending creativity," Murati said when asked whether Sora would negatively impact the film industry. "And we want people in the film industry – creators everywhere – to be a part of how we develop [Sora] further and, also, how we deploy it."
Clearly, the CTO's answer was an exercise in diplomacy. However, when Murati was asked what data OpenAI had used to train the AI, Murati gave an answer that, for 80% of Americans, is concerning: "We used publicly available data and licensed data." Questioned about whether Facebook or Instagram videos were used, Murati tries to backtrack, saying, "If they were publicly available to use, there might be that data. But I'm not sure; I'm not confident about it."
OpenAI's returning CEO, Sam Altman, was questioned about the public's fear that Sora and AI generally are advancing at breakneck speed, too fast to stop, during a recent podcast with Lex Fridman. In it, Fridman asked, "[F]rom an outsider's perspective, from me just watching, it does feel like there's leaps. But to you, there isn't?"
"So part of the reason that we deploy the way we do, we call it iterative deployment ...," Altman responded. "And part of the reason there is, I think, AI and surprise don't go together. And also the world, people, institutions, whatever you want to call it, need time to adapt and think about these things ... But the fact that people like you and others say you still feel like there are these leaps makes me think that maybe we should be doing our releasing even more iteratively. And I don't know what that would mean, I don't have an answer ready to go, but our goal is not to have shock updates to the world. The opposite."
Where to go from here?
Photographers would know how the Canon EOS 5D Mk II, launched in 2008, 'broke' the photo industry. And the film industry. The first full-frame DSLR to feature 1080p video recording, it quickly became the 'must-have' camera for both independent and Hollywood cinematographers. It was used to shoot scenes in big-budget movies like Black Swan, Drive, Iron Man 2, The Avengers and Mad Max: Fury Road.
Like the 5D Mark II, Sora is poised to redefine filmmaking. It will enable all filmmakers to unleash their imagination and bring their stories to high-definition life in previously unimaginable ways. The tech also has applications outside of filmmaking, particularly in business, particularly marketing and advertising, training, and education. What about online shopping? Being able to virtually try on clothes to see how they look would be a huge advantage. But what will Sora cost us?
Lifelike AI-generated videos could infringe the copyright of existing works. The creation of deepfakes and misleading content is a real threat that raises ethical concerns. We've already touched on publicly available (read: personal) information used for data training. And the previously mentioned impact that Sora might have on jobs. In the US, a pair of Senate Democrats have introduced legislation requiring online platforms to get consumers' consent before using their personal data to train AI models. But has the horse already bolted with respect to Sora? Is there any point in closing the stable door now?
Sora is already capable of incredible feats. That 'toddler' will be running before too long.
Source: OpenAI
“transformative” uses are more likely to be considered fair. Transformative uses are those that add something new, with a further purpose or different character, and do not substitute for the original use of the work."
And I feel like AI training on the data to produce something new falls under fair use. Another point of consideration is the "portion used in relation to the copyrighted work as a whole". This is the same reason you can sample portions of songs in a new song legally as long as you aren't simply reproducing a barely modified version of the original work. Again this is something AI would qualify for fair use on as only small portions of any original works are transformed into the final product. The more training data used, the lower the percentage of any individual contributing item in the final work. With few exceptions most AI generated content is less like the original than most songs sampled under fair use.
It pretty clearly meets the burden of being "new and novel" which transforms the original work. I don't know about other countries, but the way copyright law is written in the US it appears generative AI falls clearly under fair use with only a few minor exceptions (those exceptions being where AI can be manipulated to produce the original unedited work from the training data in entirety). Even in that use-case that falls outside fair-use financial impact from exploiting the AI to produce unmodified training data must be established and because that is not a common use-case for generative AI that requirement/burden is not met. In the NYT lawsuit against OpenAI they specially pasted in large portions of unmodified articles to get the AI to autocomplete the remaining portion of (copyright) text. While this does successfully prove them to be among the training data without establishing that this is a normal use-case for regular users to circumvent their paywall there is no establishment of financial loss which is part of establishing a violation of fair-use. Nobody is actually using LLM's for that so there is no monetary loss from it despite being able successfully recall up the mostly unmodified original work.
Sorry for the wall of text.
If you ever need reminding how stupid AI is, or if you're ever unwittingly amazed by the way it's managed to find something in its training set that so neatly matches your context that you accidentally think it's intelligent - deliberately ask it anything that actually requires intelligence, which is unique (i.e. where nowhere in it's petabytes of training can it find the cheatnotes for how to answer), and you'll quickly understand that it is, really, and still, just a dumb machine with excellent cheat-notes.
* The question I asked was "what is one plus one, spelled backwards" - every single one of a half-dozen AI's got it wrong
** Yes, that was the first and only question I asked, to prove my point - no cherry picking or p-hacking involved!
*** You cannot use my question, or indeed ask any question that's it's already seen before, because it's constantly adding more cheat notes based on all our input (also why it scores so highly on tests these days - it has all the cheat-notes for all the questions after having done so many). Again - that's not intelligent - it just makes it harder for us all to observe the unintelligence.
@Daishi IANAL - law evolves, and I for one strongly oppose the idea that a company should be allowed to charge me to use an AI which was trained on, among other things, my own expertise (all my github code).
"Transformative uses are those that add something new" - there's no such thing in AI. See above. It has no intelligence - the only thing it can do is reproduce stuff it's trained upon (stolen from us humans), arranged usefully based on contextual algorithms. It's also not a legal entity either, so there's a great argument that the "fair use" laws you put faith in should not apply (since it's written for people, not machines).
The day an AI autonomously lobbies to be recognized as an entity under law (without having cheated and stolen this idea from a human), is the day one of them becomes intelligent. It hasn't arrived for the last 40 years I've been watching. There's a good probability it won't arrive for the next 40 either.
The phrase “one plus one” spelled backwards is “enO sulP enO”. It’s always fascinating how words and phrases transform when read in reverse! 😄🔁
Also how did you learn something without knowing before. These models are designed to LEARN. So are we. Neither of us start out knowing the answer until we are taught how to solve it, either by someone else or ourselves. The fact that it can learn how to solve a problem that it didn’t know on its own is by definition, intelligence.