How VR is rewriting the rules of storytelling

Over the last couple of years advances in VR technology have shown the potential to revolutionize a broad variety of sectors, from enhancing exercise routines to military drone piloting to even helping people overcome a fear of public speaking. In the sphere of classical storytelling though, the uptake has been much more cautious. It's easy to immediately understand how new VR interfaces could change the way we make music or design buildings but is VR a threat – or even a valid competitor – to film or television? As a medium can VR even function as a storytelling device, or does the inherent immersive interactivity of the medium make it fundamentally unsuitable for those ends?

We recently caught up with two of the most progressive minds in field of VR storytelling at the Melbourne International Film Festival symposium on virtual reality. Here's what Eric Darnell, co-founder of Hollywood VR studio Baobab, and Google's principle VR filmmaker Jessica Brillhart, had to say about the challenges involved in taking traditional storytelling into this nascent medium.

Eric Darnell is probably best known as the Dreamworks filmmaker behind Antz and the Madagascar films, but in early 2015 he became a true believer in the new religion of VR after his colleague Maureen Fan placed him into a headset. Darnell and Fan went on to found Baobab Studios, which is dedicated to making VR animated work with a view to carving a new space producing content for entertainment spheres. Its first film is called Invasion (available now for free on the Oculus and Gear VR) and it's a lovely, simple short film featuring a cute bunny and some inept alien invaders.

Jessica Brillhart's creative interests have blended film and technology for years. Initially she was part of Google's Creative Lab in 2009, before moving on to become Google's Principle VR Filmmaker in 2015. Much of Brillhart's recent work has been digging deep into the theory of how we can construct story in VR environments.

Both Darnell and Brillhart are clearly excited by the prospects this young medium holds, but there is a tangible cloud hanging over VR. No one knows right now exactly how we can tell stories in a VR environment. The medium is so drastically different to other approaches that it's almost a wild west scenario, with producers trying out different things hoping to find a new way to satisfy that human storytelling need.

What is Story?

In order to understand how we could tell stories in the medium of VR, first we need to fundamentally deconstruct the notion of storytelling. We need to get to the root of what story is. "Story to me is a result of an experience," Brillhart explains. "We filter out our experiences in order to better communicate that to someone else. So I make a film about something that's happened to me or something that I believe has happened to someone and I represent that to you."

This idea that story comes after experience, and VR being a primarily experiential medium, seems to be one of the elementary aspects that set it apart from other media like film, writing or theatre. "VR, I believe, is fundamentally about the potential of story," Brillhart elaborates. "So really as a VR creator, your job is to put into place the things that will allow the greatest potential for the story to happen. The story that you sort of want to happen but you also admit to yourself that it may not be the exact thing."

The temptation in being too directive when constructing a VR story is one that many VR creators are currently grappling with. Eric Darnell, coming from a classical filmmaking background, approaches VR through a more traditional cinematic lens, "At the foundation of of everything we do at Baobab Studios is really story-driven stuff. We're not putting VR first, we're coming up with what we hope is a great story that we can tell well and then using VR to help us tell that story better."

His current strategy in creating VR stories is dominated by an experimental, trial and error process. Invasion went through a comprehensive series of test runs with the piece being viewed by over 1500 people before reaching its final form. He studied audiences while they watched early iterations of the experience in order to gauge when they were focusing on what he wanted and when their attention was drifting. Production design, sound cues and character's eyelines were all techniques utilized to help guide the audience's attention. Darnell's ideal strategy is to "help the audience to compose the shot that you would have composed if you had control of the camera."

Is he worried about being too manipulative?

"Cinema is by definition manipulative so I don't feel guilty about that and the fact is that the viewer always has the free choice to look wherever they want. If they're making their own choice to look and it just happens to be exactly where I want them to look because I have inspired them to do that, then I think for them it's going to be a more fulfilling experience because it just feels natural."

Brillhart is a little more hesitant about manipulating the audience's attention towards a singular focal point. She uses an analogy to the film The Truman Show in describing the relationship between a VR creator and an audience. The audience is like Jim Carrey's character in the film, and the creator is up in the control room managing all the details of the world hoping to guide the audience into a story without them ever feeling manipulated. She describes a VR creator as someone who is "nurturing an experience" but always understanding that a story may not happen. She says the VR creator must constantly ask themselves, "What's the gradient of potential outcomes? So actually I'm telling, potentially not just one story but I'm telling many."

VR: The "dangerous" medium

In May at the Cannes film festival Steven Spielberg made headlines when he expressed concern over what the medium of VR could do to classical storytelling, "I think we're moving into a dangerous medium with virtual reality," he said. "The only reason I say it is dangerous is because it gives the viewer a lot of latitude not to take direction from the storytellers but make their own choices of where to look."

Darnell sees Spielberg's comments as a positive reflection on how this new medium can come in and "stir things up". The fact that VR as a new toolkit for storytellers simply means we still have a lot to learn, but as a former feature filmmaker Darnell feels like he isn't particularly giving anything up in regards to narrative control through his shift into VR.

The unspoken implication in Spielberg's comments is that VR as a medium is Roland Barthes' concept of "The Death of the Author" writ large. The broad latitude given to the audience in the medium may limit, or even eliminate, any intent or control a storyteller could have over the interpretation and meaning garnered from a given experience. How can a VR creator maintain any sense of authorial control over an experience?

In response to the suggestion that VR could kill the author Brillhart is adamant in her disagreement. "We're not dead, we're actually more powerful than we were before. We build worlds. I lost a frame but I gained a world." Brillhart speaks inspiringly about the beauty in VR creators' ability to craft an entire universe, but creating a detailed universe is still a different proposition from the wonderful way a well-told story can convey ideas and concepts.

Who are you in a VR story?

So how does a VR creator go about building a story where the audience is experientially embodied as a character with free choice in the fictional world?

Darnell resists the urge with VR storytelling to mirror the world of gaming, which he sees as a medium that fundamentally makes the audience the main character with a singular goal of "winning". The subjective focus and centralization of the audience in video games is something that Darnell feels separates the interactive nature of gaming from the more interpretative qualities inherent to the way we engage with stories.

He recalls Tolkien's Lord of the Rings and the character of Samwise Gamgee as a way to envision how VR storytelling could work. In Tolkien's classic series, the main narrative is ostensibly driven by Frodo and his mission, yet the character of Sam is pivotal throughout the story. Darnell sees Sam as a cipher for the audience in a VR narrative, "Make [the viewer] not the main character, let the main character continue to drive the story. And then you come along as second. There are key moments where you participate and help and ensure this other character gets what they want. It's not about being the hero and what am I going to do next, but it's about having enough empathy for the other character to do something compassionate."

Darnell notes that Tolkien even concluded his great trilogy on a scene following the character of Sam returning to his family. The implication being that Sam was truly the main character in the story. "That to me is a clue on how we might be able to have powerful stories that are really tightly constructed and still have the viewer feel like they are part of it," Darnell concludes with excitement.

Brillhart on the other hand sees the power of a VR storyteller through a much more untraditional prism. She stresses that much can be learnt from the mistakes that game developers have made and already resolved. Modern video games understand that players need to be eased into experiences with early levels offering more subdued exploration. This can give us a template for how we structure VR experiences. "You get brought to a place and it's generally pretty simple and gives you space to explore it ... That amount of exploration in the beginning of experiences is crucial," she explains.

But any similarities between a video game and a VR experience end when we begin to consider how a creator can generate a singular narrative in a way that doesn't resemble a hammy choose-your-own-adventure story.

Brillhart resists using the term "narrative", saying the word is too limiting when discussing the creation of worlds in VR. To explain, she introduces a hypothetical VR story which follows an excited young girl visiting the fair grounds. The young girl wins a fish at a stall during the experience and if that is an important story detail that the audience needs to take in then the VR creator fills the 360 degree frame with things than mean fish, "... or wet, or games. Something that gives a sense of the activity. You may not be able to see the literal interpretation but you might be able to see the echoes of that against other things that reinforce what happened. So it's about understanding what you want to get out of each scene, which is the same as cinema has always done."

The Story Aura

The vision evoked by Brillhart of how VR can operate within a story space is excitingly innovative. Instead of trying to direct the audience to follow a linear narrative throughline by essentially yelling, "Look over here!", the creator loosens up and accepts that the audience may choose to concentrate on other secondary details.

We begin to understand how experience is the real precursor to story and Brillhart then starts with the question: "What is the story that someone will tell someone else about the thing you have done? And then you go backwards and say, who does the person need to be in the space in order to achieve that?"

It's a highly relatable concept. How do you recount your experiences to friends? What details do you concentrate on?

It's only then that Brillhart finally gets to the story, "And you think, what is this really about? What's the possible gradient of this? This is the particular story but what are the other stories?" Here Brillhart reaches a breakthrough, explaining that once we understand all the probable stories that could emerge from within our created world we can generate what she coins "a story aura".

"It's less about that girl who goes and gets a goldfish at the fairground but it's more about learning about life and death … And if someone gets a sense that this is about someone innocent learning about the meaning of something when it's gone and to appreciate something when it's alive, then that's it! The aura is getting to the truth of the story".

This concept drills down into the root of what storytelling really is. A good story is fundamentally more than simply recounting a series of events – it's about something. And through understanding what a story is really about, a VR creator can then build out world details that affirm those abstract ideas and concepts.

The idea Brillhart raises completely underlines her statement about the almost omnipotent power storytellers could ultimately wield in a VR space. The creator can control every aspect of a world within a 360 degree space and thus author every world detail that the audience then builds their own meaning and story from. It's a powerful realization where the creator and the audience work together to generate a fractal of potential stories but with a singular directed intent.

Still, maybe the idea of an omnipotent author has always been shaky ground no matter what the medium. The story an audience takes away from a specific work will invariably depend on their subjective interaction with the text. While VR certainly offers a degree of flexibility in how an audience interprets a story, it's also a medium that offers the stark reminder that the audience is, and has always been, the one in control.

Brillhart concludes with a nail-on-head sense of finality, "That's what makes VR so great! There's no wrong answer [for the viewer], unless you [the creator] do it wrong. Ideally you're never setting up the visitor for failure. You're setting them up to have an agency to tell stories that are the best ones for them and being okay with that as a creator. As long as the truth is outed, that's all you can hope for."