A few weeks ago I had a really interesting chat with Patrick Collister, Head of Design at Google, who talked to me about the evolution of storytelling as Virtual Reality technology has progressed.
As this is primarily a writing blog, rather than a tech one, I’ll very quickly give a run-down of some key terms:
- VR – Virtual Reality. Creating digital spaces that you can walk around in. This is different to computer games because the space stays still even when you move the device you’re looking at it through. Imagine you’re standing in a room, looking at it through the camera on your smartphone. The room doesn’t swing around as the phone moves – it stays still and you see different bits of the room. Exactly like that, except the room is wholly digital.
- AR – Augmented Reality. A digital overlay on real stuff. Pokemon Go is Augmented Reality.
- MR – Mixed Reality. Still in development, currently. This is basically like AR, but projected directly onto the eyeball rather than viewed through a device.
Making the Reader a Protagonist
I want to talk about VR because that’s the stuff really making waves in storytelling. Google have been doing all manner of cool things with it, and Patrick pointed me towards a particular video on their VR YouTube channel which demonstrated some of what he was talking about.
See, if you’re standing in the virtual world and a story’s unfolding around you, how do you a) interact with it if it’s just a video, and b) ensure you’re looking at the right place to see the crucial plot points? Both these questions are solved in the same way. Google 360 structure the story in very short chapters. Each chapter is triggered only when the viewer is looking in a specific direction. So you don’t miss anything but, more importantly, nothing happens if you don’t look at it. You’ve got the time to look around because the next chapter will wait for your attention.
If a tree falls in a VR wood, and you aren’t looking at it, it doesn’t finish falling.
Suddenly the viewer is critical to the process. They become a protagonist, responsible for making things happen. By way of example, here’s the video Patrick showed me. You can watch it on computer, but watching it on your phone is a WAY better medium to experience this type of storytelling. Because the point is that you move around. Give it a go.
I’m not sure what impact this will have on traditional storytelling structures, if any, as far as the written word is concerned. But it’s early days and there’s no denying video is a very powerful tool to shape how people think. And the trend in digital content over the last few years has consistently been more and more about personalisation. You want to attract people to your creation? Make it personal – give them a starring role.
So far I’ve just been an interested observer, very much on the fringes of what’s going on. Ian Thomas, Director of Talespinners – writer, game designer and all-around storytelling expert – has waaaay more experience than I do. So I asked him what he thought.
Challenges in VR Storytelling – Ian Thomas
Here’s the thing: there are a few groups of people trying to leap on VR for storytelling purposes right now, and at least two of them are coming at it from an angle which isn’t a great fit, and a lot of their problems lie in a fundamental misunderstanding of the medium through trying to apply film techniques. VR is seen as a visual medium most closely related to computer games and film, and to my mind it’s far removed from either.
The first group are film-makers. As you might imagine, the natural inclination of the film-maker when approaching VR is to take a linear piece of storytelling and then to work out how to deliver it in 360 degree surround. Directors are used to having complete control of the action; editors are used to controlling pacing (not to mention being able to cut and have multiple viewpoints, both of which are limited in VR); cinematographers are used to being able to control framing. None of those skills are really of any use in VR, and a lot of lessons are having to be unlearned very swiftly – nearly all the language of cinematography goes out of the window. VR productions coming from this angle tend to be very static, tend to be confusing for the player, don’t take enough account of the player’s presence in the world (being more of a piece for the player to watch, or a ghost train-like experience), and, when they offer any interactivity at all, it’s of the ‘trigger object to continue’ variety.
The second group are game developers – and one of the problems comes specifically from game developers working at the high end. The trouble is that many such AAA developers have spent the last twenty years or so trying to make their games more like films, picking up cinematography techniques (such as ‘frame the important object’), cuts, cutscene pacing and so on. As with film-making, those things simply don’t work – you can’t constrain the player’s head to focus on a specific object, for example. The other issue is that locomotion in VR is completely different from that in most mainstream computer games – walking along a corridor is quite a different experience in VR (and can lead to motion sickness), so you need to find other tricks and techniques; a lot of gaming has been focused around an experience of ‘continuous travelling through a space’, so that needs to be rethought. Again, people are having to unlearn lots of lessons. A lot of early attempts have been experimental VR ports of existing games, which are only really working for the hardcore gamers who are willing to put up with quirks and nausea.
However, games are a better fit than cinema, and there are games companies doing excellent work in this space. They tend to be people who’ve thrown away their preconceptions and started from scratch and spent a lot of time experimenting and getting to grips with the medium; or even to be people who have no previous background in games and are coming in fresh, with no constraints or expectations. And, in general, games companies tend to get the idea of player agency and embodiment in a way that film-makers don’t.
The fundamental storytelling issue is – a thing happens. How do you get the player to notice? Google’s answer, as you quoted, is to only trigger things when the player is looking in that direction – there are other solutions but that’s not a bad one. However, as you might imagine, pacing is therefore quite different from other media.
But there’s a deeper thing going on here, at least in this stage in the adoption of VR. You’re trying to tell a story. Perhaps an epic tale which will capture the player and sweep them up. At least that’s the intention. But behaviourally, a lot of game creators are finding that the player spends all their time just looking around the room and picking up objects, ignoring your carefully crafted dramatic content. Because that’s where they’re finding the fascination and the fun. Maybe that’s only temporary, because the experience is so new. But in any case, perhaps that should be your storytelling method – just picking things up and looking. In the games industry this is known as environmental storytelling, and existing non-VR games such as Gone Home are great exponents of this sort of experience, allowing players to piece things together at their own pace.
What I’ve found most powerful in VR so far is the sense of presence you feel when there’s another character in the scene. Even if the character isn’t modelled photorealistically, the human brain interprets them as ‘there’ in a way that I haven’t seen in any other medium – it’s absolutely uncanny. If you play through Rocksteady’s Batman Arkham VR and are nose-to-nose with the Joker… there’s no feeling like it. It’s something which took me completely by surprise, and it’s the thing I’m most interested in pursuing.
Another important thing to mention is 3D audio. Well-designed audio is hugely important in VR, and again isn’t something that film audio can adapt to very well due to the non-linear way the sounds are encountered or triggered. It’s a lot closer to game audio, but many games still treat audio as of secondary importance. In VR it’s utterly critical, as it underpins and helps define the reality of the space around you. And, where you perhaps can’t rely on camerawork in the way you could in other media, you can absolutely rely on sound and get much more out of it than in other media.
VR experiences aren’t simply translations of existing games techniques. Nor are they simply translations of film techniques. I think the closest thing we have so far is single-audience-member participatory theatre-in-the-round, but no-one’s really drawing on theatre experience yet. But at the root of it, VR is its own thing, and no-one knows quite what yet.
Ian is a games writer, designer and coder who has wrestled computers for a living for over two decades. He’s worked in interactive television, education, puppet-making, film, publishing, live events, and the games industry, where he’s helped bring to life games such as Frictional’s SOMA, The Bunker, and a wide variety of other titles from LittleBigPlanet to LEGO. He’s written action movies, children’s books about Cthulhu, interactive fiction and pulp novels. Most of his time is spent running Talespinners, a story-for-games company that helps games studios deliver their narrative. Amongst other things, he’s currently writing for a VR multiplayer RPG.