As soon as you enter The Under, Oculus's first immersive-theater-meets-VR experience, you're acutely aware of that fact that you're inhabiting a different dimension. In the absurdist world, which I was able to experience, in part, as a 15-minute segment during a private demo at Sundance, players explore a magical abandoned ship and are promptly whisked below deck to a cabaret theater. Here is where things get interesting: while performing live on-stage acts, the actors engage you, asking you to perform simple tasks or participate in the performance itself. You can also interact nonverbally with other players, who appear as avatars in the audience. As such, The Under is the first VR experience to combine elements of immersive theater—namely, live actors and interactivity—with remote-access settings. That means that while you are "in" the experience, the actors could be anywhere else in the world. With the snap of a finger, they can teleport you to other environments within The Under.
To develop this project, Oculus teamed up with Tender Claws, a multimedia studio that specializes in narrative experiences, and Piehole, an experimental theater collective that trained the actors to improv within the virtual environment. While the particulars of the narrative weren't immediately accessible to me in the demo phase, Oculus has said that the experience "revolves around themes such as fate, time, and free will, as users interact with a never-ending loop of revolving on-stage acts, both live and recorded, including line-dancing cats, dandelion goddesses, and a dolphin named Gerald."
At Sundance, I sat down with the Yelena Rachitsky (executive producer of the project), Danny Cannizzaro (developer), Samantha Gorman (developer), and Tara Ahmadinejad (theater director) to discuss the complex mechanics of bringing this ambitious interactive experience to life. The Under will be released this year as a paid experience for Oculus Quest, the company's first all-in-one gaming system in VR, retailing this spring for $399. The exact release date and price are yet to be announced.
"We created a new platform for actors to be able to act in VR when they're not right next to you."
No Film School: Would you each mind introducing yourselves and letting me know about your role in the project?
Yelena Rachitsky: I'm Yelena, the executive producer on the piece. So I help support the project from the Oculus side.
Cannizzaro: I'm Danny Gonzaro, one of the co-founders of Tender Claws, the production studio that made this.
Gorman: I'm Samantha, one of the co-founders of Tender Claws as well.
Ahmadinejad: I'm Tara and I'm a theater director from Pie Hole.
NFS: What's new and exciting about The Under Presents? Why bring it to Sundance to show it off?
Rachitsky: This is one of the first projects that we're showing on Quest. Sundance has always been the place that we brought projects that push the boundaries of what storytelling and interactivity can be. We're especially excited about The Under Presents because it's something that hasn't been done before—incorporating live actors into a storytelling experience, where you're not limited by having it to be location-based. So usually, we have to go a location and interact with actors that are motion-capped. Here, because actors are also in Quest, they can be anywhere. And that allows you to be able to experience it in your living room, which is pretty incredible.
NFS: And how did you initially make the connection to bringing an immersive theater company to the experience? What were those first conversations like?
Ahmadinejad: We were really interested in the edge of liveness and pre-recorded media in an immersive environment. We chose Pie Hole because we feel that they are the people who are uniquely equipped to really understand what operating in theater and immersive media mean, based under years of experience.
Gorman: We are not specifically an immersive theater company. We do a lot of different kinds of work, but throughout our history as a company, we've often been bringing artists from different disciplines and doing a lot of inter-media work in our performance stuff. This was our first time actually bringing theater skills to another medium. But we have similar sensibilities—just the way we think about form and content and storytelling through whatever medium you're working with, and tailoring it to that.
Cannizzaro: Something we noticed in our work in virtual reality is that a lot of our writers who were doing the best work were from coming from a playwriting or theater background. So we wanted to work more closely in theater in our next project.
"Creating constraints in communication opens up more opportunities for emergent play and discovery."
NFS: What kind of conversations would you have with the actors in terms of how you wanted them to interact with the subjects and the space? And about their physicality?
Rachitsky: Well, a big thing about the whole process with the actors has been about just entering into the VR world as a player, and seeing what grabs your attention, and how the actor can exist in that space in a way that isn't competing with the discovery process for the player. Because we want to encourage their curiosity. So that's been the big balancing act: How do I get this person's attention to convey what I need to convey and create this feeling of presence and livelihood? We wanted to make [the experience] feel like it's live, but also be flexible enough to allow players to wander off and do other things. So the actor really had to be super responsive to whatever was happening with the player and flexible enough not to get stuck on one track—to be able to go down different paths.
Ahmadinejad: We see it as a living, breathing thing. This is an early run, so we're actually experimenting with a lot of these things live now and learning as we go through this process. It's iterative.
Rachitsky: It's like previews in theater, where you actually see how the audience responds and tweak it along the way. Have you ever seen Then She Fell [the immersive theater experience in New York] or Sleep No More?
NFS: Yes! I love immersive theater.
Rachitsky: So we're calling this Sleep No More meets Journey [the video game]. You know how Sleep No More has these narratives on rails, but you can also have it as a personal, special experience?
NFS: Yes! It wasn't immediately clear to me that the one-on-ones I was having during The Under were unique, so that's amazing to know.
Ahmadinejad: There's a large cast of different characters. You may or may not experience interactions with certain ones.
Gorman: To bring it back to Journey, actually, the biggest inspiration for us was actually the moment that we were playing the game and we realized the other characters in the game weren't AI characters, but real people also playing the game.
Credit: Oculus
Cannizzaro: Even with super limited means of communication between players, you could still tell. And it's interesting that what made you be able to [distinguish] was the characters' movements. How do the [avatar's] constraints force you, as a player or as a guest, to be more creative in how you can communicate with the actors or with other players?
Rachitsky: Tender Claws has developed an interface for the actors to be able to use different tools to make different objects appear, move players, and do different kinds of magic tricks, basically, to help them be more responsive to the players and have a little bit more structure with what they're doing. So it's a constraint because they have to improvise and be able to do all of the acting in addition to doing all the technical stuff. They're doing their own cues and everything. But the idea is that once the actors get more comfortable with the interface, that actually makes them have more ability to control what's happening. So there's this learning curve, and then once they do that, then they can actually have more power.
"From a design perspective, the best VR interfaces are very spatialized—when you're reaching out and touching things."
NFS: When you were initially developing that technology, what were some challenges that presented themselves and how did you navigate around them in the beginning?
Cannizzaro: I think one of the core things we realized is that, from a design perspective, the best VR interfaces are very spatialized—when you're reaching out and touching things. That's what VR does really well: spatialize the visual. But we couldn't do that, because if you saw the actor trying to select a control tool, you would also see the character reaching out and pressing these invisible buttons. So we had to go against VR convention for that and create an interface that could be done while the character is still being expressive with their hands.
Rachitsky: What's incredible is that they actually created a new platform for actors to be able to act, in VR, from a place that's not next to you. It's kind of a stepping stone of allowing this to happen more often, which is really special.
Ahmadinejad: We had rehearsals where I was directing from New York and there were actors both from New York and LA in headsets. Tender Claws was on headsets. We were just hanging out in space.
NFS: That brings us to the interactivity element. In the experience, the actors often called out requests to the audience. It felt very rewarding when you did something and were given praise for it.
Cannizzaro: One thing we realized in rehearsals is that for crowd work, it's hard, because all the characters look almost identical right now, other than slight height differences. So it is hard for the stage performers to keep track of people, and for stage work, you need to be able to do that.
Ahmadinejad: Yeah. Making you feel like you are really seen as a player. And if you feel like you're doing a lot of stuff and they're not noticing it or acknowledging it, it feels like—
Gorman: —Why am I here?
Ahmadinejad: It really feels sad. So you really need to give people that feedback to say, "I see you, I know what you're doing," and make them feel seen for a day.
Rachitsky: That's why people go to immersive theater shows. They're looking for that special experience.
Ahmadinejad: And Cabaret, and drag shows, and any kind of live thing where you feel like you're being looked at.
'The Under'Credit: Oculus
Rachitsky: For a project we did at Oculus last year, Wheels in the Walls, the team worked with [the theater company] Third Rail, actually informing the character Lucy to be able to do gestures that naturally feel intuitive. But it wasn't live. So she did little things like tricks that immersive theater directors do—like turning your body, so they know that specific turn is gonna make you want to turn as well—but there's nothing like when your response actually creates a reaction. This is one of the first times that we've been able to see someone do that without the actor having to be live in that space.
NFS: In the experience as it currently stands, you can't talk. You can do pretty basic gestures. You're just a mask and hands. Why can't the audience laugh and talk? What if there was a real back-and-forth with the actors?
Rachitsky: For us, this was a conscious decision for several reasons. We were inspired by this idea that actually creating constraints in communication opens up more opportunities for emergent play and discovery of things that you can do outside your voice. We are working with the idea of how you be expressive when certain elements are taken out. How do you engage with somebody else in presence in that space? And what is gestural agency?
Cannizzaro: We recently implemented a "snap" ability for each player, which I'm sure will become a whole specific type of language that players use to communicate. We're also interested in adding in the idea of players holding hands with one another, or you'd grab towards someone and offer a hand and they could grab and accept it.
NFS: That was very liberating to me. Knowing that I couldn't talk really did force me to go around and explore more and pick up objects and wave them around and throw them to get attention, you know? Like a child.
Ahmadinejad: I think it's key. So what you saw in the experience was the outer world. There's more of a plot-driven experience happening that's interactive and it has live moments, but the bulk of it is actually pre-recorded. There is this real focus not just on the liveness, but on the interplay between liveness and pre-recorded.
The nice thing about that constraint is that you spend time as a witness, almost like a ghost. You're inhabiting these scenes, but you don't feel seen. And then when the moment happens where you are seen, then it feels like a real disruption of the whole experience. So it really is about the interplay, and those constraints help us [enhance] those moments. Like coming out from under the invisibility cloak.
Rachitsky: There's always that balance between interactivity and story. It's really hard to make a story interactive and still guide it—creating that constraint of the narrative that allows it to play out so you don't get super distracted from the story.
For more, see our ongoing list of coverage of the 2019 Sundance Film Festival.
No Film School's podcast and editorial coverage of the 2019 Sundance Film Festival is sponsored by RODE Microphones and Blackmagic Design.
