Transitioning into VR Filmmaking with Sundance Short Film 'SONAR'

No Film School sat down with Philipp and Dominik at the 2016 Sundance Film Festival where their classic sci-fi horror-inspired film is playing in the New Frontiers Virtual Reality program. They were kind enough to give us a behind-the-scenes look at the making of the virtual reality short film, which you can download for free from their site to play in your Gear VR, DK2, or Google Cardboard. Dive in!

NFS: How did you get the idea for your virtual reality film Sonar?

Philipp: We started out with a concept for a traditional cinematic film. Then we got our hands on an Oculus DK1 in 2014. We just played around with it and noticed that no one had done what we wanted to do in VR. So we took the concept we already had, all the storyboarding and the setting, and transferred that to a 360 environment.

Dominik: We originally wanted to make a traditional movie for movie screens. We had all our layouts, we had all our shots planned and prepared and -- along came Oculus Rift.

Credit: Oculus DK1

We had all our layouts, we had all our shots planned and prepared and…along came Oculus Rift.

NFS: So you had the script and the storyboards for a traditional narrative. What needed to change to translate to VR?

Dominik: At first we thought we could just take what we had and modulate it into this VR thing. But actually, there was more to it. We realized that there was another kind of potential in virtual reality that allows you another kind of storytelling. Since the audience always wears this headset and they are literally inside the movie at every point, not only psychologically but physically, the body starts to feel spaces that only exist in this virtual world. So you can put forward your story in ways that traditional movies don't allow on the screen.

We realized that there was another kind of potential in virtual reality that allows you another kind of storytelling.

Philipp: It's a cool concept of self-awareness of the viewer. Is he the actor, is he the user, is he passive, active, or is he a character in the movie? What kind of presence does he have? We thought about that a lot because we love cinematography actually, and in VR there's no framing anymore. It's all POV and then we thought, well if it's all POV, it's probably very boring and we don't want to create a rollercoaster ride. That's why we immediately started off the movie with a free flying camera to avoid giving the viewer any sense of his own body. That way the viewer already knows it’s going to be a passive experience. "I am nobody, I am a ghostly camera," basically. It’s the early days of VR and we couldn't do a lot of quick movement, so we kept the pace very slow.

Dominik: We tried to avoid that feeling, when people are experiencing it, that they are missing a body. If we had put a body in there, and it was animated, people look down and see a movement that isn't their own. We wanted to prevent that kind of distraction from the immersion. That's why we tried, from the beginning, to establish this feeling that there is no physical body attached to you.

We put our camera in our shop and very roughly put together a cube with six sides. We had six cameras in the beginning, capturing all the sides and then put this together and put it in the Rift to get the impression of the space around us.

NFS: Knowing that your previous films weren't virtual reality, how difficult was it to make the transition and pull this film off?

Dominik: It took us a long time to decide that we’d go for 360. We gave ourselves one week to do very basic, rudimentary research, whether we could pull it off technically, since it requires a lot of resources to represent the whole 360 degree of space at all times -- not just one angle that you would have in an ordinary film. We decided we could manage it if we restricted ourselves in certain areas. And then we just went for it.

Philipp: We are traditional animation students and filmmakers. The pipeline is completely different for VR. We thought about using a real-time engine, but we had never worked in games before. So we tried that, but it didn't work out. We thought, okay, we want to make it as mobile as possible so people can put it on their phones. We used After Effects, Premiere for editing, and simple 3D programs. Then we just rendered it out -- so no scripting, no programming.

Dominik: The first problem we had was how to get the feedback from the screen onto the Rift so we could get an impression of how it would look. We put our camera in our shop and very roughly put together a cube with six sides. We had six cameras in the beginning, capturing all the sides and then put this together and put it in the Rift to get the impression of the space around us. That's the way we went thought the whole project just trying to take the workload that we had used from the traditional movie making and try to modify it to what we needed.

Credit: SONAR

We started out with six cameras pointing in every direction in a 3D program. Render that out, put it in Premiere, and then output it again and again, so in the end you'll have one video file to put in another player for the Rift.

NFS: Can you breakdown that workflow from script to headset?

Dominik: From a script perspective there wasn't much you can write down that would make any sense. We immediately went for the animatic and we wanted to iterate on it as quickly as possible. But that's the first technical problem we had. We started out with six cameras pointing in every direction in a 3D program. Render that out, put it in Premiere, and then output it again and again, so in the end you'll have one video file to put in another player for the Rift.

Then you have a preview of your edit or your shot, but you have to go through the 3D program, through After Effects, through Premiere, to get one iteration. We had planned to do a three-minute film, because we were a small team and didn’t think we could do more than three minutes. But we noticed in the editing that we had to slow things down. People who had never seen VR were looking all over the place, and they didn't get it. So we had to stretch it out into longer shots and slower editing. We ended up with six minutes.

You definitely have to enjoy that pioneering sort of character.

NFS: So you had people try it out throughout the process for feedback?

Dominik: Since we've never done anything in this medium before, we weren't quite sure if our perception was objective in any way. So we wanted to get as many people as we could to give us their feedback. Since we had never worked in the 360 space, everything to us was just amazing. Of course, after a certain amount of time, this excitement faded away. So, feedback from the outside was just precious.

There's a lot out there at the moment that’s just using the medium for its novelty. But the projects that really stick out, that make it special, establish virtual reality as a medium and as a powerful way of storytelling.

NFS: What’s your advice for traditional filmmakers who are interested in getting into virtual reality?

Dominik: First of all, I'd say you have to be adventurous and you have to be willing to experiment. I think there's no guarantee that what you are trying to do is going to come out 100% the way you think it's going to. You definitely have to enjoy that pioneering sort of character.

Philipp: If someone wants to do it they should just start doing it, but think about traditional storytelling also because it still works. You have your arcs, and you have your climax and story points, so you have to structure your VR experience similarly, just not in the same way as before.

Dominik: There's so much going on right now with YouTube having 360 and other services coming up, so it's really getting easy to just use it if you want to get started. You can just do a little clip and post it on YouTube just to get a feeling for what it's like to work in a 360 environment.

Philipp: If you want to do anything bigger, though, there still has to be the idea.

Dominik: There's a lot out there at the moment that’s just using the medium for its novelty, but the projects that really stick out, that make it special, establish virtual reality as a medium and as a powerful way of storytelling.

Thank you Philipp and Dominik!

Even if you’re not at the Sundance Film Festival, you can check out Philipp and Dominick’s work on SONAR by downloading the 3D (and even the 2D) version free from their official site.

For more, see our complete coverage of the 2016 Sundance Film Festival.

No Film School's video and editorial coverage of the 2016 Sundance Film Festival is sponsored by Blackmagic Design.  

Source: SONAR