From tinkering on their own cameras in the pre-VR days of 3D cinema to creating VR experiences like award-winningInside the Box of Kurios and Lebron James: Striving for Greatness, creative team Félix Lajeunesse and Paul Raphaël of Felix & Paul Studios have set a high water mark for the cinematic experience of virtual reality. Their latest project, Nomads, now out on Gear VR and coming soon to Oculus in 6k, is a three part series following nomadic tribes of Maasai in Kenya, sea gypsies in Borneo, and the yak herders in Mongolia—and it’s absolutely breathtaking. No Film School sat down with Paul to talk about everything from his philosophy of composition in VR, to the fragile sense of presence, to taking on fiction in his next VR projects.
NFS: People characterize VR as a medium where there’s no cinematographer, because their are no shots, preset. What are your philosophies and strategies when composing for VR?
Paul Raphaël: I think it’s wrong to say that there’s no cinematography in VR. It’s just very different from traditional cinematography where you’re thinking inside of a frame and you’ve basically got this planar, two-dimensional image in front of you. When you’re framing in VR, you still have composition. You’re positioning the camera in a very specific place. The way we tend to go about composing for our VR [at Felix & Paul] is to think of the camera as a person. Ultimately, someone is going to put a headset on and they’re gonna see this shot as if they were there. So, by really thinking of the camera as a person, you tend to get a better feeling for where the camera can and should be. Can someone actually be here or not?
"By really thinking of the camera as a person, you tend to get a better feeling for where the camera can and should be."
Obviously, there can be scenarios where you want to go somewhere else with the camera placement, but so far it’s been a prominent philosophy in all of our work to think of the camera like a person. It’s also the way we describe the camera to people we’re working with. Whether it’s actors or, in the case of Nomads, real people, we tell them to think of the camera that way. In Borneo, on Nomads: Sea Gypsies we told them to think of the camera as one of the gang. Think of the camera not a tourist, but somebody who is one of your friends. The camera is not fly on the wall, or god like, but someone who is there with you. Treat it that way. We didn’t want them to turn around and start talking to the camera because that just points out the fact to the viewer that you can’t respond, and could break the sense of presence. But we told them to think of the camera as a quiet but present individual. That’s one way we frame.
The other way we do things is spending a lot of time walking around and trying to get a feel for the space, and then feeling what would be a good position for the camera. That means wandering around for hours, spending time with the people we are filming and seeing where we naturally end up sitting and observing in different scenarios. I particularly remember one shot in Nomads: Sea Gypsies. Sea Gypsies is set in Borneo in a village on the water, where they are living in houses on stilts and boats on the water. They are people who have no nationality, so they are kind of restricted to living on the water. After we were there for a little while, we were thinking, these people spend so much of their time diving and swimming, and on the water, so we need to get some shots where we are on the water. But we’re working with a camera that we’ve custom built, and it has exposed circuitry. It’s not waterproof, let alone splash proof! A drop of water would probably completely ruin the camera. But if we’re here, we have to do this; so we figured out how to do it. We literally spent an afternoon swimming around the village and spending time with the people.
" A shot needs to look good from all directions. It’s not just about putting the camera somewhere and capturing everything. "
There is composition; when you’re looking in one direction, there’s one composition that way. When you look in another direction there is composition that way. Instead of thinking of one shot, you’re thinking of multiple shots, in a way. And a shot needs to look good from all directions. It’s not just about putting the camera somewhere and capturing everything. It’s about choosing a very specific, deliberate way to position the camera—that’s something that really needs to be explored.” At this point it’s more of an instinctive thing; it’s hard to put words on what makes a good composition. It’s a combination of feeling natural, like you could be there and it being beautiful in traditional photography principals, because your field of view ends up becoming the frame at any given moment.
"There is composition; when you’re looking in one direction, there’s one composition that way. When you look in another direction there is composition that way. "
NFS: You mention this customized camera with exposed circuity. Are you at liberty to talk about what the heck it is that you use?
Paul: We’ve been building cameras since we started making VR. In fact, before VR we were doing a lot of 3D cinema. We were making what I now call “pre-VR” installations. You lock down the variables of the viewing angle that people had in relation to the screen, and the way we shot our films was very much like creating a limited field of view VR. We were using TVs and projection screens. We were building cameras at the time as well. What we’re doing now is a lot more sophisticated. We’ve got a team of engineers and developers that work with us. But we built our first VR camera in 2013, and we’re now onto the third generation of that camera going on to the fourth. It’s a constantly iterating process where we build a camera, we use it, and after we shoot we go back to the shot and modify the camera. It isn’t a rig of GoPros or anything, it’s separate parts of sensors, and customized circuit boards, and customized case. We’ve optimized this thing for VR capture, with as little compromise as possible. I can’t go into too much detail as to how it’s designed—it’s a bit of a trade secret! We spend a lot of time on the technology side of the filmmaking process.
NFS: With the Nomads series, you guys have captured and transported us to meet types of people who may be rapidly disappearing from our world. Do you see VR as playing an important role in documentary as a form?
Paul: To me, virtual reality is really about bringing you somewhere. "Presence" is talked a lot about in VR, but presence is not something that’s automatic. It’s not like, the second you are in a VR experience, you feel present. It’s actually something that’s very fragile and needs to be cultivated. My philosophy is that presence is more important than almost anything else in VR. One thing that I think documentaries often do—and that’s not true of all documentaries, but when talking about it in terms of cinema they can be very didactic. There’s a lot of information that is passed on, whether it’s through voice over, or text on screen. The whole structure is that there is an emotional journey, it’s not dry information. But there is usually a hefty amount of information being conveyed. I wouldn’t say the VR is successful to date at transmitting large amounts of information.
"A lot of [VR] is just actually being in the presence of someone, and not so much about the specs or the statistics or didactic information of the subject you are observing. "
But the nature of VR is best suited for, in my perspective, is really to make you feel a space. It’s a more visceral experience. It’s something that cinema has never fully been able to do. The best films really bring you somewhere, whether that’s through a feeling or an insight. With VR, there’s an unspoken feeling that doesn’t need to be described. That’s a type of communication that I’m fascinated by. I enjoy exploring the question, what was it in the past we could not communicate with films that we can now communicate using VR? A lot of that is just actually being in the presence of someone, and not so much about the specs or the statistics or didactic information of the subject you are observing. That’s why the Nomads series is very experiential. They’ll be released as part of an application that will have a bit more information, but just a bit so you have context going in to where you are going and who these people are. Ultimately, the information we want to transmit is what is it like to be with these people, or to be these people? Understanding that is the motivation behind why people travel, why people go out and live their lives—to have these experiences. That’s what most interesting to me about VR right now.
"I enjoy exploring the question, what was it in the past we could not communicate with films that we can now communicate using VR? "
NFS: You mention how presence is this fragile thing. In Nomads, you fade to black, which is relatively new in VR. What were your motivations there, and how does that fit in to the sense of presence you are creating?
Paul: We’re used to having all sorts of transitions. Well, there’s not too many: either the cut, the fade, the cross dissolve. Anytime we’ve tried to do anything other than fade to black in VR, it’s been a disaster. I’ve never seen a cut work in VR. We’ve tried. When we did Lebron we tried to really up the rhythm in contrast to many of our other pieces up to that point. At one point we had two or three cuts. We felt we had cracked it, we felt we had figured out how to cut. We’re still editing in NLE and then export and watch in VR. There are plugins now where you can edit in real time VR, but at that point we had this round trip process. It was a disaster. The cuts just completely killed the presence. I’m not saying it’s impossible, if it is possible, we haven’t figured out how to do it.
Even a cross dissolve doesn’t work that well because you never experience two realities blending into each other—unless you’re on psychedelics!
The fade to black is the most natural thing we’ve found to work. It’s almost like the transition from opening or closing your eyes. It’s the most natural transition that we have in cinematic language. Even a cross dissolve doesn’t work that well because you never experience two realities blending into each other—unless you’re on psychedelics! There’s certainly a lot of things you can’t do in VR. Even moving the camera is a delicate thing. If it works, like in our Lebron piece, usually you are in a vehicle. Your eyes are on the golf cart. In Sea Gypsies, we put you in a little boat. We’re exploring concepts where we have the camera moving, basically free floating. That’s because there is something that justifies that in the story. It’s not like in traditional cinema where the camera can move as a stylistic choice. There’s a flow and feeling when the camera dolly’s in sideways or something. You don’t even think about it. Unless you are analyzing the shot, you might not even realize the camera is moving. But if a camera moves just to move without a grounding justification, that starts feeling like a dream, or an out of body experience.
I’m fully aware that this might be a syndrome of the early days in VR. As VR makers and as an audience we’re all still developing the standards and language. The fact is that we know so little and presence is so fragile, that moving the camera at this point without justification kills presence almost instantly.
NFS: What should we look for from Felix & Paul next, and where do you hope VR is heading?
Paul: We’ve done some projects with Cirque du Soleil in the past, and we’ve got a couple new projects coming out with them. We’re developing some fiction content which we haven’t announced yet. That’s what I’m most excited about right now, whether it’s from an art studio or others, is tackling fiction in VR. VR is well suited to documentary. It’s been a great learning tool—how does this medium react to reality? How will what we see in the real world translate to VR? It’s like being in a chemistry lab and isolating the different chemicals to get a sense of what happens when different things go in front of the camera. It’s been a great way to learn.
Once you start creating fictional scenarios with people pretending to be other people, and sets that have been modified and created by someone under artificial lighting…you’re ultra aware of anything "off" in VR. So it makes fiction that much more difficult. We’ve done a few pieces that were not documentaries per se, one of which was the companion piece for the movie Wild. That was the first time we’ve tackled fiction. If 2015 was the year that people realized you could make documentaries using VR, perhaps 2016 can be the year we explore fiction; that’s what we’d like to take on next.
