Star Trek’s 1965 promise of venturing “where no man has gone before” is now closer to reality for the average earthling—virtual reality, anyway. Thanks to 26-year-old director Eliza McNitt, we are now able to travel inside a black hole at the point where it collides with another one and shreds the fabric of spacetime.
If you passed the stylish McNitt on the streets of Sundance, you might guess that she’s one of the leads in a buzzy festival film, but speak with her for five minutes and you learn that it’s not celebrity that excites her. In fact, she shines with enthusiasm when trying to distill the complexity of gravitational waves into layman’s terms.
“Virtual reality is about building worlds and bringing that into film enriches your storytelling.”
Her celestial vision, and the success of her earlier space-based VR installation Fistful of Stars (SXSW 2017), captured the attention of Darren Aronofsky’s production company, Protozoa. Their collaboration ignited an ambitious new project, which ultimately became SPHERES: Songs of Spacetime. The first 13-minute segment of the three-part project premiered in Sundance 2018’s New Frontier section and made history, becoming the first ever VR project to ink a seven-figure at the festival.
The SPHERES experience—much more sophisticated and fully realized than some other popular VR efforts—is inspired by actual recordings of sounds in space, translated into a transformative soundtrack by a team including Stranger Things composers Michael Stein and Kyle Dixon. That soundtrack, along with narration by Jessica Chastain, is your companion on a full-body interactive journey into one of the most powerful forces in the universe.
No Film School spoke with McNitt during Sundance about how the ambitious project came together, how Aronofsky changed her perception of VR storytelling, what excites her about the future of storytelling, and more.
NFS: What was the inspiration behind this project?
McNitt: The experience was inspired by my discovery that space is not silent. It's actually full of sounds. And for thousands of years we've looked to the cosmos as a way to understand our place amongst the stars. But, for the first time we listen to its music. And, this breakthrough discovery of gravitational waves has completely transformed the way that we see science because, for the first time we listen.
NFS: I know it’s hard to describe in a nutshell but what are gravitational waves?
McNitt: Gravitational waves are a ripple in the fabric of space time. So, one billion years ago two black holes collided, and they sent out this gravitational wave which has been passing through space over billions of years, stretching the fabric of space and time. And, for the first time we heard that signal here on Earth. So that discovery is what has really transformed science because we now understand that, instead of just looking out through the eyes of telescopes, we can now listen. And that expands our imagination so much deeper.
And, so for example we can now really understand, we can now have a much deeper understanding of black holes. Because, no human has ever been inside of a black hole—
NFS: Until now.
McNitt: Until now, that's right. So, I was really inspired to create a journey through the sounds of the cosmos.
Inside the black hole, event horizon from 'SPHERES: Songs of Spacetime', still courtesy of Intel
NFS: So then how did the project come together?
McNitt: My first VR experience was Fistful of Stars, which is a VR exploration of the cosmos alongside the Hubble telescope. And, because, of that experience, I was approached by Protozoa.
I met with Protozoa before mother! had been released and I told Ari [Handel] and Darren [Aronofsky] and Dylan [Golden] that I was creating this series, SPHERES, which was a journey to the deepest corners of the universe in order to look back home and to transform people's perspectives about how to protect our fragile Earth.
Darren said, “I’m also making a movie about Mother Earth,” and I at first was confused because the only thing that I had seen was the poster of Jennifer Lawrence holding her heart out. But when I saw the film I suddenly realized that we were exploring very similar themes and ideas. And Darren and Ari both come from deep scientific backgrounds. So, it was really exciting to find this incredible group of mentors who deeply believe in the fusion of science and art.
In the inception of the project I teamed up with Intel and then Oculus came on board. And Kaleidoscope which is basically a totally great resource for up and coming filmmakers because they go out and help you find money for your project.
So, I was teamed up with them, and then I saw Notes on Blindness and I just thought that it was one of the most incredible pieces in VR that I'd ever seen and the use of sound was riveting, I had to work with that team. And, that's how I met Arnaud Colinart who's the producer of Spheres. And, we brought on Nova Lab who are the people who actually created Notes on Blindness.
The lead VR technical director that I work with, his name is Clément Chériot and I'm 26, and he's 27 and that is super young which is really exciting because I think we get a fresh new perspective to science. And then Jess Engel from Crimes of Curiosity produced Fistful of Stars so I always wanted her to be a part of this project. And, she came on and really helped to make things get started and go in production. We have a really big team of people.
“Darren Aronofsky pushed me to think about the hero’s journey in a very traditional film sense.”
NFS: So just like with traditional films, building a team is a critical part of the journey.
McNitt: Yes. I think what is so unique about this project is that we have teams from different worlds and being able to fuse these worlds together I think it has helped to create a very unique experience.
When I first met with Darren he asked me what the hero’s journey was and I quickly responded saying, “but this is VR, there is no hero’s journey.” But, then he pushed me to really think about that idea of the hero’s journey in a very traditional film sense. And, suddenly I realized that you are the hero and this is your journey.
All of the language and character development, and your transformation through the experience applies but, to you as the audience member. You really become the protagonist. Having his mentorship has totally awakened a whole new approach for how I have been developing this experience in VR.
“The key piece of advice for the visual landscape was to make it weird.”
NFS: In broad strokes, how was the project technically made?
McNitt: So, the experience is developed in Unity, a real-time game engine. So, you can move around the space with 360 degrees of freedom. It's very challenging to make this experience because we were visualizing the invisible universe. There are no photographs of the interior of a black hole. No one has ever seen inside of a black hole, they're only theories. So, we turned to scientists to advise us on how to create these visuals.
Ari Handel, our executive producer comes from a neuroscientist background so he was always pushing us to dig deeper into the poetry of science. And he actually brought us to an event at Pioneer Works where Janna Levin, who's a black hole specialist, and Rainer Weiss who won the Nobel in Physics for the discovery of gravitational waves, were doing a panel.
I actually got to meet Rai and I basically grilled him for like 45 minutes about all these questions that I had about black holes. This is a man who just won the Nobel Prize and he said, "You can talk to me in six months." That's how busy he was, so having this 45 minutes with him was so incredible. I got to really ask him questions like “What color is a star when it's getting sucked into a black hole?”
There's this process called spaghettification, where a star when it's falling inside a black hole becomes ripped into pieces and stretched and pulled apart. And I wanted to really make you feel that. So, that was definitely one of the most challenging pieces to visualize.
In general, when I did meet with a scientist, the key piece of advice for the visual landscape was to make it weird. They said, “make it strange.” For them it was less about the accuracy of the visuals and more about the feeling of strangeness.
The black hole merge in 'SPHERES: Songs of Spacetime', stills courtesy of Intel
NFS: There's no school for VR yet. Did you have to learn the tools yourself or did you mainly rely on your VR partners.
McNitt: On this project, I worked with developers and engineers who are experts in the technology. I get to be extremely creative and really focus on the feeling and the experience that I want to create. And, then the technical team in France builds everything in Unity.
NFS: So how would you describe the director’s role on a VR project?
McNitt: Everything we created is CGI, and so my role was working with my team to really craft the visual language, create a cohesive tone, and be the glue that holds this experience together.
As far as process, it's really no different than making a film. You work with people in the same ways. And you have the same kind of personalities and instead of working with actors, for example, you're working with animators and it's a very similar process of collaboration.
Especially with this piece, it's extremely iterative so, the way it works is I have an Oculus Rift, I bring it in my suitcase with me and travel all over the world with it. And they will send a build to download and I will put on the headset, look around, watch it 100 times, and then write them notes.
And, then we get on Skype and have a three-hour long conversation about how to make gravitational lensing look realistic.
NFS: Audio is the heart of this experience. How did the space sounds and the composed soundtrack all come together?
McNitt: The soundscape was created by Craig Henighan who's a master of sound. He's done Black Swan, Requiem for a Dream, Mother, Stranger Things, so being able to work with him was such an honor.
When we first began, I sent him a folder of space sounds that inspired me. Those came from actual recordings from the electromagnetic waves. I sent him all of those as the initial inspiration, as if it was a mood board but it was a sound board for us. From there he would create his own interpretations of those sounds. So, our mix really incorporates the sounds of space but, with Craig's incredible artistic vision as well.
I think what I found so magical about this is that, black holes are the most powerful and violent forces in the universe, and yet the collision of two black holes creates the most fragile and delicate song. I found that so fascinating. So we wanted to kind of give you that feeling of where we build up to the moment of release when you get to experience a gravitational wave.
Inside the black hole in 'SPHERES: Songs of Spacetime', stills courtesy of Intel
NFS: I’d like to hear about the experience here at Sundance of standing at your installation all day, physically interacting with people who are watching your work.
McNitt: It's pretty incredible, because when you have a film, you go into a theater with a group of people and you watch it all together and then you leave. But with virtual reality, you put a headset on every viewer, and watch each of them have a completely different experience. It is totally mind blowing and also really terrifying to then have to go and take the headset off of them and have to experience their raw reaction to the piece.
People have experienced such a range of emotions, which I really enjoy. From people being like, “whoa this is the trippiest thing I have ever seen”, to sometimes, people just need a minute to just take a drink of water and sit down. But, I think what I love about that is how visceral the experience is for everyone.
My favorite part of Sundance has been like, walking down the street and just overhearing people talking about the project. And, it's been completely insane. I think Sundance has this special energy and I'm so honored to be a part of it. I don't think there's anywhere in the world that offers this kind of experience or catalyst for filmmakers. And every night, we go out and dance.
NFS: Finally, you started out as a filmmaker and you've worked in various media. What excites you about the landscape right now or in the near future?
McNitt: I'm really excited with the way that technology is pushing forward storytelling and how it's impacting the kinds of stories that we're telling.
SPHERES doesn't have a label. It's not a documentary or a video game or a traditional narrative. It's an interactive experience that combines all of those elements to craft an immersive, interactive experience. And, I don't know what the label is for that. I don't know if it's a category that exists yet. I think that's what's so exciting is that we are really on the precipice of developing this new form of storytelling. That embraces all of the genres to create something that's completely unique. And, I think also as a filmmaker, doing VR has completely transformed the way I think about storytelling.
Because, virtual reality is about building worlds and, bringing that into film enriches your storytelling.
For more, see our ongoing list of coverage of the 2018 Sundance Film Festival.
Featured image: 'SPHERES: Songs of Spacetime', still courtesy of Intel.
