This article is part four in a series aimed at bringing knowledge to filmmakers around the world about working with virtual production toolsets. If you missed the first three, I highly recommend you check them out starting with the basics before jumping into this one, as they will give you the fundamental knowledge you need to understand how VP is changing things in the film industry.
Get the Movie Made
If you are a producer in narrative film or TV then you likely operate with three questions in mind whenever you are producing a new project.
- How can I best tell the story?
- How can I make sure it comes in on or under budget?
- How can I get as many people to watch it as possible?
With that framework in mind, you approach new scripts or concepts with a plethora of ideas and strategies balancing distribution methods, release timing, hiring the best and most talented team, making sure everyone gets paid well and works safely, and ensuring the actual movie or series gets made in the end. Within all of these competing factors, where does virtual and volumetric production play a role?
A New World
When approaching a new project and breaking down its costs, a huge part of any conversation revolves around location. Where does the story take place, and how can we convince the audience we are there? Sometimes it’s simple: you actually go there. But most of the time, because of how ambitious the story is, you may:
- Build that location in a studio.
- Use VFX to replace aspects of the location you shoot to make it look like the location it is supposed to be.
- Use the tried-and-true method of just shooting somewhere similar enough, change a few street signs, and hope the audience doesn’t notice (they almost never do).
Now, as anyone who has ever spent a day working locations or production management will tell you… locations are never the most enjoyable part of a job. Permits, parking, transport, logistics, internet, power, water, etc. When shooting on location, you are erecting a tiny village momentarily to roll the camera a few times and then packing up to do it again tomorrow.
I could go on, but I think you get the picture. It is within this conversation of locations that the advancements being made in volumetric LED stages really begin to prove their value. Shoot downtown Manhattan in a studio while only building the bare minimum of what is on the street and letting the screens do the rest.
But how do you decide if virtual production is best for your project?
'The Martian'Credit: 20th Century Fox
Where Volume Stages Play a Role
Volumetric stages play a role when you are divvying up your script and deciding which locations would be the easiest and which would be the most difficult to find and work with. A great time to consider the use of virtual production technology is if your story has a location that otherwise would be incredibly difficult to get, maybe because it requires many flights, hotels, rental cars, and per diems, or perhaps a spaceship to get to. This is traditionally where green/blue screen and VFX have stepped in to help out.
Can’t actually go to Mars? No problem, just make some red rocks on a sound stage and you can finish it later in post-production.
This method has a lot of inherent flaws and limitations that lead to reshoots and just generally lackluster scenes as performers do their very best to pretend it is actually Mars inside of a green-washed sound stage.
So how is it different? Let's walk through every stage of production and see.
Pre-Production
The differences begin the moment you decide to shoot in a volume rather than composite the background and sets in post. This is because all the work it takes to get an environment looking "final pixel" ready must now be done before you head to set. Which comes with a mentality shift. The "fix it in post" mindset doesn’t work in this world. What you get in return for all the work done upfront is a lot more freedom to plan, explore, and imagine what else can be done within that location without needing to reshoot. The entire environment will be right there with you on stage, and you won't need to wonder what can be accomplished later within the budget and timeframe you have.
Here are some things to pay attention to in pre-production.
Credit: Luka Cyprian
VFX Producer or Supervisor
I’m sure this was already a given when you thought about a movie going to Mars but, just in case, I want to emphasize that having the right leader on the computer-generated-visuals side of things is very important. Hiring someone who has worked with this technology before and knows how to avoid its pitfalls is crucial to have a successful shoot.
Scouting
Using virtual production toolsets means a lot of your scouting will be done virtually—which luckily just so happens to be very pandemic-friendly.
The downside is that some important members of the creative team who have never done Unreal-driven group calls or VR scouting before might need some tech onboarding to understand how to scout virtual sets and interact with them properly. Imagine having to teach some of your team how to drive the morning of a scout. It’s good to leave some room to learn before their creative decisions depend on it.
Creative Approvals
While it’s not uncommon that everything from casting to wardrobe to sets involves a lot of creative sign-off, now the process of show-and-tell in pre-production will involve rendering virtual environments and the virtual sets in those environments. This includes a number of things including, but not limited to:
- Making sure the DP signs off on the color space of the environment
- Making sure the DP and director sign off on the type of sun/sky lighting used in the virtual location
- Making sure the director and PD sign off on the virtual sets being made and how they will extend or accentuate what exists on the physical sets
- Ensuring that all of these conversations involve the VFX supervisor or their appointed team member
Testing
This is a huge part of working within volumetric stages that goes underappreciated until it’s too late. Test everything you canbefore you shoot, or as much as possible.
Negotiate with your studio provider to allow for testing time to be included in your rate, or at least budget accordingly to avoid any stress around your director, DP, PD, or anyone else asking to see the environments and virtual set pieces on the screens before they shoot. Testing and tuning are the two biggest factors to succeed on the day, as a lot of what you are doing is still experimental. It is recommended that each affected department has one test per day starting with cinematography and VFX to make sure the assets work and render properly.
Okay, you’ve made it through pre-production, and now you are lying in bed the day before going to camera, wondering what "first-day syndrome" might bring to your set tomorrow. What will be forgotten, and what might you need to solve to make the day?
Production
Every Setup Is the First Setup
Part of doing something new and exciting means that a lot of people have never done it before. When crewing, try to find people that have experience and can help train others. But it’s certain that some people working on set will be adapting to the slightly changed demands of their job with this new technology.
The makeup might look different in a volume, the gaffer might have trouble wrapping their head around what a lightcard is in Unreal.
It’s okay. Everyone will learn and grow, and soon enough the production will be running smoothly.
Some Shots Might Not Work
Watch for shots that chew up a lot of time in their set-up. Some shots might not look right or might even downright not work at all in a volume.
Ensure that the on-set VFX supervisor is always a part of those conversations. When it comes time, there may be a shot or two you have to throw up a green screen for and finish later to keep the day moving.
This isn’t the end of the world, but anticipate it and budget for it when you are planning out the costs for the production.
Work with the 1st AD to Schedule with the Volume in Mind
With any luck, you are pleasantly surprised by how it is possible to go from your desert location to the top of a mountain within moments on stage, and how some setups that would normally take hours to create are happening in minutes.
Once you have seen the flow of your set and stage techs working with the environment, make sure the shooting schedule is adjusted accordingly to play into its strengths. We are used to scheduling where unit-moving takes the longest, and now it is almost the quickest part of the process.
Credit: Luka Cyprian
Post-Production
So long as everything went according to plan in pre-production, this is where things get a lot easier. What might have been a 200- to 1,000-shot VFX list could be as low as 0. I would still expect some things to have slipped through the cracks, and perhaps you have a few dozen shots to touch up, but far less than would’ve been the case in a more traditional production.
Mentally, a prepared producer would still prepare for a few potential issues that could pop up.
- Moire. Some of the shots may have moire that wasn’t noticeable on set. This one is tough. In the rush of production, a tiny bit of moire induced by the rear LEDs could be present in the image that then has to be removed later in post-production. It happens.
- Color shift between panels. With really complex camera moves, it’s possible for there to be a viewing-angle color shift between the LED panels, causing more time spent in the color grade to time this away to match.
Should You Use It?
Honestly, virtual production toolsets are sweeping across the industry and growing in popularity by the day. Their ability to bring certain aspects like scouting, pre-vis, design, and even "final pixel" production into a "real-time" time frame, whereas they previously took hours, days, or even weeks to do, makes the sales pitch on its own.
There is a learning curve to getting involved with it, but like anything, it comes with time and persistence to tell stories in the best way possible. If you are interested in learning more and engaging with people around the world who are working to make virtual production a reality, you can find a few links below to communities that I’ve gained a lot of value from joining. You will find local teams working with these toolsets who will be more than happy to chat with you about any projects you are producing and about where virtual production can fit in.
Also, if you have any questions about the knowledge I’ve shared here or just want to chat VP, reach out to me on Instagram @karljanisse, or on LinkedIn as Karl Janisse.
Additional Resources
Check out these communities on Facebook for additional information on working in this space: Unreal Engine: Virtual Production and Virtual Production.
On Reddit, check out UnrealVirtualProd.
Special thanks to the Director’s Guild of Canada for permission to use the photos from our amazing shoot in February 2021 in this article . Thanks to our shoot sponsors: ShowMax for providing the Volumetric Stage, Animism for providing the environments and VFX support, and William F. Whites for providing the Camera/Lighting and Grip gear.
Karl Janisse is a Canadian cinematographer, photographer, and visual storyteller currently residing in Vancouver, BC, who specializes in virtual production cinematography and workflow design. He believes great images make the heart bigger and the world smaller. Karl also co-founded an online school Pocket Film School that helps bring education about the film industry to people around the world.
