The way of virtual set design is changing. Instead of using greenscreens and motion-capture suits to film and create completely CG worlds, there is a new way of creating using Unreal Engine that is revolutionary. Really, virtual environments are ready to be used the day of filming.
By now, we have all seen the behind-the-scenes photos of The Mandalorian and how their stage is a huge curved screen that shows the environments. These large screens create backdrops that are rendered by using photo-based textures and realistic lighting that blends the virtual world and the real world.
Team members from Happy Mushroom, the VFX company that worked on The Mandalorian, sat down with the Corridor Crew for an episode of VFX Artists React to talk about how the virtual collection was created and how it is the future of set design.
We want to highlight a few points from their discussion!
Virtual production according to Happy Mushroom
Happy Mushroom started using game engines as their final image in 2016’s The Jungle Book and continued the same method in The Mandalorian. They used the same huge screen, but without the stage and a team of 12 designers to create the entire movie or show.
The reason they use these huge screens to showcase the virtual environments is that they give off the practical lighting that the DPs want. Traditional lighting that DPs use is still encouraged on sets like these because it is hard to fake shadows and highlights. The virtual team will go in during post-production and add those lights to the scenes to make the environment look as natural as possible.
The process of creating the environments starts with a kickoff meeting where the DP and director sit down with the virtual design team and set-design team to talk about what their vision of each scene looks like. This process includes talking about the mood for the scenes and showing the concept art. After this initial meeting, both teams go off and start building the sets.
If you're a producer and want to know more about virtual production, check out our primer!
BTS on 'The Jungle Book'Credit: Walt Disney Studios Motion Pictures
Unlike real-world scouting that includes a small team figuring out when the right day to shoot is and where to shoot, the Unreal world is designed through VR headsets, and everything is crafted to the DP’s liking. Sometimes, the virtual team would take photos of boxes that the set design team created and base the color scheme and lighting off of how it looked with those boxes.
For the second episode of The Mandalorian season 1, the design team created large rocks that were used as practicals for hiding the camera while filming. The virtual design team scanned those rocks and created an entire environment that matched the model rocks perfectly. The results are amazing.
Instead of having to go into the post-production and create shadows, those shadows are already there. Any reflections, dirt being kicked up, or really anything that can be captured realistically was done in the real world.
The goal of virtual worldbuilding is that you can hit record on the camera and not have to mess with the footage too much afterward. The best part is that the luxury of making small changes is accessible. If the red boxes in the background are too distracting, then the virtual team can go in and change the color of the boxes to whatever color they need to be.
Being able to look at something in real-time is a huge luxury that many people don’t have when filming something on a greenscreen. Nothing is in motion and the whole piece is at risk of feeling too fake.
The goal of creating these virtual sets is to make as little work as possible for post. Everything is already ready to go, and there is no need to spend weeks trying to make one thing work around moving actors. Yes, you are filming something with a completely man-made background, but that doesn’t mean that you have to throw out all of the best practices of filmmaking that have been developed for 100 years.
Ian Hubert's BTS on 'Salad Mug'Credit: Ian Hubert
Virtual production, alone
If you don’t have the luxury of working with huge LED screens that project your world, look to Ian Hubert. He just released a completely CG production that he made by himself for the past three years. He relied on photorealistic copies to make the environment feel as real as possible, matched lighting flawlessly onto his actress and the buildings in his handmade city, and did everything on a tiny stage.
The magic is in Hubert's ability to make something from the photo scans and photo-based textures. Instead of rendering an image from scratch, he had the tools needed to create what he wanted without putting extra stress and time into his project. He states that the hardest part of his whole project was using practical things while filming, and that is why his project took a little over three years to create.
Finding that balance between practical and Unreal is vital at this point in filmmaking. Unreal is perfect for building environments that are out of this world, but practical effects ground that out-of-world experience in a reality that people can understand and relate to. All it takes is a lot of time in a VR headset and pushing record on the camera.
Let us know what you think about this technology in the comments below!
Source: VFX Artists React to Bad & Great CGi 45
