But where there's a will, there's a way. A DIY way.
The Volume. It’s a high-resolution behemoth that Disney has been using to create realistic interplanetary backgrounds for their hit TV show The Mandalorian. The company invested millions to create a tool that gives them total control over the background and gives actors a realistic environment to react with. That kind of technology investment is too expensive for regular filmmakers to take advantage of. But is there a way to fake it?
Why yes, there is.
Still Images and a Big-Screen TV
As TV prices go down and screen sizes go up on ultra high-definition 4K displays, the idea of using a high-resolution television screen as a virtual background display becomes viable for indie filmmakers on a budget. Users are even turning to short-throw video projectors and virtual background software like Unreal Engine to provide production value like big-budget projects are doing.
More on that later.
But how does a filmmaker do it, and what displays work best? Here are a few ideas from a few YouTubers who are on the bleeding edge and getting the most bang for their buck.
Credit: Caleb Pike - DSLR Video Shooter
Caleb Pike, aka DSLR Video Shooter, has done some shooting using a simple 55” HiSense 4K LCD TV. He used it as a video table and a virtual background display. Opting to go for an LCD TV with a competitive price of around $300, Pike was able to connect the display to his mobile phone and laptop courtesy of the TV’s built-in Roku and his iPhone’s mirroring feature.
The Roku uses AirPlay to connect to the mobile device and then mirror what the device has on its screen. From there, Pike was able to bring up a still image as the background and then pinch and zoom to get the right angle for the camera he was shooting.
Shooting Around Reflections
Pike’s 55-inch 4K TV did have a highly reflective display, however, and as such, he was forced to consider that when lighting the foreground. Too much directional light would bounce off the TV's reflective surface and into the camera, spoiling the effect. The reflection will also show whatever device is placed on top of it. This could be a cool effect, but it’s definitely one to be aware of.
Pike admits that he used still images for most of his backgrounds when using a TV as a virtual background display. He did create motion graphics when using the TV as a video table. However, the motion graphics were locked down and had no virtual camera movement to achieve parallax.
Credit: Caleb Pike - DSLR Video Shooter
Pike could’ve opted for a TV with an anti-reflective coating, but that had the downside of affecting the display of the TV's black levels, leaving them more washed out. Additionally, the angle at which the camera was placed could affect how those black levels are displayed.
For instance, when shot head-on, the black levels could be deep and rich. But moving the camera to the far side of either angle, outside of the usable viewer angle, the LEDs begin to cause those rich black levels to become grayer.
There’s also an issue with ray tracing and patterns. Depending on the model and quality of the TV one uses as the background display, if zooming up close to the subject and the background display, this could show the pixels and multi-colored rays of those pixels, destroying the illusion.
Therefore, know the limitations of the TV make and model you decide to go with.
Standing Up the TV
Standing up the display, and placing some elements in the foreground, can make your virtual background look even more realistic. The key to selling the whole look is to keep parallax in mind when using the display in the background.
If your camera is aimed downward at your foreground subject, it’s important to match that angle with the background. This can be done with a simple pinch and move on your computer or mobile screen. By contrast, when shooting head-on or at an upward angle, you want to extend that angle in the display background by aiming to gesture and zoom the image upward.
Credit: Caleb Pike - DSLR Video Shooter
Lighting the Display
Lighting is just as important, as it always is when shooting with a virtual background, and Pike says it's important to use a lighting setup that matches the lighting of the virtual background in the display.
An RGB light can help match the background light, dialing in the color, saturation, and brightness to match the background. Doing this will add to selling the background as being realistic since the foreground shares the light as if it’s using the same source.
Credit: Caleb Pike
What Lenses to Use
Pike also recommends using a longer lens when shooting with a displayed virtual background. At least 55mm or longer. The reason to use a longer lens is that a shooter can use lens compression to their advantage. The compression also better hides the display artifacts and offers a greater sense of depth.
Anamorphic works too. Just watch your edges.Credit: Sirui
Additionally, the narrower field of view helps sell the image in the background when the lens is set at its widest for better bokeh. Wide-angle lenses, by contrast, have far too wide a field of view and far less lens compression to sell the effect, and some lens distortion can make the image appear unnatural to the eye. So pinching and zooming into the image can compensate for that.
Moreover, using an aperture set at F2.8 or lower will provide the proper amount of bokeh to make the background image out of focus with depth of field and provide a sense of realism to the human eye.
The Short Throw Projector
Using video in the background is possible but far more difficult. One way around this is by setting up a virtual camera to mimic your main camera.
Ryan Connolly of FilmRiot used this technique and Unreal Engine to create a virtual background that changed with parallax when he moved the camera.
Credit: Ryan Connolly - Film Riot
This required some special equipment provided by NVidia, and therefore, would necessitate an extra cost in equipment and the learning curve that comes with it.
Connolly also used an LG CineBeam 4K Short Throw projector so that he could move around the set without casting a shadow on the background. The short throw projector is right up against the screen and therefore doesn’t have to project the background through the set to the back screen. These projectors can be more expensive, but it opens up a host of camera setup options without getting in the way.
Credit: Ryan Connolly - Film Riot
Final Thoughts
For simple shots which require a basic out-of-focus still image, using a high-resolution 4K TV as a display for a virtual image can create a lot more production value, especially on a tight budget, and it’s worth experimenting with. It's the kind of "faking it" that can be very useful when teaching filmmakers the latest cinematic techniques in cash-strapped community college video programs.
With help from Blackmagic design, one such program has been able to create just such a virtual studio, yielding some marvelous results.
The fact is, the future is here, and thanks to high-resolution 4K displays, independent content creators can begin learning and practicing these kinds of virtual techniques, so when the opportunity comes to use the real thing, it's not foreign to them.
If you have any more advice, leave it in the comments!
Source: DSLR Video Shooter
