Rotoscoping has always been a painstakingly slow process, requiring relatively expensive software and technical prowess. While many companies like Adobe and Boris FX are shaking up the roto game up with new tools, these barriers of entry still exist.
For example, Adobe recently introduced Rotobrush 2 to After Effects, which now harnesses the power of AI and machine learning to make quick work of the roto process. I tested it out just a few months ago with very impressive results, but this tool is still limited to those with an Adobe account and an intermediate proficiency level in After Effects.
If only a rotoscoping tool was available to the masses.
Well, a company called Runway ML has been developing a tool called Green Screen, a browser-based application that allows users to upload video clips and automatically roto objects.
To use, simply click on the object in question, and Green Screen will use machine learning to detect the edges of your object from frame to frame. Skeptical? I was too. So let's take a step-by-step look at how it works.
In this example, I'm going to bring my son's bear to life, and make him walk around.
*You can use Green Screen for free, but you'll be limited to working with SD clips. A premium membership allows you to work with 4K clips.
Step 1 - Upload a Clip
For the first step, I'll upload my bear clip, which will add it to my assets. Once imported, I'll drag my clip into the work area window. Now I'm ready to rotoscope.
One feature that makes this tool so accessible is that all of the roto work is done within the cloud, which means I don't have to worry if my system is able to handle the roto workload. The user interface is also very intuitive. With nothing more than a basic timeline and a few buttons, it's really easy to find your way around.
Step 2 - Select a Region
Now I'll click on my bear to create an initial selection, which is actually very impressive. I'll make refinements using the Include and Exclude buttons. I can zoom in with the magnifying glass, and move around the window via the hand (keyboard shortcut "H") to investigate the edges up close.
Once I'm happy with my selection, I'll press the spacebar to let the machine do its learning and track the mask for all 120 frames of my four-second clip.
My bear edge was tracked very well, except for a small area where the color of my hand was blending in with the hue of the bear. To help refine the edge even more, I'll make further Exclude refinements at various locations of the timeline. Each new refinement will add a new keyframe, further helping the machine to learn the edges of the object. I can use the foreground/background button to isolate my edge view.
Step 3 - Export
Once finished, I'll click on the Export button. I can use different chroma values as the background, import a custom image, or simply choose Only Export Alpha Channel.
I'll select the Alpha option, which isn't really a video clip that contains an actual alpha channel. It's more of a clip you can use as a luma matte, which is what I plan to do with the clip in Adobe Premiere Pro and After Effects.
Step 4 - Final Touches
For my bear walking scene, I actually shot a clean plate to make my composite a little easier. Now I'll import my original bear clip, the clean plate, and my new alpha clip from Runway into Adobe Premiere Pro.
I'll create a new sequence and place the clean plate at the bottom, the original on the second layer, and the alpha on top. Next, I'll go into the Effects panel and place the Track Matte Key effect on my Original clip. In the Effect Controls, I'll set the Track Matte source to the Alpha layer, and switch the composite to Matte Luma.
To make the composite even better, I can drag the original clip on top and create a simple garbage matte to retain the shadows of the bear. Now I've got a seriously good-looking walking bear.
Let us know what you make!
Disclosure: Boone Loves Video is a participant in affiliate advertising programs designed to provide means for individuals to earn advertising fees by reviewing and linking to products.
