The Easiest Way to Rotoscope in 2021

The easiest way to rotoscope in 2021
Harness the power of machine learning and simplify your rotoscoping workflow with Runway ML's Green Screen app.

Rotoscoping has always been a painstakingly slow process, requiring relatively expensive software and technical prowess. While many companies like Adobe and Boris FX are shaking up the roto game up with new tools, these barriers of entry still exist. 

For example, Adobe recently introduced Rotobrush 2 to After Effects, which now harnesses the power of AI and machine learning to make quick work of the roto process. I tested it out just a few months ago with very impressive results, but this tool is still limited to those with an Adobe account and an intermediate proficiency level in After Effects

If only a rotoscoping tool was available to the masses.

Well, a company called Runway ML has been developing a tool called Green Screen, a browser-based application that allows users to upload video clips and automatically roto objects. 

To use, simply click on the object in question, and Green Screen will use machine learning to detect the edges of your object from frame to frame. Skeptical? I was too. So let's take a step-by-step look at how it works.

In this example, I'm going to bring my son's bear to life, and make him walk around. 

*You can use Green Screen for free, but you'll be limited to working with SD clips. A premium membership allows you to work with 4K clips. 

Step 1 - Upload a Clip

For the first step, I'll upload my bear clip, which will add it to my assets. Once imported, I'll drag my clip into the work area window. Now I'm ready to rotoscope.

One feature that makes this tool so accessible is that all of the roto work is done within the cloud, which means I don't have to worry if my system is able to handle the roto workload. The user interface is also very intuitive. With nothing more than a basic timeline and a few buttons, it's really easy to find your way around.  

The Easiest Way to Roto 2021 - Runway ML Green Screen

Step 2 - Select a Region

Now I'll click on my bear to create an initial selection, which is actually very impressive. I'll make refinements using the Include and Exclude buttons. I can zoom in with the magnifying glass, and move around the window via the hand (keyboard shortcut "H") to investigate the edges up close.

Once I'm happy with my selection, I'll press the spacebar to let the machine do its learning and track the mask for all 120 frames of my four-second clip. 

The Easiest Way to Roto 2021 - Runway ML Green Screen

My bear edge was tracked very well, except for a small area where the color of my hand was blending in with the hue of the bear. To help refine the edge even more, I'll make further Exclude refinements at various locations of the timeline. Each new refinement will add a new keyframe, further helping the machine to learn the edges of the object. I can use the foreground/background button to isolate my edge view. 

Step 3 - Export

Once finished, I'll click on the Export button. I can use different chroma values as the background, import a custom image, or simply choose Only Export Alpha Channel.

I'll select the Alpha option, which isn't really a video clip that contains an actual alpha channel. It's more of a clip you can use as a luma matte, which is what I plan to do with the clip in Adobe Premiere Pro and After Effects. 

The Easiest Way to Roto 2021 - Runway ML Green Screen
The Easiest Way to Roto 2021 - Runway ML Green Screen
These various export options make this a very universal tool. Having the ability to export with a green screen background opens this up to folks using iMovie and other budget-friendly editing applications. 

Step 4 - Final Touches

For my bear walking scene, I actually shot a clean plate to make my composite a little easier. Now I'll import my original bear clip, the clean plate, and my new alpha clip from Runway into Adobe Premiere Pro.

I'll create a new sequence and place the clean plate at the bottom, the original on the second layer, and the alpha on top. Next, I'll go into the Effects panel and place the Track Matte Key effect on my Original clip. In the Effect Controls, I'll set the Track Matte source to the Alpha layer, and switch the composite to Matte Luma. 

The Easiest Way to Roto 2021 - Runway ML Green Screen
Now my bear is walking alone on my porch. Is this cool or just creepy? I don't know...

To make the composite even better, I can drag the original clip on top and create a simple garbage matte to retain the shadows of the bear. Now I've got a seriously good-looking walking bear.  

The Easiest Way to Roto 2021 - Runway ML Green Screen
If you'd like to give Green Screen a try, head on over to Runway's website and sign up for a free account. They have a bunch of other cool tools over there as well.

Let us know what you make!     

Disclosure: Boone Loves Video is a participant in affiliate advertising programs designed to provide means for individuals to earn advertising fees by reviewing and linking to products.

You Might Also Like

Your Comment