How To Use Runway AI for Video Editing

As we’ve been covering for the past several months, the world of artificial intelligence is moving fast. Scarily fast. In many ways, Runway, the AI startup which helped develop the AI image generator Stable Diffusion, is one of the older brands in the game.

Yet, already with a mobile app and Gen-2 on the way, this AI-powered video editing software has proven to be one of the bigger voices in this space and is certainly one of the most popular options to explore for those looking to dive into the world of AI editing today.

So, let’s take a look at this web-based video editor to explore how you can use AI to help with a variety of features, including frame interpolation, background removal, effects blur, and other tools for cleaning audio and motion tracking.

Here's what you need to know about getting started with Runway AICredit: Runway

Getting Started with Runway AI

Launched all the way back in 2018 (you know, the prehistoric days of AI), Runway has been making inroads in the world of both amateur and professional video editing for several years now. In fact, Runway tools have even been used on Oscar-winning films like Everything Everywhere All at Once as one of the tools used by the film’s visual effects team.

As far as ambitious AI programs go, Runway is actually one of the easiest to get started with. All you have to do is go to the Runway ML website and sign up for free. There are paid account versions, but you can ignore those and try the free version to start.

From there, it’s as easy as logging in and starting a new project of your own.

AI Magic Tools

Now, once you’re up in running, there’s actually quite a lot to explore.

While you can choose to start a project from scratch, the majority of users are going to want to begin with the AI magic tools, as that’s where all of the fun stuff will be found. In particular, the tools you might want to check out first would include:

• Image to Image
• Text to Image
• Infinite Image
• Train your own Generator
• Video to Video (Gen-1)

These magic tools are going to be your base playthings for getting started with AI and creating your own compositions. The first few are pretty self-explanatory as image-to-image and text-to-image are exactly what they sound like. You’ve seen these in action, perhaps with MidJourney, DALL-E, or the like.

The Infinite Image tool can help you expand an existing image with context-aware Text to Image generation creating new layers and some (sometimes insane) results, whereas the "Train Your Own Generator" tool can help you create your own portrait, animal, or style generator for Text to Image results as well.

Editing Tools

All of the above are obviously more about generative AI and having fun with new AI creations. What might be more exciting though to many curious video editors has to do with the myriad of editing tools housed in Runway ML. These tools can do everything from frame interpolation to creating super slow motion to even generating transcripts and subtitles.

Here are some of the fun tools worth checking out for video editing:

• Expand Image
• Frame Interpolation
• Erase and Replace
• Background Remix
• Image Variation
• Text to 3D Texture
• Text to Color Grade (LUT)
• Inpainting
• Super-Slow Motion
• Blur Faces
• Add Bokeh
• Scene Detection
• Extract Depth
• Clean Audio
• Remove Silence
• Generate Transcript
• Generate Subtitles
• Colorize
• Upscale Image
• Replace Background
• Blur Background
• Export Alpha Matte
• Add Green Background
• Motion Tracking

This is a huge list of tools, and they’re all quite functional and powerful in their own right. In particular, the transcription generation tools and audio cleaning ones might be the most useful for those in a pinch, whereas the erase and replace and generative background remix controls might be the most ambitious for the future.

When to Upgrade

Ultimately the best way to get started with Runway AI is to simply dive in. However, as you’ll quickly discover, Runway ML operates on a credit system that only allocates 125 credits to those trying the app for free. You can then upgrade to Standard or Pro to get more credits per month with the option to purchase more as needed.

The Standard Plan, in particular, isn’t too bad of an option if you feel like you’re going to actually use some of these tools in your day-to-day editing life, and the Pro option might be worth it if you want your whole team to integrate today.

The different plans for Runway AICredit: Runway

However, if you want to simply keep tabs on this new AI and explore some of its basic generative possibilities, there’s nothing wrong with signing up and trying it out for free for now, then circling back later as your workflows need to advance.

How do you feel about these new AI tools like Runway ML? Let us know your thoughts in the comments below.