How To Use Runway’s New Director Mode and AI Custom Camera Controls

Runway Gen-2 Director Mode

Probably the most interesting new feature added to Runway’s AI offering in this version Gen-2, Runway’s Director Mode unlocks several new custom camera controls that add directional movement. These movements cover horizontal, vertical, zoom, and even a roll control feature. After generating your director mode control, you can then choose to either extend it further or add a different one.

While still pretty rudimentary in terms of actual direction and camera movement, this is indeed a major step forward for generative AI as it allows for much easier and quicker controls of these basic camera moves.

Here’s a summary of how to use this feature in the Runway app:

• Add an image to Gen-2, click on the “Motion” button, and click “Custom camera control”
• Choose one or multiple directions to move your camera
• Then, adjust “Speed” for your motion
• Clicking the “Extend” button will reuse the existing camera direction for your extensions to continue the motion seamlessly, up to 16s
• For even more control, try using the Image + Description feature, which will allow you to dial in a more specific prompt

Runway Gen-2 Directional Looping Video

On top of this new director mode in Runway, the AI app has also unveiled a directional looping video feature which is a quick trick to use to turn your short AI-generated clips into longer, looping videos, which you can then export either as a video file or as a GIF.

Here’s a summary of how to use this feature in the Runway app:

• Add an image to Gen-2
• Navigate to the advanced camera controls section to use Director Mode
• Turn on Upscale
• Choose a camera path and generate a video for each direction that you want to present in your video
• Pro tip: the next step is easier if you rename outputs with keywords or directions
• When editing video files, play the clip forwards; when it reaches the end, duplicate the video and play it backwards, so that it resets back to the first frame of the video
• Play with keyframes to make things feel more dynamic and snappy
• For even more loop-ability, try exporting your video as a GIF

Runway Gen-2 Motion Slider

There’s also another new feature called motion slider which allows users to control the length and intensity of the motion in their videos.

Here’s a summary of how to use this feature in the Runway app:

• Open Gen-2 and input a prompt
• Increase or decrease motion value by typing or dragging the slider
• The default is 5; the range is 0-10 (0 = less motion, 10 = more motion)
• Click Generate

Runway Gen-2 Workflow w/ AI Magic Tools

Finally, Runway has a small array of other AI magic tools that, when combined, can allow users to create generated videos that have a range of camera moves and frame interpolation.

Here’s a summary of how to use this feature in the Runway app:

• Start with Text to Image
  • generate an image you want to bring to life with Gen-2, then add it to your assets
• Open Gen-2, and use that image as your reference image alongside a text prompt
• Try bringing your video into Super-Slow Motion to extend the clip time
• Or drop your video into Text to Color Grade (LUT) to give it a color treatment
• Bring your video into Inpainting to clean up any artifacts or parts of the frame
• Toss it into Green Screen to extract your creation from the scene or create alpha mattes
• The possibilities are endless with 30 AI Magic Tools!

And that’s pretty much it, at least until a new Runway Gen-3 comes out with what could undoubtedly be more advanced features. For now, these tools are a nice introduction to generative AI and a safe way to explore how AI can be used for video or filmmaking.