Adobe Firefly – A Look Into the Future of Generative AI for Video

You can try to run, you can try to hide, but whether you like it or not, generative AI is here to stay. The only real question now is do you plan to work with generative AI as a way to optimize your creativity and workflows, or do you want to be left behind?

As we’ve seen so far from NAB 2023, AI is indeed one of the biggest trends here in Vegas. As we’ve covered already, Adobe is certainly bringing a lot to the table to explore different AI tools and features.

However, by far the most exciting tool Adobe has unveiled is the first Adobe Firefly model. Let’s take a sneak peek at this AI art generator and how you’ll be able to use everyday language to generate extraordinary new content which has a beta version available today.

The Promises of Generative AI

Alright, before you click the link to dive into trying this new AI art generator yourself, let’s go over a little bit about what Adobe Firefly is and what it promises to do for the industry. Sure, there's a lot to consider and be both excited and concerned about with generative AI.

And to be perfectly frank, for all of those of us who have worked in filmmaking and video editing for years, the idea of an AI program doing any sort of legwork for us is a bit scary and hard to comprehend to a level.

But as we’ve seen with Midjourney and ChatGPT, this bold new world of generative AI is coming whether we like it or not. Yet, despite many of these AI technologies being out for a bit now, it doesn’t quite appear that any of these AI techs are mass-replacing creators. Instead, it's aiding those in the creative space.

Adobe FireflyCredit: Adobe

Generative AI and the Creative Cloud

With that being said, let’s take a look at Adobe’s Beta model for Firefly and how this AI art generator is planned to be integrated into the Creative Cloud.

For just a bit more context though, it’s worth mentioning that Adobe has had AI-driven features in the Creative Cloud and different apps for over 10 years now with their AI and machine learning framework Adobe Sensei. You’ve seen these AI-powered features in Premiere Pro and After Effects with Auto Reframe, Remix, and Content Aware Fill for several years now.

However, Adobe is looking to usher in a major evolution of AI-driven creativity and productivity with Adobe Firefly, their new family of creative generative AI models which will begin with image generation and text effects.

Trained on Adobe Stock images, openly licensed content, and public domain content, this new Adobe Firefly beta is designed to generate unique content which Adobe users can for commercial or personal projects in different Adobe apps (like Premiere Pro and After Effects) here in the near future.

How Adobe Firefly Will Change Video Editing

As video continues to change and heads towards more short-form content for bite-sized consumption, there are certainly some elements of this generative AI movement that could indeed drastically help workflows and creative development.

And that’s the crux of the promise with Adobe Firefly, and how it could become a useful tool for content creators looking to streamline their processes and work quicker, smarter, and, ultimately, more creatively.

From Adobe’s NAB presentation, here are some of the ways they see Firefly working with video editing in particular:

• Text-to-color enhancements: Change color schemes, time of day, or even the seasons in already-recorded videos, instantly altering the mood and setting to evoke a specific tone and feel. With a simple prompt like “Make this scene feel warm and inviting,” the time between imagination and the final product can all but disappear.
• Advanced music and sound effects: Creators can easily generate royalty-free custom sounds and music to reflect a certain feeling or scene for both temporary and final tracks.
• Stunning fonts, text effects, graphics, and logos: With a few simple words and in a matter of minutes, creators can generate subtitles, logos, title cards, and custom contextual animations.
• Powerful script and B-roll capabilities: Creators can dramatically accelerate pre-production, production, and post-production workflows using AI analysis of script-to-text to automatically create storyboards and previsualizations, as well as recommending B-roll clips for rough or final cuts.
• Creative assistants and co-pilots: With personalized generative AI-powered “how-to's,” users can master new skills and accelerate processes from initial vision to creation and editing.

Adobe estimates that they expect to start introducing new generative AI features for video, audio, animation, and motion graphics sometime later this year. If you’re excited to dive in and check things out, you can sign up for the beta of Adobe Firefly and even join the conversation on Discord.

What are your thoughts about Adobe Firefly and generative AI for video? Let us know in the comments below.

No Film School's complete coverage of NAB 2023 is brought to you by Blackmagic Design and Lexar.