While the AI news just keeps coming in, we have more advancements in the world of AI to share with you. And, just like the last few major announcements, this one’s quite exciting and/or concerning to those in the animation space.

In a partnership between the generative AI platform Pika Labs (which we’ve covered before) and the AI audio platform ElevenLabs (which we’ve also covered before), Pika’s platform will now provide a new Lip Sync feature that will allow creators to give words to their AI-generated characters in their AI-generated videos.

Let’s take a look at how this new technology works, plus check out how it looks so far.

AI-Powered Lip Sync

With AI moving as fast as it is, it’s hard to nail down any definitive statements about the technology and just how useful it might be today or tomorrow. The news of OpenAI’s longer-form generative video platform Sora really blew everything we knew about video AI out of the water, so in the wake, we’re still scrambling to understand what is and is not possible.

However, we’ve known for some time that lip sync has been difficult for generative AI videos to pull off. There are probably a lot of reasons for this, but it’s also always been probable that it would eventually be solved. And that appears to be the case now.

Here are some examples shared by Pika Labs of this new AI technology in action.

AI Lip Sync for Animation

What’s interesting about this is despite all of the lifelike and quite impressive AI videos that we’ve seen so far, we haven’t seen any hard examples of AI-generating scenes with characters talking. Which, as we all know, is a huge part of film and television content. (And, you know, real life as well.)

This new technology, which makes use of Elevenlabs audio AI models in conjunction with Pika Labs video generation, plus throw in some advanced AI-power learning and tracking for human facial movements, looks like the perfect solution to this AI issue.

For now, though, it sounds like this tech will only be available to Pika Lab users who are subscribed to their Pro plan which currently costs $58 a month. However, it’s also quite likely that other AI players will solve this soon as well, and if you’re looking to work in animation (or any type of video) in the future, you might just expect to see AI features like these integrated into your animation and video editing workflows.

From Your Site Articles
pika labsanimation lipsai video toolsai video editingai filmmaking toolsai lip syncingai newsai filmmakinglip matchingai videoai animationlip syncelevenlabsai lip sync
AITopics