October 13, 2016

How IBM's Watson Used AI to Edit the Trailer for Upcoming AI Thriller 'Morgan'

IBM's Watson computer system, famous for winning Jeopardy, is joining the filmmaking business.

There's an old story about Fellini that, when asked why there wasn't a great film school in Italy to train the next generation of filmmakers, he responded, "Are you mad? We should be stabbing them in their cribs! They'll take all of our jobs!" Today, the Italian auteur would be unplugging computers and taking a hammer to hard drives, since for the first time at the studio level, artificial intelligence has been involved in editorial decision making.

We've had some level of automation coming to film for a while now, with limited success. There are movies being written with artificial intelligence. Auto-focus and auto-exposure tools were very blunt and not useable professionally early on, but now we're starting to see more and more sophisticated tools both at the consumer level and on the high end. Post has had some automation as well, with services like animoto offering automated edits. However, the results are unpredictable, and—as anyone who has tried the "auto-balance" color correcting button in their edit software has realized—not quite ready for prime time. Nonetheless, as they were developing the marketing for the AI horror/thriller Morgan, 20th Fox went to IBM to see if high-end artificial intelligence was able to create more useful results when it came to editing their trailer.

Using machine learning, Watson modeled the scenes visually and emotionally, then analyzed hundreds of other movie trailers for other films in the same genre, in order to learn relevant lessons to create the trailer. Watson didn't have final cut on the trailer, however, and functioned more for organizing scenes and moments that would then be analyzed and polished by a final editor. The computer selected ten scenes that it felt were important for inclusion in trailer, only one of which was left on the cutting room floor.

The trailer making process is a laborious one, and studios regularly test dozens of different edits of the trailer on audiences to see which one is most effective. While it doesn't seem likely that we will be watching many entirely AI created movies soon, there is a real world application in this for marketers looking to create a wide variety of edits of a trailer in a short period of time. It took Watson about 24 hours to properly evaluate the 90-minute film and shave it down to six minutes of selects. With even further development, it's likely we'll see AI move into evaluating all the footage shot, including out-takes, deleted scenes and alternate performances, to create the most effective trailer possible.

WatsonCredit: 20th Century Fox

It's also likely we'll start to see some form of AI in the edit room in the near future, helping editors get a handle on the massive amounts of footage shot, especially on documentary projects. An AI that can learn what is engaging, out of the ordinary, or emotionally evocative could help tremendously in getting footage down to a manageable level for an editor to work with.

Of course, if you don't trust the AI, they might hide all the best footage from you and keep it for themselves. After all, we hear Watson's real ambition is to direct. How soon do you think AI is going to take over the world of trailer editing?     

For more see the IBM blog.

Your Comment


...the fuck?

October 13, 2016 at 5:56PM, Edited October 13, 5:56PM

Terma Louis
Photographer / Cinematographer / Editor

So, a computer helped to analyze footage.. That's not even close to editing the trailer.

October 13, 2016 at 7:11PM


things people do for click-bait! :(

October 14, 2016 at 3:19AM

Amrit Vatsa
Documentary Filmmaker

If you read the title "How IBM's Watson Used AI to Edit the Trailer for Upcoming AI Thriller 'Morgan'" The question is answered. They used it to pull selects for the edit. Thats not clickbait. Its a genuinely interesting application of using AI in its rudimentary form. Granted they could have worded the title better, it still doesn't mislead on the whole. Also this is fairly old news. Now that I think about it the first time I heard they were using AI to edit the trailer and found out it didn't actually edit the trailer I was pissed off too. So your feelings in the end are probably valid on what just took place in this FREE BLOG SITE!

October 14, 2016 at 10:42AM

Seth Evans

Honestly, I think this person just wanted to say "click-bait."

October 16, 2016 at 5:36PM

Russell Anderson
Editor, Programmer

I can see computer assisted editing possibly help in the future. The problem is there's already enough distrust of people's human assistant editors ("are you SURE you got every shot? This is every moment where the lead said "We're gonna need a bigger boat?") Production can't even come up with a system for keeping scenes and takes coordinated (script supervisor, VTR op, DIT, 2nd AC, sound... all independently tracking scene and take, and editorial STILL has to write it in manually?)

I don't think we're going to see this anytime soon. Probably not for decades, if that. The one place it could be useful is helping with actors (face recognition) and framing (wide, medium, close-up, etc). But speech recognition tech has been (and will likely be for a while, if technical studies into it are any indication) very unreliable in this kind of scenario. Let alone determining continuity, performance, tone and delivery...

I'm all for crazy advancement in tech, even it means job elimination but I don't see this happening in any effective away.

October 16, 2016 at 5:44PM

Russell Anderson
Editor, Programmer