The Results of the NFS AI vs. Human Screenwriting Challenge
Let's unpack the implications and lessons from our different screenwriting approaches.
If you keep up regularly with No Film School, you know that last week, we were issued a challenge. You can read all about it here, but the basic gist was that a writer who uses AI to create pages said they would challenge us to a ten-page write-off.
They claimed AI would produce better pages, and I felt optimistic I would take home the gold. I took up this valiant challenge, mostly because, like Marty McFly, no one calls me "Yellow."
Also, initially, I thought I would be combating AI writing its own script, but it turned out that it was humans who used AI to brainstorm and then rewrote AI pages, which is different.
But still, I thought it was worth it to suss all this stuff out and see what happens.Thence forth, we entered the short-form writing Thunderdome, let the public vote, and now it's time to unpack what we learned.
Right up top, I'll tell you that I, Jason Hellerman, wrote script A. And Matt Allen and Krista Suh plus AI wrote script B
So, let's dive in and chat about all this stuff.
The Results
As you can see, we had a pretty close chase here. In the end, script A took the better screenplay award, but, for some reason, people thought AI wrote it, and not me.
I'll try not to be insulted. But more on that later.
Lastly, we had a basic 50/50 split on which script people would want to keep reading.
I was hoping we would get over 1,000 votes for each category, but as you can see the totals for each poll averaged around 500. Overall, social media did its job: get the pages in front of the public.
Also, we had a small issue with bots, but we don't think it swayed anything too much, but it would be impossible to tell without real numbers from X, which we will never get. It was mostly comments that eventually got deleted, or accounts with 0 tweets and 0 followers interacting with the post. We have no way of knowing if they voted, which sucks. But internally, we all thought it was worth mentioning.
Again, we don't think it mattered or drastically affected the outcome, but since we saw so many and had to manually block or delete them, we're telling you here out of transparency.
Honestly, I think if you post something about AI, bots just find it on X.
Okay, now it's time to dive into the actual process of how the pages of each screenplay were produced.
The AI Writers Speak
Since I'm not an AI expert, I am extremely happy that Matt and Krista made this video detailing how they wrote their pages.
In short, they use a combo of ChatGPT and Claude 3.5. And Matt built a GPT and a "Project" for each system they use to do the writing.
They then prompt it and then rewrite pages it produces.
After talking to Matt post contest, he had this to say about the process:
"For me, AI is just like having a tireless writing partner that's always willing to bounce ideas off of, and then generate pages at your discretion. Yes, we need to rewrite those pages, but the speed we can write now is irrefutable. And with No Film School's readers essentially split on which pages are AI/human versus human only, I'm guessing most producers in Hollywood will opt for speed—especially for the first draft."
My Thoughts
I, Robot
As a proud WGA writer, I was a little nervous about platforming this AI stuff. We got a few complaints from readers, and I feel personally responsible for that. So I wanted to address them here to start.
I'm with you, I think AI sucks and makes us lazier and less inventive. And I want to thank the WGA for protecting writers in general from this kind of stuff.
But AI is out there, and aside from my ego and wanting to jump into the fray to fight for that, I also think NFS has an obligation to look into these things, especially when people in film and TV are using them.
I will be honest, the topic also generates a lot of clicks and social interaction. As a free, open source website... we need that*. If you ever have concerns, just reach out to me Jason@nofilmschool.com and we can chat about it.
*(editor's note: we also have a massive library great filmmaking content you can read as well!)
As our resident screenwriter, that obligation of coverage falls on me. And it's time to focus on my thoughts on this whole thing.
I genuinely know I will never use AI to write screenplays; it just doesn't make sense for me. And this contest legitimately didn't sell me on trying it. I have my process, and I trust it.
Basically, I looked at what our AI screenwriters did, and see two pro writers who had to extensively rewrite what a computer spit out in order to meet a self-imposed deadline of speed, and it looks messy to me.
On the surface level, all AI gave them was a speedier prompt, but it also came with a whole new set of limitations. In a world where they weren't doing a timed challenge, why would you not just talk to your writing partner about ideas? Or talk to real human beings about their life experiences? Why even consult a computer?
Or why not just get ideas yourself and then litmus test them by putting them onto the page and seeing how they flow? What if you hate all their scene ideas? Do you just keep prompting it? Why not think of a scene you like yourself?
That's the job of any storyteller in any medium.
And in real life, the only time you really face a time limit as a writer is when you're being paid for an assignment.
For those who don't know, if we were in a setting where an exec hired us to write a feature film for a company, we would have three months to hand in the script.
When I get paid to write, I don't care how long I spend on a particular set of pages. I just care that they're good. Some scenes I nail right away. Others take finesse. I'm incredibly confident in my writing speed overall because I have done this for over a decade and I know what I need to hit my deadlines.
My process is very, very simple: I brainstorm and write ideas with a pen in a Moleskin. Then I type a really detailed outline once I decide on which of those ideas are good. Then I have that open on one side of my screen and Final Draft on the other. From there I simply go back and forth, and write!
In a little over a month, I'll have a first draft of a feature. And then, if I'm on assignment, I spend the rest of the time before the deadline trying new things, getting notes from my rep and friends, and perfecting what's on the page.
So, when it comes to the AI argument that execs want speed... I actually don't think that's true. They want "great and ready to shoot."
Outside of this challenge, I would never show an exec ten pages of an idea. That's not how Hollywood works. You're either writing a spec, which you shouldn't show to anyone with buying power until you think it's 'finished' or you're on assignment and being paid to write, and therefore you should finish it and then send it in—preferably after a few drafts—because you want them to get something good.
Even in conversations with execs, even if we were 'cracking it together,' I would only ever just pitch them an idea. Or see if they would buy a treatment. There's no benefit to showing pages early. Zero. They're either into the idea and excited to read the whole thing, or they're not into the idea.
Again, even if you were under contract, you maybe would hand in the project early and get their notes. Regardless of all that, I was given a week to complete my ten page challenge.
So...what did I do with all that time?
Basically, I clocked in here to work and wrote a bunch of articles, turned in an assignment for producers, and worked on my latest spec. When it came to the ten pages I produced here, I did that in my spare time.
And in my spare time, I went right to ideation. We were given a logline generated by Google Gemini. The logline was: "A burnt-out bodyguard and a naive, fame-obsessed pop star must team up to take down a ruthless crime syndicate after accidentally witnessing a murder, leading to a series of chaotic mishaps and unlikely friendships."
I had to find a personal way into this story, so I just took a walk. I smoked a cigar at the V Cut on Melrose. I had a great IPA at All Seasons Brewing. I lived my life because a lot of writing is just being a human being and having some real-life experiences—not sitting and chatting with a bot.
And guess what I heard about?
While chatting in the cigar shop, I spoke to this lawyer who was telling me about feeling responsible for one of his younger associates. That sparked an idea I really liked. One with actual emotions I understood.
Once I had my idea, I wrote an outline with scene ideas and then I opened my screenwriting software. I ran a timer while writing (thanks, Write Sprint on Final Draft), and I spent around three hours getting to those ten pages you saw. If you want the exact time, it was three hours and twenty four minutes. That doesn't count my drinking and smoking.
I knocked out the preliminary draft of 10 in under ninety minutes, because I did an outline and knew what my scenes would be. The next two hours I spent refining, and adding my voice into the pages. And doing some notes a few friends sent over after I shared an early draft with them.
As an aside, it actually hurts me deeply that more people thought script A was AI, because I think I am really unique in terms of my voice on the page. Real ego shot there. Also, I legit thought 'Jessie Jablonski' was a really funny name. Surprised to see so many people bump on it.
Regardless of that, did I learn anything in this process?
Not really.
I mean, I never want to spend a couple hours on pages and put them on the internet for people to pick apart again. I read all the comments, even when my wife told me not to, and the only ones I can remember are the mean ones, even though the nice ones outnumbered them.
But at the end of the day, all I gained was more confidence that AI is a waste of time. Just brainstorm and live a full life! That's the job!
Look, if you like AI brainstorming, good for you. But... are you actually telling a story humans will connect to if the pieces are assembled by a robot?
I think the biggest thing proven in this whole thing is that you have to rewrite so much of the ideas AI spits out that it's not any faster than actually writing it down when you get an idea.
And is it more pleasurable than a beer, a conversation, and a cigar?
I doubt it.
For me, AI is the tool I would use for lazy work. If I need to make some sort of excel sheet or a giant list, or just a bunch of bulleted ideas that would take me a long time to type, I would use AI.
No part of me believes that AI could foster creativity, because it just regurgitates what it finds online. So you're basically getting the bologna of ideas. It's all the parts of the icky pig, but there's also the process that can be as fulfilling as a nice juicy steakhouse sirloin.
The times I have used AI it's been because I'm not doing a creative endeavor. And it's been because I have real shit I want to do, and I'm phoning in something else.
It would never be a factor in my creative process because it's the one thing in my world I take completely seriously as almost a vocation. My calling is telling human stories. When I sign on to write anything, it's because I truly believe in the idea.
Even for this dumb challenge, I took the convo from the V and decided I wanted to tell the story of a pop singer who thinks she's all alone, only to learn she had a bodyguard who may actually be her family/father figure there all the time. And for the bodyguard, I wanted to tell the story of a guy who thinks he's also alone, only to find out he may have a surrogate daughter in the pop star.
I decided that was the right idea because I knew the cliche one would probably be the romance between the two. I've seen that before in some many things. And I didn't want to see it again. I wanted to see something I believed in.
Something that I manifested through hard work.
Something that I knew other humans were feeling and that I could empathize with as I typed the pages.
To be original, truly original, you have to look into your own soul and decide what you believe about the world. You have to feel like you're naked on the page, and the world is staring back at you.
Writing with AI is writing in a snowsuit. That's not you. That's zeroes and ones with aggregated data scooped on the bowels of the internet.
You also need to take the necessary time to build, to tear down, and to rebuild your ideas. Not give things way or promise to be speedy just to get the attention of an exec. You have to take the shots that are the difficult notes, and you have to come out the other side only having made your story stronger.
AI is never going to make me better at any of that. And I don't think it would make you better, either. Because, at the end of the day, is that writing really you?
Is it really what you have to say?
My best writing comes from me not letting myself get lazy. And, man, that can be a struggle. I want things now. I want success and fame and enough money to buy a new car and to get out of my shitty apartment now.
But you know when I feel the best? When I sit with an exec or talent or a director and they tell me the story of why they connected with my work. That shit has kept be going for over a decade, and kept me and the Hellermaniacs corporation in business, too.
I am content with the success I have in Hollywood right now, and excited about my future in this industry. It's a marathon. And I'm not sprinting, I'm keeping my pace.
I am still learning things about writing every single day. Not from a computer—I'm learning from the people I meet in generals, from the scripts I get sent from friends, and from the assignments I get paid to write.
All of that is making me better at my own speed. AI is just a temptation to get lazy.
My next challenge is to you, along with myself.
Don't get lazy, friends. Why cheat yourself? Why cheat the world from what your voice and have to say with it?
Go at your own pace and tell the stories that are going to change the world. Show me that thing in your heart you needed to get out.
Entertain me. Force me to have a human reaction.
I'm rooting for you and for everyone doing their heart's work.
As always, let me know what you think in the comments.
- Is the Royal "We" a Screenwriting Sin? Do Any Screenwriting Rules Matter? ›
- I Got My Black List Script Rated By AI … And This Is What It Scored ›
- Why the Basketball Game from 'The Office' is Genius Writing ›
- We Challenged An AI Screenwriter to See Who Could Write A Better First Ten Pages ›